peft vs trl
Two of the top projects in Training and fine-tuning, side by side: who leads on each score part and who each one suits, then setup, license, activity and what each review found.
peft
Python library for LoRA and other parameter-efficient fine-tuning of Transformers models
trl
Post-training library with SFT, DPO, GRPO and KTO trainers
Verdict
peft ranks higher, 89 to 87. Of the 4 score parts we measure for both, peft leads on 2, trl on none and 2 are even.
- Adoptionpeft leads by 4 points, 81 to 77: close to even.
- FreshnessEven at 100.
- Maintenancepeft leads by 2 points, 91 to 89: close to even.
- Agent-readyEven at 70.
Pick peft if you are
ML engineers fine-tuning LLMs and diffusion models on limited GPU memory
- Adapter checkpoints are MBs instead of GBs; 19MB vs 11GB for T0_3B
- LoRA trained a 12B model (mt0-xxl) on one 80GB GPU where full finetuning ran out of memory
Pick trl if you are
ML engineers fine-tuning LLMs with SFT, preference or RL methods
- Covers SFT, DPO, GRPO, KTO and reward modeling with one trainer API
- Scales from one GPU to multi-node via Accelerate, DDP, DeepSpeed and FSDP
| What we compare | peft | trl |
|---|---|---|
| Score parts, out of 100 | ||
| Adoption | 81, widely used | 77, popular |
| Freshness | 100, active | 100, active |
| Maintenance | 91, healthy | 89, healthy |
| Agent-ready | 70, partly | 70, partly |
| Facts from GitHub and the README | ||
| Stars | 21.9k | 19.7k |
| License | Apache-2.0 (permissive) | Apache-2.0 (permissive) |
| Last commit | Nov 2026 | Nov 2026 |
| Last release | Oct 2026 | Oct 2026 |
| Language | Python | Python |
| Docker | Yes | Yes |
| GPU | Optional | Optional |
| arm64 or Apple Silicon | Mentioned | Not stated |
peft
PEFT wraps a pretrained model with adapter methods such as LoRA, soft prompts and IA3 so only a small set of extra parameters is trained. Adapters are saved as small checkpoints (19MB versus 11GB for T0_3B in the README's example). It integrates with Transformers, Diffusers, Accelerate and TRL, and can be combined with quantization such as QLoRA.
Who it is for: ML engineers fine-tuning LLMs and diffusion models on limited GPU memory
Strengths
- Adapter checkpoints are MBs instead of GBs; 19MB vs 11GB for T0_3B
- LoRA trained a 12B model (mt0-xxl) on one 80GB GPU where full finetuning ran out of memory
- Integrates with Transformers, Diffusers, Accelerate and TRL; works with quantized base models
- Install with pip; get_peft_model wraps an existing model in a few lines
Weaknesses
- Library only: no training UI, scheduler or serving; training loop is your own or Trainer
- Transformers integration lacks some PEFT features, such as merging adapters into the base model
- Models without official support need manual config for target modules
- Memory figures in the README assume an A100 80GB with over 64GB CPU RAM
- GPU optional
- Docker
- Needs PyTorch, Transformers
- Models: LoRA, Soft prompts, IA3, QLoRA, Transformers models
trl
TRL provides trainer classes (SFTTrainer, GRPOTrainer, DPOTrainer, KTOTrainer, RewardTrainer) that wrap the Hugging Face Transformers trainer for post-training language models. It scales through Accelerate with DDP, DeepSpeed ZeRO and FSDP, supports LoRA/QLoRA via PEFT, and ships a `trl` CLI for running SFT, DPO and KTO without code.
Who it is for: ML engineers fine-tuning LLMs with SFT, preference or RL methods
Strengths
- Covers SFT, DPO, GRPO, KTO and reward modeling with one trainer API
- Scales from one GPU to multi-node via Accelerate, DDP, DeepSpeed and FSDP
- Integrates PEFT (LoRA/QLoRA) and Unsloth for training on modest hardware
- CLI commands (trl sft, dpo, kto) run fine-tuning without writing code
Weaknesses
- Code-first Python library with no web UI
- Features under trl.experimental may change or be removed without notice
- Built around the Hugging Face Transformers ecosystem
- README gives no RAM or VRAM requirements
- GPU optional
- Docker
- Needs transformers, accelerate, peft, datasets, deepspeed, unsloth
- Models: Hugging Face Transformers models, LoRA/QLoRA adapters, AWS Trainium (TorchNeuron)
More in Training and fine-tuning
Building with these?
We help teams pick an open-source AI stack and run it in production. Client work never changes a rank.