#88th of 12 in Training and fine-tuning
trl
Post-training library with SFT, DPO, GRPO and KTO trainers
- Stars
- 19.5k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- Python
Missed the latest GitHub check; these facts may be older.
Overview
TRL provides trainer classes (SFTTrainer, GRPOTrainer, DPOTrainer, KTOTrainer, RewardTrainer) that wrap the Hugging Face Transformers trainer for post-training language models. It scales through Accelerate with DDP, DeepSpeed ZeRO and FSDP, supports LoRA/QLoRA via PEFT, and ships a `trl` CLI for running SFT, DPO and KTO without code.
Who it is for: ML engineers fine-tuning LLMs with SFT, preference or RL methods
Strengths
- Covers SFT, DPO, GRPO, KTO and reward modeling with one trainer API
- Scales from one GPU to multi-node via Accelerate, DDP, DeepSpeed and FSDP
- Integrates PEFT (LoRA/QLoRA) and Unsloth for training on modest hardware
- CLI commands (trl sft, dpo, kto) run fine-tuning without writing code
Weaknesses
- Code-first Python library with no web UI
- Features under trl.experimental may change or be removed without notice
- Built around the Hugging Face Transformers ecosystem
- README gives no RAM or VRAM requirements
What it needs
- GPU optional
- Needs transformers, accelerate, peft, datasets, deepspeed, unsloth
- Models: Hugging Face Transformers models, LoRA/QLoRA adapters, AWS Trainium (TorchNeuron)
Also in Training and fine-tuning
See all 12| Rank | Project | Score |
|---|---|---|
| 1 | unslothDesktop and web app to run, fine-tune and export open models | 88 out of 100 |
| 2 | deepspeedPyTorch library for distributed training of large models with ZeRO | 79 out of 100 |
| 3 | llamafactoryFine-tuning toolkit for 100+ LLMs with a CLI and Gradio web UI | 74 out of 100 |
| #4 | verlReinforcement learning training library for LLM post-training with PPO and GRPO | 68 out of 100 |
| #5 | minimindTrain a 64M-parameter LLM from scratch in plain PyTorch | 66 out of 100 |