#66th of 12 in Training and fine-tuning
peft
Python library for LoRA and other parameter-efficient fine-tuning of Transformers models
Live demo ↗
(opens in a new tab)Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 21.8k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- Python
Missed the latest GitHub check; these facts may be older.
Overview
PEFT wraps a pretrained model with adapter methods such as LoRA, soft prompts and IA3 so only a small set of extra parameters is trained. Adapters are saved as small checkpoints (19MB versus 11GB for T0_3B in the README's example). It integrates with Transformers, Diffusers, Accelerate and TRL, and can be combined with quantization such as QLoRA.
Who it is for: ML engineers fine-tuning LLMs and diffusion models on limited GPU memory
Strengths
- Adapter checkpoints are MBs instead of GBs; 19MB vs 11GB for T0_3B
- LoRA trained a 12B model (mt0-xxl) on one 80GB GPU where full finetuning ran out of memory
- Integrates with Transformers, Diffusers, Accelerate and TRL; works with quantized base models
- Install with pip; get_peft_model wraps an existing model in a few lines
Weaknesses
- Library only: no training UI, scheduler or serving; training loop is your own or Trainer
- Transformers integration lacks some PEFT features, such as merging adapters into the base model
- Models without official support need manual config for target modules
- Memory figures in the README assume an A100 80GB with over 64GB CPU RAM
What it needs
- GPU optional
- Docker
- Needs PyTorch, Transformers
- Models: LoRA, Soft prompts, IA3, QLoRA, Transformers models
Also in Training and fine-tuning
See all 12| Rank | Project | Score |
|---|---|---|
| 1 | unslothDesktop and web app to run, fine-tune and export open models | 88 out of 100 |
| 2 | deepspeedPyTorch library for distributed training of large models with ZeRO | 79 out of 100 |
| 3 | llamafactoryFine-tuning toolkit for 100+ LLMs with a CLI and Gradio web UI | 74 out of 100 |
| #4 | verlReinforcement learning training library for LLM post-training with PPO and GRPO | 68 out of 100 |
| #5 | minimindTrain a 64M-parameter LLM from scratch in plain PyTorch | 66 out of 100 |