#4040th of 53 in Training and fine-tuning
alignment-handbook
Training scripts and YAML recipes for SFT, DPO and ORPO of LLMs
Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 5.7k
- License
- Apache-2.0
- Last commit
- Sep 2026
- Language
- Python
5,549 on 21 Apr 20265,712 on 10 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Hugging Face's reference repo for post-training language models. It provides scripts for continued pretraining, SFT, DPO and ORPO, plus YAML recipes that reproduce models such as Zephyr 7B, StarChat2, SmolLM2 and SmolLM3. Each script supports full-weight training with DeepSpeed ZeRO-3 or LoRA/QLoRA.
Who it is for: ML engineers fine-tuning and preference-aligning open LLMs on their own GPUs
Strengths
- One YAML file per training run, so published models can be reproduced
- Covers continued pretraining, SFT, DPO and ORPO, with full-weight (ZeRO-3) or LoRA/QLoRA
- Recipes for Zephyr, StarChat2, SmolLM2, SmolLM3 and a Constitutional AI setup
- Apache-2.0 licensed, maintained by Hugging Face
Weaknesses
- Pins PyTorch 2.6.0 and flash-attn 2.7.4.post1, so setup is tied to specific CUDA builds
- Requires Flash Attention 2, which needs a compatible NVIDIA GPU
- Reference recipes, not a general trainer; guides on DPO and preferences are still in progress
- No Docker image or release; installed from source with uv
What it needs
- GPU required
- Needs PyTorch 2.6.0, Flash Attention 2 (2.7.4.post1), DeepSpeed, Hugging Face Hub account, Git LFS
- Models: Zephyr 7B, Gemma 7B, Mixtral 8x22B, SmolLM / SmolLM2 / SmolLM3, StarChat2 15B
Quick answers
Is alignment-handbook open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Does alignment-handbook need a GPU?
Yes, it needs a GPU.
Also in Training and fine-tuning
See all 53| Rank | Project | Score |
|---|---|---|
| 1 | unslothDesktop and web app to run and fine-tune local models | 88 out of 100 |
| 2 | peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models | 88 out of 100 |
| 3 | deepspeedPyTorch library for distributed training of very large models with ZeRO | 86 out of 100 |
| #4 | verlReinforcement learning training library for LLM post-training with PPO and GRPO | 86 out of 100 |
| #5 | trlPost-training library with SFT, DPO, GRPO and KTO trainers | 86 out of 100 |