11st of 2 in Training and fine-tuning
llamafactory
Fine-tuning toolkit for 100+ LLMs with a CLI and Gradio web UI
Live demo ↗
(opens in a new tab)Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 75.4k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- May 2026
- Language
- Python
Checked on GitHub.
Overview
LlamaFactory fine-tunes language and multimodal models through YAML-configured CLI runs or the LLaMA Board Gradio UI. It covers pre-training, SFT, reward modeling, PPO, DPO, KTO and ORPO, with full-tuning, freeze-tuning, LoRA and 2 to 8-bit QLoRA. Trained models can be served through an OpenAI-style API using vLLM or SGLang workers.
Who it is for: Engineers fine-tuning open LLMs on their own GPUs, with or without code
Strengths
- Broad method coverage: SFT, PPO, DPO, KTO, ORPO, LoRA, QLoRA via AQLM/AWQ/GPTQ/HQQ/EETQ
- Supports GaLore, BAdam, APOLLO, Muon, DoRA, PiSSA, plus Unsloth and Liger Kernel speedups
- Covers multimodal and audio models such as Qwen3-VL, LLaVA and Qwen2-Audio
- Runs on NVIDIA, AMD ROCm and Ascend NPU, with hosted Colab and PAI-DSW notebooks
Weaknesses
- Official documentation is marked WIP in the README
- No VRAM or RAM figures in the visible README; hardware requirements unknown here
- Web UI is for training and chat, not a production serving layer
- Many optional backends (vLLM, SGLang, Unsloth, KTransformers) add install complexity
What it needs
- GPU required
- Docker + Compose
- Needs PyTorch, Hugging Face Transformers, Gradio, vLLM (optional), SGLang (optional)
- Models: LLaMA, LLaVA, Mistral, Mixtral-MoE, Qwen3
Head to head
Also in Training and fine-tuning
See all 2| Rank | Project | Score |
|---|---|---|
| 2 | minimindTrain a 64M-parameter LLM from scratch in plain PyTorch | 52 out of 100 |