#2929th of 31 in Training and fine-tuning
lmflow
Toolbox for full, LoRA and LISA finetuning of large language models
Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 8.5k
- License
- Apache-2.0
- Last commit
- Aug 2026
- Last release
- Jul 2025
- Language
- Python
8,439 on 1 Apr 20268,482 on 21 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
LMFlow is a Python toolbox that finetunes decoder language models with full-parameter training, LoRA/QLoRA or LISA, driven by shell scripts and Accelerate. It adds conversation templates, optional DeepSpeed, FlashAttention-2, vLLM or SGLang inference, DPO via TRL, and a Gradio chatbot demo. The README lists estimated memory per model size, for example 16GB for 7B with LoRA.
Who it is for: ML engineers finetuning open LLMs on their own GPUs
Strengths
- Covers full, LoRA, QLoRA and LISA finetuning from ready-made shell scripts
- README gives memory estimates per method, from 0.5B to 70B models
- Optional extras for DeepSpeed, FlashAttention, vLLM, SGLang and TRL-based DPO
- Preset conversation templates such as llama3, llama2, phi3 and chatml
Weaknesses
- Tested only on Ubuntu 20.04; macOS and Windows may hit unexpected errors
- vLLM and SGLang need incompatible CUDA/PyTorch versions, so separate envs are required
- LISA script is only tested on a single GPU
- Last release is 2025-07-11; no Docker image or compose file
What it needs
- GPU required
- Needs Python 3.9+, conda, mpi4py, PyTorch, WandB (optional, on by default)
- Models: Hugging Face decoder models, LLaMA, Llama-2, Llama-3, Qwen
Also in Training and fine-tuning
See all 31| Rank | Project | Score |
|---|---|---|
| 1 | unslothDesktop and web app to run, fine-tune and export open models | 88 out of 100 |
| 2 | deepspeedPyTorch library for distributed training of large models with ZeRO | 84 out of 100 |
| 3 | verlReinforcement learning training library for LLM post-training with PPO and GRPO | 84 out of 100 |
| #4 | peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models | 84 out of 100 |
| #5 | trlPost-training library with SFT, DPO, GRPO and KTO trainers | 81 out of 100 |