#2121st of 41 in Training and fine-tuning
liger-kernel
Triton kernels that cut memory and raise throughput for LLM training
- Stars
- 6.7k
- License
- BSD-2-Clause
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- Python
6,201 on 4 Apr 20266,681 on 31 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Liger Kernel is a PyTorch library of Triton kernels for RMSNorm, RoPE, SwiGLU, CrossEntropy and FusedLinearCrossEntropy, plus chunked post-training losses such as DPO, ORPO, SimPO, KTO and JSD. A one-line patch swaps them into Hugging Face models. The README reports about 20% higher multi-GPU throughput and 60% lower memory on LLaMA 3-8B, with exact (non-approximated) computation.
Who it is for: Engineers training or fine-tuning LLMs on GPUs who need less memory use
Strengths
- One-line patching of Hugging Face models; also usable as standalone modules
- Works with FSDP, DeepSpeed and DDP, and integrates with Axolotl, LLaMA-Factory and TRL
- Exact computation, checked by unit and convergence tests against runs without Liger
- Chunked post-training losses (DPO, ORPO, SimPO, KTO) claim up to 80% memory savings
Weaknesses
- Benchmarks come from one setup: LLaMA 3-8B on 8 A100s with FSDP1
- Model patching APIs need transformers >= 5.0.0, and per-model support varies
- cuTile and CuTe DSL backends are opt-in and CUDA-only, with newer-GPU targets (Hopper, Blackwell)
- Training-side only; it does not serve or run inference
What it needs
- GPU required
- Needs torch >= 2.1.2, triton >= 2.3.1, transformers >= 5.0.0 (optional)
- Models: Hugging Face transformers models, LLaMA 3, Qwen2-VL, Llama 3.2-Vision
Quick answers
Is liger-kernel open source?
Yes. Its code is on GitHub under the BSD-2-Clause license.
Does liger-kernel need a GPU?
Yes, it needs a GPU.
Also in Training and fine-tuning
See all 41| Rank | Project | Score |
|---|---|---|
| 1 | unslothDesktop and web app to run and fine-tune local models | 88 out of 100 |
| 2 | peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models | 86 out of 100 |
| 3 | deepspeedPyTorch library for distributed training of very large models with ZeRO | 85 out of 100 |
| #4 | verlReinforcement learning training library for LLM post-training with PPO and GRPO | 84 out of 100 |
| #5 | trlPost-training library with SFT, DPO, GRPO and KTO trainers | 83 out of 100 |