#3434th of 68 in Training and fine-tuning
llm-compressor
Python library that quantizes and prunes Hugging Face models for vLLM
- Stars
- 4k
- License
- Apache-2.0
- Last commit
- Nov 2026
- Last release
- Oct 2026
- Language
- Python
3,133 on 25 Apr 20263,973 on 14 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
llmcompressor applies post-training quantization and expert pruning to Hugging Face models through a `oneshot` API driven by recipes. It supports GPTQ, AWQ, SmoothQuant, AutoRound, SpinQuant/QuIP rotations and REAP, across int8, FP8, NVFP4, MXFP4 and related schemes. Output is saved in the compressed-tensors format, which vLLM loads directly.
Who it is for: Engineers compressing models for vLLM deployment
Strengths
- Wide scheme coverage: W8A8, W4A16, NVFP4, MXFP4/6/8, plus KV cache and attention quantization
- Multiple algorithms selectable via recipe: GPTQ, AWQ, SmoothQuant, AutoRound, rotations, REAP pruning
- Handles large models with DDP, sequential onloading and disk offloading
- Checkpoints load directly in vLLM; many prebuilt checkpoints published under RedHatAI on Hugging Face
Weaknesses
- Output targets vLLM via compressed-tensors; other runtimes are not mentioned
- Some features are labeled experimental, such as NVFP4 attention with SpinQuant
- Users must choose scheme, algorithm and calibration setup themselves
- Architectures without a shipped model definition need the model-free path
What it needs
- Needs PyTorch, Hugging Face Transformers, compressed-tensors, vLLM (for inference)
- Models: Hugging Face Transformers LLMs, MoE models, Vision-language models, Audio-language models, compressed-tensors output
What changed
- 0.15.0 ↗ (opens in a new tab)Breaking
“Breaking Changes and Removals”
From the release notes. No flag means the note does not say so, not that the upgrade is safe.
Also in Training and fine-tuning
See all 68| Rank | Project | Score |
|---|---|---|
| 1 | peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models | 89 out of 100 |
| 2 | unslothDesktop and web app to run and fine-tune local models | 88 out of 100 |
| 3 | trlPost-training library with SFT, DPO, GRPO and KTO trainers | 88 out of 100 |
| #4 | deepspeedPyTorch library for distributed training of very large models with ZeRO | 87 out of 100 |
| #5 | megatron-lmDistributed training library for large transformer models on NVIDIA GPUs | 86 out of 100 |