#3434th of 68 in Training and fine-tuning

llm-compressor

Python library that quantizes and prunes Hugging Face models for vLLM

Stars
4k
License
Apache-2.0
Last commit
Nov 2026
Last release
Oct 2026
Language
Python
Stars, last 29 weeks+840
25 Apr 2026: 3,133 stars2 May 2026: 3,151 stars9 May 2026: 3,178 stars16 May 2026: 3,220 stars23 May 2026: 3,254 stars30 May 2026: 3,275 stars6 Jun 2026: 3,319 stars13 Jun 2026: 3,371 stars20 Jun 2026: 3,402 stars27 Jun 2026: 3,435 stars4 Jul 2026: 3,474 stars11 Jul 2026: 3,513 stars18 Jul 2026: 3,543 stars25 Jul 2026: 3,566 stars1 Aug 2026: 3,599 stars8 Aug 2026: 3,633 stars15 Aug 2026: 3,669 stars22 Aug 2026: 3,699 stars29 Aug 2026: 3,727 stars5 Sept 2026: 3,751 stars12 Sept 2026: 3,769 stars19 Sept 2026: 3,792 stars26 Sept 2026: 3,817 stars3 Oct 2026: 3,837 stars10 Oct 2026: 3,860 stars17 Oct 2026: 3,883 stars24 Oct 2026: 3,909 stars31 Oct 2026: 3,933 stars7 Nov 2026: 3,955 stars14 Nov 2026: 3,973 stars

3,133 on 25 Apr 20263,973 on 14 Nov 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

llmcompressor applies post-training quantization and expert pruning to Hugging Face models through a `oneshot` API driven by recipes. It supports GPTQ, AWQ, SmoothQuant, AutoRound, SpinQuant/QuIP rotations and REAP, across int8, FP8, NVFP4, MXFP4 and related schemes. Output is saved in the compressed-tensors format, which vLLM loads directly.

Who it is for: Engineers compressing models for vLLM deployment

Strengths

  • Wide scheme coverage: W8A8, W4A16, NVFP4, MXFP4/6/8, plus KV cache and attention quantization
  • Multiple algorithms selectable via recipe: GPTQ, AWQ, SmoothQuant, AutoRound, rotations, REAP pruning
  • Handles large models with DDP, sequential onloading and disk offloading
  • Checkpoints load directly in vLLM; many prebuilt checkpoints published under RedHatAI on Hugging Face

Weaknesses

  • Output targets vLLM via compressed-tensors; other runtimes are not mentioned
  • Some features are labeled experimental, such as NVFP4 attention with SpinQuant
  • Users must choose scheme, algorithm and calibration setup themselves
  • Architectures without a shipped model definition need the model-free path

What it needs

  • Needs PyTorch, Hugging Face Transformers, compressed-tensors, vLLM (for inference)
  • Models: Hugging Face Transformers LLMs, MoE models, Vision-language models, Audio-language models, compressed-tensors output

What changed

  1. “Breaking Changes and Removals”

From the release notes. No flag means the note does not say so, not that the upgrade is safe.

Also in Training and fine-tuning

See all 68
Also in Training and fine-tuning
RankProjectScore
1peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models Live demo ↗ (opens in a new tab)21.9k starsApache-2.089 out of 100
2unslothDesktop and web app to run and fine-tune local models Live demo ↗ (opens in a new tab)79.7k starsApache-2.088 out of 100
3trlPost-training library with SFT, DPO, GRPO and KTO trainers19.8k starsApache-2.088 out of 100
#4deepspeedPyTorch library for distributed training of very large models with ZeRO43.3k starsApache-2.087 out of 100
#5megatron-lmDistributed training library for large transformer models on NVIDIA GPUs18.4k starscustom license86 out of 100