#2525th of 53 in Training and fine-tuning
model-optimizer
Python library for quantizing, pruning and distilling models for NVIDIA inference
- Stars
- 5.6k
- License
- Apache-2.0
- Last commit
- Nov 2026
- Last release
- Nov 2026
- Language
- Python
2,499 on 21 Apr 20265,587 on 10 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
NVIDIA Model Optimizer (ModelOpt) is a Python library that applies post-training quantization, quantization-aware training and distillation, pruning, NAS, speculative decoding and sparsity to Hugging Face, PyTorch or ONNX models. It exports quantized checkpoints (FP8, NVFP4 and others) for deployment in TensorRT-LLM, TensorRT, vLLM or SGLang. It integrates with Megatron-Bridge, Megatron-LM and Hugging Face Accelerate for techniques that need training.
Who it is for: ML engineers compressing LLMs and diffusion models for NVIDIA GPU inference
Strengths
- Covers PTQ, QAT/QAD, pruning, NAS, distillation, speculative decoding and sparsity in one library
- Exports to TensorRT-LLM, TensorRT, vLLM and SGLang; unified HF export covers transformers and diffusers
- Publishes pre-quantized checkpoints on Hugging Face, including NVFP4 and FP8 variants
- Installable from PyPI as nvidia-modelopt; Apache-2.0 with per-technique examples and support matrices
Weaknesses
- Pre-1.0 API; deprecated features are removed after about one release
- Tuned for the NVIDIA stack; NVFP4 and TensorRT paths target NVIDIA GPUs
- Pulls in additional third-party packages whose licenses need separate review
- QAT, QAD and pruning workflows depend on Megatron or Accelerate setups
What it needs
- GPU required
- Needs PyTorch, Hugging Face Transformers, ONNX, TensorRT-LLM, vLLM, SGLang, Megatron-Bridge
- Models: Hugging Face, PyTorch, ONNX, Diffusers, FP8
Quick answers
Is model-optimizer open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Does model-optimizer need a GPU?
Yes, it needs a GPU.
Also in Training and fine-tuning
See all 53| Rank | Project | Score |
|---|---|---|
| 1 | unslothDesktop and web app to run and fine-tune local models | 88 out of 100 |
| 2 | peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models | 88 out of 100 |
| 3 | deepspeedPyTorch library for distributed training of very large models with ZeRO | 86 out of 100 |
| #4 | verlReinforcement learning training library for LLM post-training with PPO and GRPO | 86 out of 100 |
| #5 | trlPost-training library with SFT, DPO, GRPO and KTO trainers | 86 out of 100 |