#5959th of 89 in Image and videoFor image, video and design, image and design

nunchaku

4-bit SVDQuant inference engine for FLUX, Qwen-Image and SANA diffusion models

Stars
4k
License
Apache-2.0
Last commit
Sep 2026
Last release
Jan 2026
Language
Python
Stars, last 29 weeks+210
25 Apr 2026: 3,759 stars2 May 2026: 3,768 stars9 May 2026: 3,776 stars16 May 2026: 3,790 stars23 May 2026: 3,799 stars30 May 2026: 3,809 stars6 Jun 2026: 3,815 stars13 Jun 2026: 3,825 stars20 Jun 2026: 3,837 stars27 Jun 2026: 3,853 stars4 Jul 2026: 3,861 stars11 Jul 2026: 3,870 stars18 Jul 2026: 3,879 stars25 Jul 2026: 3,886 stars1 Aug 2026: 3,893 stars8 Aug 2026: 3,902 stars15 Aug 2026: 3,905 stars22 Aug 2026: 3,911 stars29 Aug 2026: 3,913 stars5 Sept 2026: 3,924 stars12 Sept 2026: 3,931 stars19 Sept 2026: 3,937 stars26 Sept 2026: 3,941 stars3 Oct 2026: 3,946 stars10 Oct 2026: 3,946 stars17 Oct 2026: 3,950 stars24 Oct 2026: 3,955 stars31 Oct 2026: 3,958 stars7 Nov 2026: 3,964 stars14 Nov 2026: 3,969 stars

3,759 on 25 Apr 20263,969 on 14 Nov 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

Nunchaku is a Python inference engine that runs diffusion models quantized to 4-bit weights and activations using SVDQuant, with fused kernels for the low-rank branch. The README reports a 3.6x memory reduction on 12B FLUX.1-dev and about 3x speedup over an NF4 W4A16 baseline on RTX 4090. It supports INT4 and NVFP4 (RTX 5090), LoRA, ControlNet, CPU offloading, and a ComfyUI plugin.

Who it is for: Self-hosters running diffusion models on consumer NVIDIA GPUs with limited VRAM

Strengths

  • Reported 3.6x memory reduction and about 3x speedup over NF4 on FLUX.1-dev
  • Supports INT4 and NVFP4 precision, including 20-series and RTX 5090 GPUs
  • Offloading cuts Qwen-Image VRAM to about 3 GiB and FLUX to 4 GiB
  • ComfyUI plugin, Gradio demo apps and Python examples for many models

Weaknesses

  • Needs NVIDIA GPU; no CPU-only or non-NVIDIA backend is described
  • Models must be pre-quantized, or quantized separately with DeepCompressor
  • Covers diffusion models only; no LLM serving or HTTP API
  • Last tagged release is January 2026, while main has later commits

What it needs

  • GPU required
  • Needs PyTorch, ComfyUI (optional plugin), DeepCompressor (optional, for quantization)
  • Models: FLUX.1-dev, FLUX.1-Kontext, FLUX.1-Krea-dev, FLUX.1-tools, Qwen-Image

What changed

From the release notes. No flag means the note does not say so, not that the upgrade is safe.

Also in Image and video

See all 89
Also in Image and video
RankProjectScore
1frigateLocal NVR with realtime object detection for IP cameras37.6k starsMIT93 out of 100
2MoneyPrinterTurboGenerates short videos from a topic, with script, footage, voiceover and subtitles144.3k starsMIT88 out of 100
3toonflow-appInfinite-canvas tool for AI short drama and comic video production22.1k starsMIT88 out of 100
#4InvokeAIWeb UI for diffusion image generation with inpainting canvas and node workflows29.1k starsApache-2.087 out of 100
#5ComfyUINode-graph editor for running diffusion image, video, audio and 3D workflows147.4k starsGPL-3.085 out of 100