#236236th of 242 in ML frameworks and libraries

sageattention

8-bit quantized attention GPU kernels that replace PyTorch scaled_dot_product_attention

Stars
4k
License
Apache-2.0
Last commit
Jan 2026
Language
Cuda
Stars, last 29 weeks+716
25 Apr 2026: 3,297 stars2 May 2026: 3,309 stars9 May 2026: 3,319 stars16 May 2026: 3,340 stars23 May 2026: 3,354 stars30 May 2026: 3,370 stars6 Jun 2026: 3,389 stars13 Jun 2026: 3,397 stars20 Jun 2026: 3,408 stars27 Jun 2026: 3,423 stars4 Jul 2026: 3,443 stars11 Jul 2026: 3,465 stars18 Jul 2026: 3,479 stars25 Jul 2026: 3,491 stars1 Aug 2026: 3,513 stars8 Aug 2026: 3,595 stars15 Aug 2026: 3,633 stars22 Aug 2026: 3,659 stars29 Aug 2026: 3,676 stars5 Sept 2026: 3,695 stars12 Sept 2026: 3,717 stars19 Sept 2026: 3,823 stars26 Sept 2026: 3,938 stars3 Oct 2026: 3,948 stars10 Oct 2026: 3,959 stars17 Oct 2026: 3,968 stars24 Oct 2026: 3,977 stars31 Oct 2026: 3,987 stars7 Nov 2026: 4,001 stars14 Nov 2026: 4,013 stars

3,297 on 25 Apr 20264,013 on 14 Nov 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

SageAttention provides CUDA and Triton attention kernels that quantize QK^T to INT8 and PV to FP8 or FP16, targeting Ampere, Ada and Hopper GPUs. Its `sageattn` function picks a kernel per GPU and can be swapped in for `F.scaled_dot_product_attention` in video and image diffusion models. The repo also holds SageAttention2++ and a Blackwell-targeted SageAttention3 (FP4) directory.

Who it is for: Engineers speeding up diffusion and transformer inference on NVIDIA GPUs

Strengths

  • Drop-in `sageattn` call; one-line patch of scaled_dot_product_attention shown for CogVideoX
  • Kernels for Ampere, Ada and Hopper, with RTX 5090 compile support
  • Supports GQA, varlen, differing q and kv lengths, torch.compile and distributed inference
  • Includes benchmark scripts against FlashAttention2 and FlashAttention3, plus model examples

Weaknesses

  • NVIDIA CUDA only; needs CUDA 12.0 to 12.8 depending on GPU generation
  • Global SDPA patch does not work for every model; attention class may need editing
  • Inputs must be FP16 or BF16; building from source needs a CUDA toolchain
  • SageAttention3 is FP4 for Blackwell; README advises SageAttention2 where precision matters

What it needs

  • GPU required
  • Needs python>=3.9, torch>=2.3.0, triton>=3.0.0, CUDA>=12.0, flash-attn (benchmarking only)
  • Models: CogVideoX, Mochi, FP16/BF16 PyTorch attention

Also in ML frameworks and libraries

See all 242
Also in ML frameworks and libraries
RankProjectScore
1transformersModel-definition library for pretrained text, vision, audio and multimodal models169.3k starsApache-2.094 out of 100
2ultralyticsPython library and CLI for training and running YOLO vision models63.5k starsAGPL-3.094 out of 100
3pytorch-image-modelsPyTorch library of image models, pretrained weights and training scripts37.3k starsApache-2.094 out of 100
#4pytorchPython tensor library with GPU support and tape-based autograd105.8k starscustom license93 out of 100
#5supervisionPython toolkit for loading, annotating and counting computer vision detections Live demo ↗ (opens in a new tab)51.7k starsMIT92 out of 100