#236236th of 242 in ML frameworks and libraries
sageattention
8-bit quantized attention GPU kernels that replace PyTorch scaled_dot_product_attention
- Stars
- 4k
- License
- Apache-2.0
- Last commit
- Jan 2026
- Language
- Cuda
3,297 on 25 Apr 20264,013 on 14 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
SageAttention provides CUDA and Triton attention kernels that quantize QK^T to INT8 and PV to FP8 or FP16, targeting Ampere, Ada and Hopper GPUs. Its `sageattn` function picks a kernel per GPU and can be swapped in for `F.scaled_dot_product_attention` in video and image diffusion models. The repo also holds SageAttention2++ and a Blackwell-targeted SageAttention3 (FP4) directory.
Who it is for: Engineers speeding up diffusion and transformer inference on NVIDIA GPUs
Strengths
- Drop-in `sageattn` call; one-line patch of scaled_dot_product_attention shown for CogVideoX
- Kernels for Ampere, Ada and Hopper, with RTX 5090 compile support
- Supports GQA, varlen, differing q and kv lengths, torch.compile and distributed inference
- Includes benchmark scripts against FlashAttention2 and FlashAttention3, plus model examples
Weaknesses
- NVIDIA CUDA only; needs CUDA 12.0 to 12.8 depending on GPU generation
- Global SDPA patch does not work for every model; attention class may need editing
- Inputs must be FP16 or BF16; building from source needs a CUDA toolchain
- SageAttention3 is FP4 for Blackwell; README advises SageAttention2 where precision matters
What it needs
- GPU required
- Needs python>=3.9, torch>=2.3.0, triton>=3.0.0, CUDA>=12.0, flash-attn (benchmarking only)
- Models: CogVideoX, Mochi, FP16/BF16 PyTorch attention
Also in ML frameworks and libraries
See all 242| Rank | Project | Score |
|---|---|---|
| 1 | transformersModel-definition library for pretrained text, vision, audio and multimodal models | 94 out of 100 |
| 2 | ultralyticsPython library and CLI for training and running YOLO vision models | 94 out of 100 |
| 3 | pytorch-image-modelsPyTorch library of image models, pretrained weights and training scripts | 94 out of 100 |
| #4 | pytorchPython tensor library with GPU support and tape-based autograd | 93 out of 100 |
| #5 | supervisionPython toolkit for loading, annotating and counting computer vision detections | 92 out of 100 |