#8282nd of 105 in ML frameworks and libraries
apex
NVIDIA PyTorch utilities for mixed precision and distributed training
- Stars
- 9k
- License
- BSD-3-Clause
- Last commit
- Sep 2026
- Language
- Python
8,901 on 1 Apr 20269,004 on 21 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Apex is an NVIDIA-maintained PyTorch extension library with fused CUDA kernels and utilities for mixed precision and distributed training. It provides FusedAdam, FusedLayerNorm, FusedRMSNorm, SyncBatchNorm, DistributedDataParallel and amp, plus many optional apex.contrib modules. Some of the code is intended to land in upstream PyTorch eventually.
Who it is for: PyTorch engineers training models on NVIDIA GPUs who need fused kernels
Strengths
- Fused CUDA kernels for Adam, LayerNorm, RMSNorm and SyncBatchNorm
- Maintained by NVIDIA; extensions come preinstalled in NGC PyTorch containers
- Python-only build available when CUDA extensions cannot be compiled
- Many optional contrib modules, each enabled by its own build flag
Weaknesses
- Source install compiles C++/CUDA extensions, which can be slow and memory-heavy
- Contrib modules may only work with PyTorch nightlies, not stable releases
- Python-only build drops fused kernels, so some components run slower
- Windows support is experimental; no tagged release
What it needs
- GPU required
- Needs PyTorch, CUDA, Ninja (recommended), cuDNN >=8.4/8.5 (some modules), NCCL >=2.10/2.19 (some modules)
Also in ML frameworks and libraries
See all 105| Rank | Project | Score |
|---|---|---|
| 1 | transformersPython library defining open models for text, vision, audio and multimodal tasks | 94 out of 100 |
| 2 | pytorchPython tensor library with GPU support and tape-based autograd for deep learning | 94 out of 100 |
| 3 | ultralyticsPython library and CLI for training and running YOLO vision models | 93 out of 100 |
| #4 | pytorch-image-modelsPyTorch library of image models, pretrained weights and reference training scripts | 92 out of 100 |
| #5 | supervisionPython toolkit for loading, annotating and counting computer vision detections | 91 out of 100 |