#6666th of 75 in ML frameworks and libraries
deepep
GPU communication kernels for MoE expert-parallel dispatch and combine
- Stars
- 10.3k
- License
- MIT
- Last commit
- Oct 2026
- Last release
- Sep 2025
- Language
- Cuda
9,003 on 31 Mar 202610,315 on 20 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
DeepEP is a CUDA and PyTorch library providing all-to-all kernels for MoE expert parallelism, including FP8 dispatch, behind a single EPBuffer API for training, prefill and decoding. It also offers NVLink weight prefetch and gradient reduction for redundant experts, plus experimental pipeline-parallel, Engram and bucketed collective primitives. Kernels are JIT-compiled at runtime through DeepJIT and use an NCCL Gin backend.
Who it is for: Engineers building MoE training and inference stacks on multi-GPU clusters
Strengths
- One EPBuffer API covers training, prefill and decoding, with FP8 dispatch support
- Analytical SM and QP count estimate replaces auto-tuning
- Reuses existing NCCL communicators; NVSHMEM is no longer a dependency
- Redundant-expert weight prefetch and FP32 gradient reduction over NVLink
Weaknesses
- Requires Hopper or newer NVIDIA GPUs, NVLink, and RDMA for multi-node
- Needs CUDA Toolkit 13.1+, NCCL 2.32.3+, PyTorch 2.10+, and a C++20 compiler
- Bucket, Engram and PP features are experimental
- Dispatch and combine consume GPU SMs; zero-SM RDMA EP is unsupported
What it needs
- GPU required
- Needs PyTorch 2.10+, NCCL 2.32.3+, CUDA Toolkit 13.1+, DeepJIT, NVLink, RDMA network
- Models: MoE models (BF16, FP8 dispatch)
Also in ML frameworks and libraries
See all 75| Rank | Project | Score |
|---|---|---|
| 1 | transformersPython library defining open models for text, vision, audio and multimodal tasks | 94 out of 100 |
| 2 | pytorchPython tensor library with GPU support and tape-based autograd for deep learning | 94 out of 100 |
| 3 | ultralyticsPython library and CLI for training and running YOLO vision models | 92 out of 100 |
| #4 | pytorch-image-modelsPyTorch library of image models, pretrained weights and reference training scripts | 91 out of 100 |
| #5 | supervisionPython toolkit for loading, annotating and counting computer vision detections | 90 out of 100 |