#3333rd of 37 in ML frameworks and libraries
candle
Minimalist Rust ML framework with PyTorch-like tensors and CPU, CUDA and WASM backends
Live demo ↗
(opens in a new tab)Documentation ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 21.2k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Language
- Rust
Missed the latest GitHub check; these facts may be older.
Overview
Candle is a Rust tensor and neural-network library from Hugging Face, with an API modeled on PyTorch. It runs on an optimized CPU backend (optional MKL or Accelerate), CUDA with NCCL multi-GPU, and WASM in the browser. The repo includes example implementations of LLaMA, Mistral, Mixtral, Gemma, Phi, Whisper, Stable Diffusion, YOLO and others, and it loads safetensors, npz, ggml and PyTorch files.
Who it is for: Rust developers embedding ML inference in lightweight binaries
Strengths
- Builds into small Rust binaries without Python, aimed at serverless inference
- Many model examples: LLaMA 1-3, Mistral, Mixtral, Gemma, Phi, Whisper, Stable Diffusion, YOLO
- Loads safetensors, npz, ggml and PyTorch files; supports llama.cpp quantized types
- WASM backend runs models in the browser, with hosted demos for Whisper, LLaMA2, T5 and SAM
Weaknesses
- Library, not a server; OpenAI-compatible APIs come from external projects like candle-vllm
- Requires writing Rust; no Python bindings or ready-made UI
- Model list in the README is mostly 2023-2024 era architectures
- MKL and Accelerate builds can hit linker errors that need manual crate imports
What it needs
- GPU optional
- Needs Rust toolchain, CUDA (optional), cuDNN (optional), Intel MKL (optional), trunk (WASM examples)
- Models: safetensors, npz, ggml, PyTorch, llama.cpp quantized types
Also in ML frameworks and libraries
See all 37| Rank | Project | Score |
|---|---|---|
| 1 | transformersPython library defining open models for text, vision, audio and multimodal tasks | 95 out of 100 |
| 2 | pytorchPython tensor library with GPU support and tape-based autograd for deep learning | 94 out of 100 |
| 3 | ultralyticsPython library and CLI for training and running YOLO vision models | 90 out of 100 |
| #4 | supervisionPython toolkit for loading, annotating and counting computer vision detections | 87 out of 100 |
| #5 | pytorch-image-modelsPyTorch library of image models, pretrained weights and reference training scripts | 86 out of 100 |