#2121st of 57 in Model serving
magnitude
Local inference engine that tunes kernels per model on your hardware
- Stars
- 7.7k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- Rust
0 on 13 Jun 20267,650 on 31 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Magnitude is a desktop app and CLI inference engine that runs open-weight models on Apple Silicon, NVIDIA, AMD or CPU. When a model is downloaded it spends up to a minute tuning kernels on the local chip, and ships a speculative decoding drafter per model. It connects to agents such as Pi, OpenCode, Codex, Claude Code and Cline, or any client through an OpenAI-compatible API.
Who it is for: Developers running local open models for coding agents who want faster inference
Strengths
- Per-model kernel tuning at download time; README reports ~2.2x faster decode than Ollama on Mac and NVIDIA
- Runs on Apple Silicon, NVIDIA, AMD or CPU across macOS, Windows and Linux
- One-click connection to eight named agents; OpenAI-compatible API for the rest
- `magnitude server setup` installs a self-updating boot service for remote use
Weaknesses
- Optimized kernels cover popular open-weight families only; full model list is on the website
- Benchmarks are the project's own and compare mainly against Ollama
- No Docker or compose setup mentioned in the README
What it needs
- GPU optional
- Models: Open-weight LLM families listed at magnitude.dev/models, OpenAI-compatible API
Quick answers
Is magnitude open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Does magnitude need a GPU?
No. It runs on a CPU and can use a GPU when you have one.
Also in Model serving
See all 57| Rank | Project | Score |
|---|---|---|
| 1 | llama.cppC/C++ inference engine serving GGUF models over an OpenAI-compatible API | 93 out of 100 |
| 2 | OllamaLocal model runner with CLI and REST API for open-weight LLMs | 91 out of 100 |
| 3 | LocalAISelf-hosted inference server with OpenAI-compatible API and pluggable backends | 90 out of 100 |
| #4 | vLLMInference engine for Hugging Face LLMs with PagedAttention and continuous batching | 88 out of 100 |
| #5 | colibriC inference engine that streams MoE experts from disk to run huge models | 85 out of 100 |