#7272nd of 105 in Model serving
mtplx
MTP speculative-decoding inference server and Mac app for Apple Silicon
- Stars
- 2.7k
- License
- Apache-2.0
- Last commit
- Nov 2026
- Last release
- Nov 2026
- Language
- Python
4 on 30 Apr 20262,730 on 19 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
MTPLX runs Qwen, Gemma 4 and Ternary Bonsai models on Apple Silicon using each model's own multi-token prediction heads for speculative decoding, verified with exact rejection sampling. It ships as a native Mac app and a CLI, and serves OpenAI- and Anthropic-compatible APIs on 127.0.0.1:8000. README-reported speedups are 1.6x on a 16 GB M4 Mac mini and 2.24x on an M5 Max.
Who it is for: Mac users running local LLMs for coding agents on Apple Silicon
Strengths
- Uses the model's own MTP heads, so no second draft model uses RAM
- Serves OpenAI and Anthropic APIs, plus embeddings and rerank endpoints
- Auto-tunes draft depth per Mac and keeps plain decoding as baseline
- Forge converts a Hugging Face repo to MLX and trains and verifies an MTP adapter
Weaknesses
- Apple Silicon and macOS 14+ only; no Linux or Windows support
- Works only with MTPLX-prepared model packs, not arbitrary MLX models or MTP sidecars
- Flagship Flash Next 125B needs 96 GB or more and a 106 GB+ download
- Image-request sampling equivalence has not been re-verified since 2.12.0
What it needs
- RAM ≥ 8 GB
- GPU required
- Needs Apple Silicon Mac, macOS 14+, Python 3.11+, Hugging Face model downloads
- Models: Qwen 3.8 Flash Next, Qwen 3.8 27B, Ternary Bonsai 2 27B, Qwen 3.6, Qwen 3.5
- port 8000
What changed
From the release notes. No flag means the note does not say so, not that the upgrade is safe.
Known problems
Open now, reported in the last 90 days
Titles are the reporters' own words, taken from the project's GitHub issues and ranked by reactions. We have not tried to reproduce any of them.
Also in Model serving
See all 105| Rank | Project | Score |
|---|---|---|
| 1 | llama.cppC/C++ inference engine for GGUF LLMs and VLMs with an OpenAI-compatible server | 94 out of 100 |
| 2 | OllamaRuns open models locally through a CLI and REST API | 91 out of 100 |
| 3 | LocalAISelf-hosted model server with OpenAI, Anthropic and ElevenLabs-compatible APIs | 91 out of 100 |
| #4 | strataDesktop and terminal app that runs GGUF models locally with OpenAI-compatible APIs | 89 out of 100 |
| #5 | modularMojo language and MAX inference framework with an OpenAI-compatible server | 89 out of 100 |