#7272nd of 105 in Model serving

mtplx

MTP speculative-decoding inference server and Mac app for Apple Silicon

Releases and problems feed (RSS)

Stars
2.7k
License
Apache-2.0
Last commit
Nov 2026
Last release
Nov 2026
Language
Python
Stars, last 29 weeks+2,726
30 Apr 2026: 4 stars7 May 2026: 211 stars14 May 2026: 350 stars21 May 2026: 561 stars28 May 2026: 606 stars4 Jun 2026: 652 stars11 Jun 2026: 705 stars18 Jun 2026: 774 stars25 Jun 2026: 835 stars2 Jul 2026: 875 stars9 Jul 2026: 983 stars16 Jul 2026: 1,026 stars23 Jul 2026: 1,072 stars30 Jul 2026: 1,096 stars6 Aug 2026: 1,127 stars13 Aug 2026: 1,152 stars20 Aug 2026: 1,497 stars27 Aug 2026: 1,714 stars3 Sept 2026: 2,003 stars10 Sept 2026: 2,207 stars17 Sept 2026: 2,346 stars24 Sept 2026: 2,415 stars1 Oct 2026: 2,483 stars8 Oct 2026: 2,534 stars15 Oct 2026: 2,579 stars22 Oct 2026: 2,617 stars29 Oct 2026: 2,648 stars5 Nov 2026: 2,677 stars12 Nov 2026: 2,700 stars19 Nov 2026: 2,730 stars

4 on 30 Apr 20262,730 on 19 Nov 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

MTPLX runs Qwen, Gemma 4 and Ternary Bonsai models on Apple Silicon using each model's own multi-token prediction heads for speculative decoding, verified with exact rejection sampling. It ships as a native Mac app and a CLI, and serves OpenAI- and Anthropic-compatible APIs on 127.0.0.1:8000. README-reported speedups are 1.6x on a 16 GB M4 Mac mini and 2.24x on an M5 Max.

Who it is for: Mac users running local LLMs for coding agents on Apple Silicon

Strengths

  • Uses the model's own MTP heads, so no second draft model uses RAM
  • Serves OpenAI and Anthropic APIs, plus embeddings and rerank endpoints
  • Auto-tunes draft depth per Mac and keeps plain decoding as baseline
  • Forge converts a Hugging Face repo to MLX and trains and verifies an MTP adapter

Weaknesses

  • Apple Silicon and macOS 14+ only; no Linux or Windows support
  • Works only with MTPLX-prepared model packs, not arbitrary MLX models or MTP sidecars
  • Flagship Flash Next 125B needs 96 GB or more and a 106 GB+ download
  • Image-request sampling equivalence has not been re-verified since 2.12.0

What it needs

  • RAM ≥ 8 GB
  • GPU required
  • Needs Apple Silicon Mac, macOS 14+, Python 3.11+, Hugging Face model downloads
  • Models: Qwen 3.8 Flash Next, Qwen 3.8 27B, Ternary Bonsai 2 27B, Qwen 3.6, Qwen 3.5
  • port 8000

What changed

From the release notes. No flag means the note does not say so, not that the upgrade is safe.

Known problems

Open now, reported in the last 90 days

Titles are the reporters' own words, taken from the project's GitHub issues and ranked by reactions. We have not tried to reproduce any of them.

Also in Model serving

See all 105
Also in Model serving
RankProjectScore
1llama.cppC/C++ inference engine for GGUF LLMs and VLMs with an OpenAI-compatible server135.9k starsMIT94 out of 100
2OllamaRuns open models locally through a CLI and REST API187.8k starsMITknown advisory in latest version91 out of 100
3LocalAISelf-hosted model server with OpenAI, Anthropic and ElevenLabs-compatible APIs50.5k starsMIT91 out of 100
#4strataDesktop and terminal app that runs GGUF models locally with OpenAI-compatible APIs46.4k stars+16,488 (+55%) in 30 daysMIT89 out of 100
#5modularMojo language and MAX inference framework with an OpenAI-compatible server30.2k stars+228 (+0.8%) in 30 dayscustom license89 out of 100