#3737th of 76 in Model serving

rapid-mlx

MLX-based LLM inference server for Apple Silicon with coding-agent tool calling

Stars
4.2k
License
Apache-2.0
Last commit
Nov 2026
Last release
Nov 2026
Language
Python
Stars, last 29 weeks+3,925
25 Apr 2026: 234 stars2 May 2026: 486 stars9 May 2026: 1,859 stars16 May 2026: 2,190 stars23 May 2026: 2,284 stars30 May 2026: 2,490 stars6 Jun 2026: 2,553 stars13 Jun 2026: 2,667 stars20 Jun 2026: 2,900 stars27 Jun 2026: 3,006 stars4 Jul 2026: 3,068 stars11 Jul 2026: 3,133 stars18 Jul 2026: 3,176 stars25 Jul 2026: 3,295 stars1 Aug 2026: 3,362 stars8 Aug 2026: 3,397 stars15 Aug 2026: 3,458 stars22 Aug 2026: 3,512 stars29 Aug 2026: 3,548 stars5 Sept 2026: 3,652 stars12 Sept 2026: 3,721 stars19 Sept 2026: 3,782 stars26 Sept 2026: 3,837 stars3 Oct 2026: 3,888 stars10 Oct 2026: 3,959 stars17 Oct 2026: 4,000 stars24 Oct 2026: 4,029 stars31 Oct 2026: 4,086 stars7 Nov 2026: 4,120 stars14 Nov 2026: 4,159 stars

234 on 25 Apr 20264,159 on 14 Nov 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

Rapid-MLX serves MLX models on Apple Silicon over OpenAI-compatible (/v1/chat/completions, /v1/responses) and Anthropic Messages (/v1/messages) APIs. It uses continuous batching, a radix prefix cache and 27 tool-call parser modules, and `rapid-mlx launch <agent>` configures Codex, Claude Code, OpenCode and others against a local model. It also serves image generation and editing models through an Images API.

Who it is for: Mac users running local models behind coding agents or OpenAI-style clients

Strengths

  • OpenAI and Anthropic-compatible endpoints, so most local-endpoint clients work without an adapter
  • `rapid-mlx launch <agent>` sets up a private profile without editing existing agent settings
  • 27 tool-call parser modules with auto-detect fallback, aimed at coding agents
  • README benchmarks show higher decode tok/s than oMLX and LM Studio (single run, vendor-measured)

Weaknesses

  • Apple Silicon only; Windows and Linux desktop builds are not available
  • Speed comparisons are the project's own, single run per engine
  • License is reported as NOASSERTION by detection, although the README badge says Apache 2.0
  • Audio (TTS, STT) and embeddings need opt-in extras beyond the base install

What it needs

  • RAM ≥ 8 GB
  • GPU required
  • Needs Homebrew or uv/pip for install, Apple Silicon Mac, MLX
  • Models: MLX 4-bit LLMs (Qwen3.5, Qwen3.6, LFM2.5), vision models, image generation (FLUX, SDXL, Qwen Image), video generation, TTS/STT (extra)
  • port 8000

What changed

  1. “Catalog names: 44 niche or superseded aliases leave discovery; their repository IDs retain the same execution profiles. Old names warn and forward for one release (#5684).”

From the release notes. No flag means the note does not say so, not that the upgrade is safe.

Also in Model serving

See all 76
Also in Model serving
RankProjectScore
1llama.cppC/C++ inference engine for GGUF LLMs and VLMs with an OpenAI-compatible server135.7k starsMIT94 out of 100
2OllamaRuns open models locally through a CLI and REST API187.5k starsMITknown advisory in latest version92 out of 100
3LocalAISelf-hosted model server with OpenAI, Anthropic and ElevenLabs-compatible APIs50.4k starsMIT91 out of 100
#4vLLMInference engine serving LLMs with PagedAttention and an OpenAI-compatible API97.1k starsApache-2.088 out of 100
#5strataRuns a 125B-parameter MoE model on one consumer GPU plus system RAM45.4k starsMIT88 out of 100