#3737th of 76 in Model serving
rapid-mlx
MLX-based LLM inference server for Apple Silicon with coding-agent tool calling
- Stars
- 4.2k
- License
- Apache-2.0
- Last commit
- Nov 2026
- Last release
- Nov 2026
- Language
- Python
234 on 25 Apr 20264,159 on 14 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Rapid-MLX serves MLX models on Apple Silicon over OpenAI-compatible (/v1/chat/completions, /v1/responses) and Anthropic Messages (/v1/messages) APIs. It uses continuous batching, a radix prefix cache and 27 tool-call parser modules, and `rapid-mlx launch <agent>` configures Codex, Claude Code, OpenCode and others against a local model. It also serves image generation and editing models through an Images API.
Who it is for: Mac users running local models behind coding agents or OpenAI-style clients
Strengths
- OpenAI and Anthropic-compatible endpoints, so most local-endpoint clients work without an adapter
- `rapid-mlx launch <agent>` sets up a private profile without editing existing agent settings
- 27 tool-call parser modules with auto-detect fallback, aimed at coding agents
- README benchmarks show higher decode tok/s than oMLX and LM Studio (single run, vendor-measured)
Weaknesses
- Apple Silicon only; Windows and Linux desktop builds are not available
- Speed comparisons are the project's own, single run per engine
- License is reported as NOASSERTION by detection, although the README badge says Apache 2.0
- Audio (TTS, STT) and embeddings need opt-in extras beyond the base install
What it needs
- RAM ≥ 8 GB
- GPU required
- Needs Homebrew or uv/pip for install, Apple Silicon Mac, MLX
- Models: MLX 4-bit LLMs (Qwen3.5, Qwen3.6, LFM2.5), vision models, image generation (FLUX, SDXL, Qwen Image), video generation, TTS/STT (extra)
- port 8000
What changed
- v1.0.3 ↗ (opens in a new tab)Breaking
“Catalog names: 44 niche or superseded aliases leave discovery; their repository IDs retain the same execution profiles. Old names warn and forward for one release (#5684).”
From the release notes. No flag means the note does not say so, not that the upgrade is safe.
Also in Model serving
See all 76| Rank | Project | Score |
|---|---|---|
| 1 | llama.cppC/C++ inference engine for GGUF LLMs and VLMs with an OpenAI-compatible server | 94 out of 100 |
| 2 | OllamaRuns open models locally through a CLI and REST API | 92 out of 100 |
| 3 | LocalAISelf-hosted model server with OpenAI, Anthropic and ElevenLabs-compatible APIs | 91 out of 100 |
| #4 | vLLMInference engine serving LLMs with PagedAttention and an OpenAI-compatible API | 88 out of 100 |
| #5 | strataRuns a 125B-parameter MoE model on one consumer GPU plus system RAM | 88 out of 100 |