#7474th of 105 in Model serving
splash
Apple silicon inference server with speculative decoding and OpenAI/Anthropic APIs
- Stars
- 2.6k
- License
- Apache-2.0
- Last commit
- Nov 2026
- Last release
- Nov 2026
- Language
- C++
53 on 17 Sept 20262,586 on 19 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Splash is a C++ inference engine for Macs that serves GGUF and MLX models using DFlash 2 speculative decoding, custom Metal kernels and automatic memory planning. It exposes OpenAI Chat Completions, Responses and Completions plus Anthropic Messages endpoints on 127.0.0.1:8000, with a built-in chat page and launchers for coding agents such as opencode, claude and codex.
Who it is for: Mac users running local coding agents or API-compatible apps on Apple silicon
Strengths
- README reports 74 tok/s on Qwen3.8-27B, 4.5–5.3× llama.cpp decode on the same GGUF weights
- Serves both OpenAI and Anthropic API formats, with tool calls, JSON Schema output, images and PDFs
- Prefix caching, request batching and an optional SSD cache for long contexts
- Memory and context sized automatically; --max-memory caps Metal usage
Weaknesses
- Requires Apple M3 or newer on macOS 26.4+; no Linux, Windows or NVIDIA support
- Only five listed model families, each needing a matching DFlash2 draft
- Whole-BF16 and F16 GGUF models are not supported
- Listens without authentication by default; LAN use needs extra configuration
What it needs
- GPU required
- Needs Homebrew, macOS 26.4 or later, Apple M3 or newer
- Models: GGUF, MLX, Qwen3.8-27B, Qwen3.6-35B-A3B, Ornith-1.5-122B
- port 8000
What changed
From the release notes. No flag means the note does not say so, not that the upgrade is safe.
Known problems
Open now, reported in the last 90 days
- Just wanted to say thanks ↗ (opens in a new tab)24 reactions
- Apple7/8 (M1/M2): native Metal kernels, 2.1x decode and 2.3-2.7x prefill vs stock kernels on M1 Max ↗ (opens in a new tab)16 reactions
- [Showcase] Splash Monitor — Native macOS GUI for Splash inference engine ↗ (opens in a new tab)8 reactions
- [Showcase] Splash on iPhone 18 Pro Max: Qwen3.6-35B-A3B with experts streamed from flash (unofficial port) ↗ (opens in a new tab)3 reactions
- New Dspine Drafter By Tencent ↗ (opens in a new tab)2 reactions
Reported, then fixed by a linked change
- Startup memory check counts warmup's idle lane states, failing 24 GB Macs by tens of MiB at [memory_audit] ↗ (opens in a new tab)0 reactions · closed 26 Oct 2026Fixed by Check the host's reserve before Ready without the warmups' idle lanes ↗ (opens in a new tab)
- brew 1.1.0: launcher rejects --request-timeout ('unrecognized arguments') though server implements it ↗ (opens in a new tab)0 reactions · closed 28 Sept 2026Fixed by fix: accept and forward --request-timeout in the serve launcher ↗ (opens in a new tab)
Titles are the reporters' own words, taken from the project's GitHub issues and ranked by reactions. We have not tried to reproduce any of them.
Also in Model serving
See all 105| Rank | Project | Score |
|---|---|---|
| 1 | llama.cppC/C++ inference engine for GGUF LLMs and VLMs with an OpenAI-compatible server | 94 out of 100 |
| 2 | OllamaRuns open models locally through a CLI and REST API | 91 out of 100 |
| 3 | LocalAISelf-hosted model server with OpenAI, Anthropic and ElevenLabs-compatible APIs | 91 out of 100 |
| #4 | strataDesktop and terminal app that runs GGUF models locally with OpenAI-compatible APIs | 89 out of 100 |
| #5 | modularMojo language and MAX inference framework with an OpenAI-compatible server | 89 out of 100 |