#7474th of 105 in Model serving

splash

Apple silicon inference server with speculative decoding and OpenAI/Anthropic APIs

Releases and problems feed (RSS)

Stars
2.6k
License
Apache-2.0
Last commit
Nov 2026
Last release
Nov 2026
Language
C++
Stars, last 9 weeks+2,533
17 Sept 2026: 53 stars24 Sept 2026: 823 stars1 Oct 2026: 1,131 stars8 Oct 2026: 1,348 stars15 Oct 2026: 1,522 stars22 Oct 2026: 1,787 stars29 Oct 2026: 1,991 stars5 Nov 2026: 2,152 stars12 Nov 2026: 2,287 stars19 Nov 2026: 2,586 stars

53 on 17 Sept 20262,586 on 19 Nov 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

Splash is a C++ inference engine for Macs that serves GGUF and MLX models using DFlash 2 speculative decoding, custom Metal kernels and automatic memory planning. It exposes OpenAI Chat Completions, Responses and Completions plus Anthropic Messages endpoints on 127.0.0.1:8000, with a built-in chat page and launchers for coding agents such as opencode, claude and codex.

Who it is for: Mac users running local coding agents or API-compatible apps on Apple silicon

Strengths

  • README reports 74 tok/s on Qwen3.8-27B, 4.5–5.3× llama.cpp decode on the same GGUF weights
  • Serves both OpenAI and Anthropic API formats, with tool calls, JSON Schema output, images and PDFs
  • Prefix caching, request batching and an optional SSD cache for long contexts
  • Memory and context sized automatically; --max-memory caps Metal usage

Weaknesses

  • Requires Apple M3 or newer on macOS 26.4+; no Linux, Windows or NVIDIA support
  • Only five listed model families, each needing a matching DFlash2 draft
  • Whole-BF16 and F16 GGUF models are not supported
  • Listens without authentication by default; LAN use needs extra configuration

What it needs

  • GPU required
  • Needs Homebrew, macOS 26.4 or later, Apple M3 or newer
  • Models: GGUF, MLX, Qwen3.8-27B, Qwen3.6-35B-A3B, Ornith-1.5-122B
  • port 8000

What changed

From the release notes. No flag means the note does not say so, not that the upgrade is safe.

Known problems

Open now, reported in the last 90 days

Reported, then fixed by a linked change

Titles are the reporters' own words, taken from the project's GitHub issues and ranked by reactions. We have not tried to reproduce any of them.

Also in Model serving

See all 105
Also in Model serving
RankProjectScore
1llama.cppC/C++ inference engine for GGUF LLMs and VLMs with an OpenAI-compatible server135.9k starsMIT94 out of 100
2OllamaRuns open models locally through a CLI and REST API187.8k starsMITknown advisory in latest version91 out of 100
3LocalAISelf-hosted model server with OpenAI, Anthropic and ElevenLabs-compatible APIs50.5k starsMIT91 out of 100
#4strataDesktop and terminal app that runs GGUF models locally with OpenAI-compatible APIs46.4k stars+16,488 (+55%) in 30 daysMIT89 out of 100
#5modularMojo language and MAX inference framework with an OpenAI-compatible server30.2k stars+228 (+0.8%) in 30 dayscustom license89 out of 100