#6666th of 105 in Model serving

project-maya

Runs very large MoE LLMs locally by streaming experts between GPU, RAM and SSD

Releases and problems feed (RSS)

Stars
3.1k
License
MIT
Last commit
Nov 2026
Last release
Nov 2026
Language
C++
Stars, last 6 weeks+2,931
8 Oct 2026: 203 stars15 Oct 2026: 952 stars22 Oct 2026: 1,574 stars29 Oct 2026: 2,271 stars5 Nov 2026: 2,611 stars12 Nov 2026: 2,892 stars19 Nov 2026: 3,134 stars

203 on 8 Oct 20263,134 on 19 Nov 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

Project Maya is an inference engine with custom GGUF quants for GLM-5.3, GLM-5.3-Flash, DeepSeek-V4.1-Flash, MiniMax-M3.1-Flash and Hy4-Flash. It keeps frequently used experts in VRAM, the next ones in RAM and the rest on NVMe, tuning itself to the measured hardware. It serves an OpenAI- and Anthropic-compatible API with tool calls, plus a browser chat, a live monitor and a Models page.

Who it is for: Self-hosters running 200B+ open LLMs on one or a few GPUs

Strengths

  • Tiered expert placement across VRAM, RAM and SSD lets 200-750B models run on one GPU
  • Publishes user benchmarks with hardware and version, e.g. 81 tokens/s on one RTX PRO 6000
  • OpenAI- and Anthropic-compatible API with tool calls; up to 8 concurrent requests on three models
  • Runs on NVIDIA from V100/RTX 20, AMD and Windows, with a one-command setup

Weaknesses

  • Needs roughly 56 GB to 192 GB of RAM plus VRAM, depending on the model
  • Each model needs its own engine; support is limited to the five listed families
  • Mixed licenses: GLM-5.3, MiniMax and Hy4 models carry their own non-MIT terms
  • Quants differ from originals, e.g. 83-90% next-token match on GLM-5.3-Flash

What it needs

  • RAM ≥ 56 GB
  • GPU required
  • Docker
  • Needs NVMe SSD
  • Models: GLM-5.3, GLM-5.3-Flash, DeepSeek-V4.1-Flash-1017, MiniMax-M3.1-Flash, Hy4-Flash
  • port 8080

What changed

From the release notes. No flag means the note does not say so, not that the upgrade is safe.

Known problems

Open now, reported in the last 90 days

Reported, then fixed by a linked change

Titles are the reporters' own words, taken from the project's GitHub issues and ranked by reactions. We have not tried to reproduce any of them.

Also in Model serving

See all 105
Also in Model serving
RankProjectScore
1llama.cppC/C++ inference engine for GGUF LLMs and VLMs with an OpenAI-compatible server135.9k starsMIT94 out of 100
2OllamaRuns open models locally through a CLI and REST API187.8k starsMITknown advisory in latest version91 out of 100
3LocalAISelf-hosted model server with OpenAI, Anthropic and ElevenLabs-compatible APIs50.5k starsMIT91 out of 100
#4strataDesktop and terminal app that runs GGUF models locally with OpenAI-compatible APIs46.4k stars+16,488 (+55%) in 30 daysMIT89 out of 100
#5modularMojo language and MAX inference framework with an OpenAI-compatible server30.2k stars+228 (+0.8%) in 30 dayscustom license89 out of 100