ArchestackBest of Open-Source AI: List on GitHub ↗ (opens in a new tab)

#1919th of 47 in Model serving

kimi-k3-in-c

Kimi K3 inference engine in portable C99 that streams weights from disk

Repository on GitHub ↗ (opens in a new tab)
Stars
9.1k
License
Apache-2.0
Last commit
Oct 2026
Last release
Oct 2026
Language
C
Stars, last 12 weeks+9,146
29 Jul 2026: 0 stars5 Aug 2026: 2,583 stars12 Aug 2026: 5,198 stars19 Aug 2026: 6,022 stars26 Aug 2026: 6,449 stars2 Sept 2026: 6,953 stars9 Sept 2026: 7,340 stars16 Sept 2026: 7,924 stars23 Sept 2026: 8,398 stars30 Sept 2026: 8,758 stars7 Oct 2026: 8,900 stars14 Oct 2026: 9,036 stars21 Oct 2026: 9,146 stars

0 on 29 Jul 20269,146 on 21 Oct 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

Runs the 2.78T-parameter Kimi K3 checkpoint (1.56 TB) on CPU in C99, with no BLAS, framework or GPU. The dense trunk is packed and streamed from disk, and routed experts are multiplied directly from their 4-bit form through an LRU cache. Output is stated to be byte-identical from 8 GB up to 224 GB of RAM; only speed changes, from 26.5 s per token at 8 GB to 5.6 s at 128 GB+.

Who it is for: Engineers curious about running trillion-parameter models on commodity CPUs

Strengths

  • Runs a 2.78T-parameter model in 8.24 GB peak RSS, measured
  • Small C99 codebase: needs only a compiler, libm and OpenMP
  • make test checks the engine against PyTorch fixtures without a checkpoint
  • Builds on Linux x86-64, macOS arm64 and Windows; AVX2/FMA and NEON

Weaknesses

  • Needs about 1.7 TB free disk for checkpoint and packed trunk
  • 26.5 s per token at 8 GB and 5.6 s at 128 GB+; too slow for interactive use
  • Targets Kimi K3 only; no other model formats mentioned
  • No HTTP API or server mode in the visible README

What it needs

  • RAM ≥ 8 GB
  • no GPU
  • Needs C99 compiler (GCC >= 9 or Clang >= 10), OpenMP, GNU make or CMake, Python 3.9+ for download and pack tools, Hugging Face token for checkpoint download
  • Models: Kimi K3 (safetensors, 4-bit packed experts)

Also in Model serving

See all 47
Also in Model serving
RankProjectScore
1LocalAISelf-hosted inference server with OpenAI-compatible API and pluggable backends49.8k starsMIT83 out of 100
2llama.cppC/C++ inference engine serving GGUF models over an OpenAI-compatible API132k starsMIT81 out of 100
3OllamaLocal model runner with CLI and REST API for open-weight LLMs184.2k starsMIT80 out of 100
#4strataRuns a 125B-parameter MoE model on one consumer GPU plus system RAM30k starsMIT79 out of 100
#5vLLMInference engine for Hugging Face LLMs with PagedAttention and continuous batching94.9k starsApache-2.077 out of 100