#6363rd of 65 in Model serving
mini-sglang
Compact SGLang reimplementation for serving LLMs, in about 5,000 lines of Python
- Stars
- 5.6k
- License
- MIT
- Last commit
- May 2026
- Language
- Python
4,010 on 21 Apr 20265,560 on 10 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Mini-SGLang is a compact Python reimplementation of SGLang. It serves Hugging Face models over an OpenAI-compatible API and has a terminal chat shell mode. It includes radix cache, chunked prefill, overlap scheduling, tensor parallelism and FlashAttention/FlashInfer kernels. It is meant as a readable reference for how LLM serving works.
Who it is for: Engineers and researchers studying or prototyping LLM serving on NVIDIA GPUs
Strengths
- Small, type-annotated codebase (~5,000 lines of Python) that is practical to read and modify
- Implements radix cache, chunked prefill, overlap scheduling and tensor parallelism via --tp
- OpenAI-compatible server plus a terminal chat shell via --shell
- README includes benchmarks against SGLang on H200 GPUs with Qwen3 models
Weaknesses
- Linux only (x86_64, aarch64); macOS and Windows need WSL2 or Docker
- Needs an NVIDIA GPU and a CUDA Toolkit matching the driver; kernels are JIT-compiled
- No tagged release; installed from source with uv pip install -e .
- Documented as a compact subset, so likely narrower in features than full SGLang
What it needs
- GPU required
- Docker
- Needs NVIDIA CUDA Toolkit, FlashInfer, sgl-kernel, uv
- Models: Hugging Face models, Qwen3, Llama 3.1, ModelScope download
- port 1919
Quick answers
Is mini-sglang open source?
Yes. Its code is on GitHub under the MIT license.
Can you run mini-sglang with Docker?
Yes. Its repository has a Docker setup: a Dockerfile, a Compose file or a published image.
Does mini-sglang need a GPU?
Yes, it needs a GPU.
Also in Model serving
See all 65| Rank | Project | Score |
|---|---|---|
| 1 | llama.cppC/C++ inference engine for GGUF LLMs and VLMs with an OpenAI-compatible server | 94 out of 100 |
| 2 | OllamaLocal model runner with CLI and REST API for open-weight LLMs | 92 out of 100 |
| 3 | LocalAISelf-hosted inference server with OpenAI-compatible API and pluggable backends | 91 out of 100 |
| #4 | vLLMInference engine serving LLMs with PagedAttention and an OpenAI-compatible API | 88 out of 100 |
| #5 | strataRuns a 125B-parameter MoE model on one consumer GPU plus system RAM | 88 out of 100 |