#2525th of 42 in Gateways and observability
semantic-router
Policy-driven router that selects or combines models behind one API endpoint
- Stars
- 6.4k
- License
- Apache-2.0
- Last commit
- Nov 2026
- Last release
- Sep 2026
- Language
- Go
3,569 on 3 Apr 20266,360 on 30 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
vLLM Semantic Router sits between an agent harness and several configured model backends (local, private, cloud) and selects or combines models for each call based on a user-defined policy. Clients call one inference endpoint with a public model ID such as `vllm-sr/auto`. Policy can weigh quality, latency and cost, and keep calls within approved locations. Written in Go.
Who it is for: Teams routing agent traffic across multiple LLM backends by policy
Strengths
- One endpoint (vllm-sr/auto) fronts many local, private and cloud backends
- Policy can set quality, latency and cost priorities and location limits
- Publishes its own routing models on Hugging Face and a hallucination detection endpoint
- Hosted playground available; Apache-2.0 with recent commits and releases
Weaknesses
- README gives no RAM, GPU or port requirements
- Install is a curl-to-bash script; other methods documented only externally
- README is mostly news and links; configuration details live on the docs site
- Does not run the agent loop or tools; needs a separate agent harness
What it needs
- Docker
- Models: local, private and cloud backends
Quick answers
Is semantic-router open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Can you run semantic-router with Docker?
Yes. Its repository has a Docker setup: a Dockerfile, a Compose file or a published image.
Also in Gateways and observability
See all 42| Rank | Project | Score |
|---|---|---|
| 1 | OmniRouteLLM gateway routing one OpenAI-compatible endpoint across 372 providers | 96 out of 100 |
| 2 | new-apiGateway that unifies LLM providers behind OpenAI, Anthropic and Gemini APIs | 90 out of 100 |
| 3 | MLflowTracing, evals, prompt registry and AI gateway plus classic ML tracking | 89 out of 100 |
| #4 | LiteLLMOpenAI-format gateway and Python SDK for calling 100+ LLM providers | 87 out of 100 |
| #5 | freellmapiOpenAI-compatible gateway pooling free-tier LLM providers with automatic failover | 86 out of 100 |