#77th of 36 in Gateways and observability
llm-gateway
OpenAI-compatible gateway routing requests across providers, subscriptions, local models
Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 7.6k
- License
- MIT
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- TypeScript
4,067 on 1 Apr 20267,595 on 21 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Manifest LLM Gateway exposes one OpenAI-compatible endpoint in front of 35 built-in provider connections, 18 subscription flows, local runtimes and custom endpoints. Sending "model": "auto" lets it pick the model. It logs full request bodies for successes and errors, tracks spend with limits and notifications, and falls back to other models when a query fails.
Who it is for: Teams routing agent traffic across many LLM providers and subscriptions
Ranked among open-source alternatives to OpenRouter (#5 of 10).
Strengths
- Supports 35 provider connections plus any custom /v1/chat/completions or /v1/messages endpoint
- Can route through 18 subscription flows (e.g. ChatGPT, Claude, Copilot), not only API keys
- Connects local runtimes: Ollama, LM Studio (port 1234), llama.cpp (port 8080)
- Full body logs, cost tracking, limits and model fallback on failure
Weaknesses
- Self-hosting is Docker-only; the npm-based path is no longer supported
- Volume-backed templates are single-instance; scaling out needs S3-compatible storage
- Needs a database and recording storage; Postgres appears in the hosted templates
- RAM and CPU requirements are not stated
What it needs
- no GPU
- Docker + Compose
- Needs Docker, PostgreSQL, S3-compatible storage (for scaled deployments)
- Models: OpenAI, Anthropic, Google Gemini, AWS Bedrock, OpenRouter
- port 2099
Also in Gateways and observability
See all 36| Rank | Project | Score |
|---|---|---|
| 1 | LiteLLMOpenAI-format gateway and Python SDK for calling 100+ LLM providers | 90 out of 100 |
| 2 | OmniRouteLLM gateway routing one OpenAI-compatible endpoint across 372 providers | 87 out of 100 |
| 3 | new-apiGateway that unifies LLM providers behind OpenAI, Anthropic and Gemini APIs | 83 out of 100 |
| #4 | freellmapiOpenAI-compatible router that pools free LLM provider tiers behind one endpoint | 80 out of 100 |
| #5 | LangfuseTracing, prompt management and evals for LLM apps on ClickHouse | 75 out of 100 |