Self-hosted LLM gateway and agent router

OmniRoute leads with a score of 96. Updated from public GitHub data.

Full category: Gateways and observability.

Which one?

1st pickOmniRoute
LLM gateway routing one OpenAI-compatible endpoint across 372 providers
Score 96 · 88.5k stars · MIT
Switch tonew-api
Gateway that unifies LLM providers behind OpenAI, Anthropic and Gemini APIs
as a strong alternative · Score 91 · 52.1k stars · AGPL-3.0
Switch toLiteLLM
OpenAI-format gateway and Python SDK for calling 100+ LLM providers
as a strong alternative · Score 88 · 63.5k stars · custom license
Self-hosted LLM gateway and agent router
RankProjectAdoptionFreshnessMaintenanceAgent-readyScore
1OmniRouteLLM gateway routing one OpenAI-compatible endpoint across 372 providers88.5k starsMITlast commit Nov 202699100977096 out of 100
2new-apiGateway that unifies LLM providers behind OpenAI, Anthropic and Gemini APIs52.1k starsAGPL-3.0last commit Nov 2026known advisory in latest version93100857091 out of 100
3LiteLLMOpenAI-format gateway and Python SDK for calling 100+ LLM providers63.5k starscustom licenselast commit Nov 202696100873088 out of 100

Momentum, Verified build, Docs and Privacy are not measured yet; their weight goes to the signals shown. A dash means the signal is not scored for that kind of project. Hover a number for its rating in words.

Reviews

196 out of 100

OmniRoute

LLM gateway routing one OpenAI-compatible endpoint across 372 providers

88.5k starsMITlast commit Nov 2026

OmniRoute is a TypeScript gateway that exposes one endpoint at localhost:20128/v1 and routes requests to 372 providers, including many free tiers, with automatic fallback between them. It accepts OpenAI, Claude, Gemini and Responses API formats, and a dashboard handles keys and free-tier quota tracking. Users bring their own provider accounts; the README's free-token figures are estimates, not a grant.

Strengths

  • Single /v1 endpoint covers 372 providers with automatic fallback to healthy targets
  • Dashboard tracks free-tier pools and quota use at /dashboard/free-tiers
  • Works with coding CLIs such as Claude Code, Codex, Cline and Cursor via one config
  • Installable via npm, Docker or Electron; MIT licensed; includes MCP and A2A support

Weaknesses

  • Free-token headline is an estimate; provider limits and terms change and can lower it
  • Providers flagged tos: avoid are excluded from automatic routing by default
  • Fresh install may have no eligible route until you connect a provider account
  • Large surface (372 providers, 19 routing strategies) makes setup and review heavy
  • no GPU
  • Docker + Compose
  • Needs Provider accounts or API keys, Node.js (npm) or Docker
  • Models: OpenAI API, Claude API, Gemini API, Responses API
  • port 20128
291 out of 100

new-api

Gateway that unifies LLM providers behind OpenAI, Anthropic and Gemini APIs

52.1k starsAGPL-3.0last commit Nov 2026known advisory in latest version

New API is a Go and Gin gateway that connects upstream providers such as OpenAI, Anthropic, Gemini, Azure OpenAI, AWS Bedrock, Vertex AI, DeepSeek and Qwen. It exposes them through OpenAI Chat/Responses, Anthropic Messages and Gemini endpoints. A web console handles channel routing, users, quotas, pricing and usage logs. It runs on SQLite by default, or on MySQL or PostgreSQL with optional Redis.

Strengths

  • Converts between OpenAI, Anthropic Messages and Gemini request formats via RelayKit
  • Channel priorities, weights, retries and multiple upstream keys per channel
  • Per-user quotas, subscriptions, usage logs and expression-based pricing
  • Starts on SQLite with one container; Compose file adds PostgreSQL and Redis

Weaknesses

  • AGPL-3.0 with extra terms requiring attribution and a visible project link in modified UIs
  • Protocol conversion is not exact; tools and fields may not map across protocols
  • Multi-node setups need shared database, secrets and Redis, or rate limits count per node
  • Public or resale use brings filing and compliance duties the README leaves to the operator
  • no GPU
  • Docker + Compose
  • Needs SQLite, MySQL >= 5.7.8 or PostgreSQL >= 9.6, Redis (optional), ClickHouse (optional, log database)
  • Models: OpenAI, Anthropic, Google Gemini, Azure OpenAI, AWS Bedrock
  • port 3000
388 out of 100

LiteLLM

OpenAI-format gateway and Python SDK for calling 100+ LLM providers

63.5k starscustom licenselast commit Nov 2026

LiteLLM translates requests in the OpenAI format to more than 100 LLM providers, and runs either as a Python SDK or as a proxy server. The proxy adds virtual keys, spend tracking, guardrails, load balancing and an admin dashboard. It also fronts A2A agents and MCP servers, and the README cites 8ms P95 latency at 1k RPS.

Strengths

  • Covers chat, responses, embeddings, images, audio, batches and rerank endpoints across many providers
  • Works as an in-process Python SDK or a standalone proxy server
  • Proxy includes virtual keys, spend tracking, guardrails and load balancing
  • Also fronts MCP servers and A2A agents through the same gateway

Weaknesses

  • Endpoint support varies per provider; the README table shows many gaps
  • License is detected as NOASSERTION; check the repo's license files before use
  • An Enterprise tier exists; the README excerpt does not say which features are paid
  • Two SDK packages (litellm, litellm-core) overlap and cannot be installed together
  • no GPU
  • Models: OpenAI, Anthropic, Gemini, AWS Bedrock, Azure
  • port 4000