#3333rd of 57 in Model serving
kserve
Kubernetes inference platform for generative and predictive models via InferenceService
- Stars
- 6.2k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Sep 2026
- Language
- Go
5,258 on 3 Apr 20266,181 on 30 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
KServe is a Kubernetes custom-resource platform that deploys LLMs and classic ML models as InferenceService objects. It supports vLLM and llm-d backends with an OpenAI-compatible protocol for generative models, and TensorFlow, PyTorch, scikit-learn, XGBoost and ONNX for predictive ones. Canary rollouts, InferenceGraph pipelines, autoscaling and scale-to-zero are included, depending on the install mode. It is a CNCF incubating project.
Who it is for: Platform and MLOps engineers serving models on Kubernetes
Strengths
- Covers both LLM serving (vLLM, llm-d) and predictive frameworks in one platform
- OpenAI-compatible inference protocol for generative models
- Scale-to-zero, canary rollouts and InferenceGraph pipelines for predictive workloads
- CNCF incubating project; also available as a Kubeflow add-on
Weaknesses
- Requires a Kubernetes cluster; not a single-process local server
- Standard Kubernetes install lacks canary and scale-to-zero autoscaling; those need Knative
- Several install modes (Knative, ModelMesh, raw Kubernetes) add operational complexity
- README gives no hardware requirements or example ports
What it needs
- GPU optional
- Docker
- Needs Kubernetes, Knative (serverless mode), ModelMesh (optional)
- Models: vLLM, llm-d, Hugging Face, TensorFlow, PyTorch
Quick answers
Is kserve open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Can you run kserve with Docker?
Yes. Its repository has a Docker setup: a Dockerfile, a Compose file or a published image.
Does kserve need a GPU?
No. It runs on a CPU and can use a GPU when you have one.
Also in Model serving
See all 57| Rank | Project | Score |
|---|---|---|
| 1 | llama.cppC/C++ inference engine serving GGUF models over an OpenAI-compatible API | 93 out of 100 |
| 2 | OllamaLocal model runner with CLI and REST API for open-weight LLMs | 91 out of 100 |
| 3 | LocalAISelf-hosted inference server with OpenAI-compatible API and pluggable backends | 90 out of 100 |
| #4 | vLLMInference engine for Hugging Face LLMs with PagedAttention and continuous batching | 88 out of 100 |
| #5 | colibriC inference engine that streams MoE experts from disk to run huge models | 85 out of 100 |