#1313th of 47 in Model serving
cog
Packages Python ML models into Docker images with a generated HTTP API
- Stars
- 9.5k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- Go
9,271 on 1 Apr 20269,490 on 21 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Cog builds a Docker image from a cog.yaml file and a Python class that defines a model's setup and run methods. It picks compatible CUDA, cuDNN, PyTorch, TensorFlow and Python versions, generates an OpenAPI schema from the Python type annotations, and serves a REST API from a Rust/Axum server. Images can be run on your own infrastructure or deployed to Replicate.
Who it is for: ML engineers shipping Python models as containerized HTTP services
Strengths
- Inputs and outputs are declared in Python types; OpenAPI schema is generated and validated
- Resolves CUDA, cuDNN, PyTorch, TensorFlow and Python version combinations automatically
- Same cog.yaml drives local runs (cog run, cog serve) and production builds (cog build)
- Output is a standard Docker image that runs anywhere Docker does
Weaknesses
- Requires Docker, plus Buildx if using Docker Engine instead of Docker Desktop
- No native Windows support; Windows 11 needs WSL 2
- Model code must follow the BaseRunner interface, so existing serving code needs porting
- Packages one model per container; README describes no multi-model routing or scheduling
What it needs
- GPU optional
- Needs Docker, Docker Buildx
- Models: PyTorch, TensorFlow, arbitrary Python models
- port 5000
Also in Model serving
See all 47| Rank | Project | Score |
|---|---|---|
| 1 | LocalAISelf-hosted inference server with OpenAI-compatible API and pluggable backends | 83 out of 100 |
| 2 | llama.cppC/C++ inference engine serving GGUF models over an OpenAI-compatible API | 81 out of 100 |
| 3 | OllamaLocal model runner with CLI and REST API for open-weight LLMs | 80 out of 100 |
| #4 | strataRuns a 125B-parameter MoE model on one consumer GPU plus system RAM | 79 out of 100 |
| #5 | vLLMInference engine for Hugging Face LLMs with PagedAttention and continuous batching | 77 out of 100 |