#2626th of 37 in Model serving
openvino
Inference toolkit for converting and running models on Intel CPUs, GPUs and NPUs
Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 11k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- C++
9,893 on 31 Mar 202611,032 on 20 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
OpenVINO converts models from PyTorch, TensorFlow, ONNX, TensorFlow Lite, PaddlePaddle and JAX/Flax and runs them on x86 and ARM CPUs, Intel integrated and discrete GPUs, and Intel NPUs. It has C++, Python, C and NodeJS APIs, with a separate GenAI repo for LLM, speech and image pipelines. Quantization (NNCF) and serving (OpenVINO Model Server) are separate repos.
Who it is for: Engineers deploying models on Intel CPUs, GPUs or NPUs, edge to cloud
Strengths
- Converts models from several frameworks, so the original framework is not needed at deploy time
- One API targets x86 and ARM CPUs, Intel GPUs and Intel NPUs
- APIs in C++, Python, C and NodeJS; installs with pip install openvino
- Integrates with Optimum Intel, ONNX Runtime, vLLM, LangChain, LlamaIndex and Keras 3
Weaknesses
- GPU and NPU support is limited to Intel hardware
- Core library is not an HTTP server; serving needs OpenVINO Model Server
- GenAI pipelines and quantization live in separate repos from the core
- Collects usage telemetry by default; opting out requires running a command
What it needs
- GPU optional
- Models: PyTorch, TensorFlow, ONNX, TensorFlow Lite, PaddlePaddle
Also in Model serving
See all 37| Rank | Project | Score |
|---|---|---|
| 1 | LocalAISelf-hosted inference server with OpenAI-compatible API and pluggable backends | 83 out of 100 |
| 2 | llama.cppC/C++ inference engine serving GGUF models over an OpenAI-compatible API | 81 out of 100 |
| 3 | OllamaLocal model runner with CLI and REST API for open-weight LLMs | 80 out of 100 |
| #4 | vLLMInference engine for Hugging Face LLMs with PagedAttention and continuous batching | 77 out of 100 |
| #5 | strataRuns a 125B-parameter MoE model on one consumer GPU plus system RAM | 77 out of 100 |