#4242nd of 57 in Model serving
tensorrt
Open-source plugins, ONNX parser and samples for NVIDIA TensorRT inference
- Stars
- 13.4k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- C++
12,797 on 10 Apr 202613,425 on 30 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
This repository holds the open-source parts of NVIDIA TensorRT 11.x: the plugin sources, the ONNX parser and sample applications. It is a subset of the GA release and builds with CMake against a separately downloaded TensorRT package. The Python package installs with `pip install tensorrt`, so building is only needed for the C++ components.
Who it is for: Engineers optimizing models for NVIDIA GPU inference, incl. Jetson
Strengths
- Build targets cover Linux x86-64, aarch64, Jetson Thor, DriveOS, QNX and Windows
- Import paths documented for ONNX, Torch-TensorRT, HuggingFace/Optimum and the network API
- Per-model support matrix covers LLM, vision, audio, diffusion and multimodal models
- Python package installs with a single pip command
Weaknesses
- Only a subset of TensorRT; the core engine is a separate GA download
- Requires an NVIDIA GPU and CUDA 12.9 or 13.4
- 11.x removed weakly-typed networks, implicit quantization, IPluginV2 and TREX, so migration is needed
- Not an HTTP model server; it is a library, so serving needs other tooling
What it needs
- GPU required
- Docker
- Needs TensorRT 11.4.0.106 GA build, CUDA 12.9 or 13.4, CMake >= 3.31, Python 3.10-3.14, cuDNN 8.9 (optional), NCCL (optional)
- Models: ONNX, Torch-TensorRT, HuggingFace/Optimum
Quick answers
Is tensorrt open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Can you run tensorrt with Docker?
Yes. Its repository has a Docker setup: a Dockerfile, a Compose file or a published image.
Does tensorrt need a GPU?
Yes, it needs a GPU.
Also in Model serving
See all 57| Rank | Project | Score |
|---|---|---|
| 1 | llama.cppC/C++ inference engine serving GGUF models over an OpenAI-compatible API | 93 out of 100 |
| 2 | OllamaLocal model runner with CLI and REST API for open-weight LLMs | 91 out of 100 |
| 3 | LocalAISelf-hosted inference server with OpenAI-compatible API and pluggable backends | 90 out of 100 |
| #4 | vLLMInference engine for Hugging Face LLMs with PagedAttention and continuous batching | 88 out of 100 |
| #5 | colibriC inference engine that streams MoE experts from disk to run huge models | 85 out of 100 |