#5353rd of 70 in Agent frameworks and SDKs
vision-agents
Python framework for real-time voice and video agents with vision models
Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 8.2k
- License
- Apache-2.0
- Last commit
- Sep 2026
- Last release
- Aug 2026
- Language
- Python
7,403 on 1 Apr 20268,183 on 21 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Vision Agents is a Python library for building agents that watch and listen to live video calls over WebRTC. It chains processors such as YOLO or Roboflow with realtime LLMs like Gemini Live and OpenAI Realtime. It also adds STT, TTS, turn detection, MCP tool calling, RAG, and Twilio or Telnyx phone calls. It is built by Stream and uses Stream's edge network by default.
Who it is for: Python developers building real-time video or voice agents
Strengths
- Pluggable processor pipeline for YOLO, Roboflow, or custom PyTorch/ONNX models alongside LLMs
- Many integrations: OpenAI, Gemini, xAI, Deepgram, ElevenLabs, Cartesia, Kokoro, Twilio, Telnyx
- Guides for HTTP server, Prometheus metrics, horizontal scaling and Kubernetes deployment
- Uses native OpenAI, Gemini and Claude SDK methods rather than a wrapper API
Weaknesses
- Quickstart needs a Stream API key; free tier is 333,000 participant minutes per month
- Video understanding struggles with small text and degrades after about 30 seconds
- Realtime models need audio or text to respond; video alone does not trigger output
- No Docker or compose files in the repo
What it needs
- GPU optional
- Needs Stream API key, LLM provider API (OpenAI, Gemini, etc.), uv
- Models: OpenAI, Gemini, xAI, OpenRouter, Hugging Face
Also in Agent frameworks and SDKs
See all 70| Rank | Project | Score |
|---|---|---|
| 1 | ppt-masterAgent skill that turns documents into natively editable PowerPoint decks | 94 out of 100 |
| 2 | langchainPython framework for building LLM agents and apps from interoperable components | 89 out of 100 |
| 3 | browser-usePython library that lets an LLM agent control a web browser | 89 out of 100 |
| #4 | openai-agents-pythonPython SDK for multi-agent workflows with handoffs, guardrails and tracing | 89 out of 100 |
| #5 | mastraTypeScript framework for building AI agents and workflows | 89 out of 100 |