xiaozhi-esp32 vs whisper.cpp
Two of the top projects in Voice, side by side: who leads on each score part and who each one suits, then setup, license, activity and what each review found.
xiaozhi-esp32
ESP32 firmware for a voice chatbot with MCP device and cloud tool control
whisper.cpp
C/C++ port of OpenAI Whisper for local speech-to-text transcription
Verdict
xiaozhi-esp32 ranks higher, 80 to 78. Of the 3 score parts we measure for both, xiaozhi-esp32 leads on none, whisper.cpp on 1 and 2 are even.
- Adoptionwhisper.cpp leads by 11 points, 88 to 77.
- FreshnessEven at 100.
- MaintenanceWe do not score xiaozhi-esp32 on this; whisper.cpp has 97.
- Easy to runWe do not score xiaozhi-esp32 on this; whisper.cpp has 50.
- Agent-readyEven at 30.
Pick xiaozhi-esp32 if you are
Hardware hobbyists and embedded engineers building ESP32 voice assistants
- Supports 138 board directories and 171 release variants across ESP32, C3, C5, C6, S3 and P4
- Documented WebSocket and MQTT+UDP protocols let third-party servers and clients interoperate
Pick whisper.cpp if you are
Developers embedding offline speech-to-text in apps, servers or devices
- No dependencies in core; zero memory allocations at runtime
- Wide backend coverage: Metal, Core ML, CUDA, Vulkan, ROCm, OpenVINO, CANN, MUSA, NPU
| What we compare | xiaozhi-esp32 | whisper.cpp |
|---|---|---|
| Score parts, out of 100 | ||
| Adoption | 77, popular | 88, widely used |
| Freshness | 100, active | 100, active |
| Maintenance | Not scored for this kind | 97, healthy |
| Easy to run | Not scored for this kind | 50, easy |
| Agent-ready | 30, minimal | 30, minimal |
| Facts from GitHub and the README | ||
| Stars | 31.7k | 54.5k |
| License | MIT (permissive) | MIT (permissive) |
| Last commit | Oct 2026 | Oct 2026 |
| Last release | Oct 2026 | Oct 2026 |
| Language | C++ | C++ |
| Docker | No | Yes |
| GPU | Not needed | Optional |
| arm64 or Apple Silicon | Not stated | Mentioned |
xiaozhi-esp32
C++ firmware that turns ESP32 boards into a voice chatbot. It does offline wake-word detection with ESP-SR and streams Opus audio over WebSocket or MQTT+UDP to a server that handles ASR, LLM and TTS. MCP is used on the device (speaker, LED, servo, GPIO) and in the cloud (smart home, email, search). It connects to the hosted xiaozhi.me server by default, and self-hosted server projects are listed.
Who it is for: Hardware hobbyists and embedded engineers building ESP32 voice assistants
Strengths
- Supports 138 board directories and 171 release variants across ESP32, C3, C5, C6, S3 and P4
- Documented WebSocket and MQTT+UDP protocols let third-party servers and clients interoperate
- Offline wake-word detection with customizable wake words; Wi-Fi, Ethernet and Cat.1 4G networking
- Prebuilt firmware can be flashed without setting up a development environment
Weaknesses
- Models do not run on the device; a server is needed for ASR, LLM and TTS
- Default setup depends on the hosted xiaozhi.me service and an account
- Requires ESP-IDF 6.0.1 or later; 5.x is unsupported and the v6.1 tag has a reported C3 Wi-Fi bug
- Beginner tutorials are on Feishu and bilibili, mostly in Chinese
- no GPU
- Needs ESP-IDF 6.0.1+, ESP32 board, xiaozhi.me or compatible server
- Models: Qwen, DeepSeek, Realtime end-to-end voice models
whisper.cpp
whisper.cpp runs OpenAI's Whisper speech recognition models in plain C/C++ on top of the ggml library, with a C-style API in whisper.h and a whisper-cli example for transcribing 16-bit WAV files. It runs on CPU or can offload to Metal, Core ML, CUDA, Vulkan, ROCm, OpenVINO, Ascend CANN, Moore Threads MUSA and AMD Ryzen AI NPUs. It supports integer quantization and voice activity detection, and builds for desktop, mobile, WebAssembly and Docker.
Who it is for: Developers embedding offline speech-to-text in apps, servers or devices
Strengths
- No dependencies in core; zero memory allocations at runtime
- Wide backend coverage: Metal, Core ML, CUDA, Vulkan, ROCm, OpenVINO, CANN, MUSA, NPU
- Runs on iOS, Android, WebAssembly, Raspberry Pi, and Java bindings exist
- Q5_0-style integer quantization shrinks models; base model needs about 388 MB RAM
Weaknesses
- whisper-cli accepts only 16-bit WAV unless built with optional FFmpeg support
- Core ML, OpenVINO and VitisAI encoders need a separate Python conversion step
- Models must be in ggml format; downloaded via script, not bundled
- Library and CLI only; no built-in web UI or multi-user features
- GPU optional
- Docker
- Needs CMake, ffmpeg (optional), CUDA / ROCm / Vulkan / OpenVINO (optional backends)
- Models: Whisper ggml (tiny, base, small, medium, large-v1/v2/v3, large-v3-turbo), Quantized ggml (e.g. q5_0)
More in Voice
Building with these?
We help teams pick an open-source AI stack and run it in production. Client work never changes a rank.