ArchestackGitHub ↗ (opens in a new tab)

Open-source alternatives to Deepgram

whisper.cpp leads with 89, ahead of sherpa-onnx (78) and funasr (74). Updated from public GitHub data.

The highest-scoring open-source projects in Voice. They are picks, not exact replacements, so read the weaknesses before you switch. The same picks fit if you are leaving AssemblyAI or OpenAI Whisper API.

Most stars
whisper 111,701 stars
No GPU needed
moonshine runs without one
Try before you install
sherpa-onnx live demo
Open-source alternatives to Deepgram
RankProjectScore
1whisper.cppC/C++ port of OpenAI Whisper for local speech-to-text transcriptionVoice54.8k starsMIT89 out of 100
2sherpa-onnxOffline speech recognition, TTS, diarization and VAD runtime built on ONNX Runtime Live demo ↗ (opens in a new tab)Voice15.7k starsApache-2.078 out of 100
3funasrSpeech recognition toolkit with VAD, punctuation, speaker labels and streaming Live demo ↗ (opens in a new tab)Voice20.9k starsMIT74 out of 100
#4whisperlivekitReal-time streaming speech-to-text server with diarization and translationVoice11.2k starsApache-2.071 out of 100
#5moonshineOn-device speech-to-text and voice agent toolkit with streaming modelsVoice11.4k starscustom license69 out of 100
#6vibevoiceMicrosoft's open speech models for long-form ASR and streaming TTS Live demo ↗ (opens in a new tab)Voice55.1k starsMIT67 out of 100
#7faster-whisperPython Whisper speech-to-text library built on CTranslate2Voice26.4k starsMIT65 out of 100
#8whisperxBatched Whisper transcription with word-level timestamps and speaker diarization Live demo ↗ (opens in a new tab)Voice25k starsBSD-2-Clause61 out of 100
#9sensevoiceMultilingual speech recognition model with emotion and audio-event tags Live demo ↗ (opens in a new tab)Voice9.7k starsMIT56 out of 100
#10whisperMultilingual speech recognition and translation model with Python and CLI interfacesVoice111.7k starsMIT55 out of 100

Reviews

189 out of 100

whisper.cpp

C/C++ port of OpenAI Whisper for local speech-to-text transcription

54.8k starsMITlast commit Oct 2026

whisper.cpp runs OpenAI's Whisper speech recognition models in plain C/C++ on top of the ggml library, with a C-style API in whisper.h and a whisper-cli example for transcribing 16-bit WAV files. It runs on CPU or can offload to Metal, Core ML, CUDA, Vulkan, ROCm, OpenVINO, Ascend CANN, Moore Threads MUSA and AMD Ryzen AI NPUs. It supports integer quantization and voice activity detection, and builds for desktop, mobile, WebAssembly and Docker.

Strengths

  • No dependencies in core; zero memory allocations at runtime
  • Wide backend coverage: Metal, Core ML, CUDA, Vulkan, ROCm, OpenVINO, CANN, MUSA, NPU
  • Runs on iOS, Android, WebAssembly, Raspberry Pi, and Java bindings exist
  • Q5_0-style integer quantization shrinks models; base model needs about 388 MB RAM

Weaknesses

  • whisper-cli accepts only 16-bit WAV unless built with optional FFmpeg support
  • Core ML, OpenVINO and VitisAI encoders need a separate Python conversion step
  • Models must be in ggml format; downloaded via script, not bundled
  • Library and CLI only; no built-in web UI or multi-user features
  • GPU optional
  • Docker
  • Needs CMake, ffmpeg (optional), CUDA / ROCm / Vulkan / OpenVINO (optional backends)
  • Models: Whisper ggml (tiny, base, small, medium, large-v1/v2/v3, large-v3-turbo), Quantized ggml (e.g. q5_0)
Repository on GitHub ↗ (opens in a new tab)
278 out of 100

sherpa-onnx

Offline speech recognition, TTS, diarization and VAD runtime built on ONNX Runtime

Live demo ↗ (opens in a new tab)15.7k starsApache-2.0last commit Oct 2026

sherpa-onnx runs speech-to-text (streaming and non-streaming), text-to-speech, speaker diarization, speaker identification and verification, VAD, keyword spotting, speech enhancement and source separation locally. It is written in C++ with bindings for C, Python, JavaScript, Java, C#, Kotlin, Swift, Go, Dart, Rust and Pascal. Targets include Android, iOS, HarmonyOS, Windows, macOS, Linux, WebAssembly and several NPUs.

Strengths

  • Covers ASR, TTS, diarization, VAD, keyword spotting and enhancement in one library
  • Bindings for 12 languages, plus Flutter and Tauri demo apps
  • Runs on x64, x86, arm32, arm64 and riscv64, including Android, iOS and WebAssembly
  • NPU support for Rockchip, Qualcomm, Ascend, Axera and Intel; many pre-built APKs and browser demos

Weaknesses

  • Library and SDK, not an end-user app; you integrate it in code
  • Models are downloaded separately, and you choose among many ASR and TTS variants
  • No Docker or compose files detected
  • RAM and VRAM requirements are not stated in the README; they depend on the model
  • GPU optional
  • Needs ONNX Runtime
  • Models: ONNX, Zipformer, Paraformer, Whisper, SenseVoice
Live demo ↗ (opens in a new tab)Documentation ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
374 out of 100

funasr

Speech recognition toolkit with VAD, punctuation, speaker labels and streaming

Live demo ↗ (opens in a new tab)20.9k starsMITlast commit Nov 2026

FunASR is a Python toolkit that runs speech recognition checkpoints (Fun-ASR-Nano, SenseVoiceSmall, Paraformer) through an AutoModel API, with optional FSMN-VAD, punctuation and CAM++ speaker clustering. It also ships a CLI, an OpenAI-compatible transcription server (funasr-server), a streaming runtime, and llama.cpp/GGUF binaries for CPU and edge devices without Python.

Strengths

  • Offline and chunked streaming ASR (Paraformer-zh-streaming) in one toolkit
  • GGUF/llama.cpp binaries run SenseVoice on CPU, Vulkan or CUDA without Python
  • Composable pipeline: VAD, punctuation, anonymous speaker labels, emotion and event tags
  • OpenAI-style /v1/audio/transcriptions endpoint plus MCP server and CLI with SRT/JSON output

Weaknesses

  • Speaker labels are anonymous per recording, not known-person identification
  • Language coverage is per checkpoint; base Nano is only zh/en/ja plus Chinese dialects
  • Bundled server is unauthenticated and should stay on loopback
  • Runtime support differs by backend; GGUF weights are not usable from Python AutoModel
  • GPU optional
  • Needs PyTorch, torchaudio, FastAPI, uvicorn, vLLM (optional, Nano GPU serving)
  • Models: Fun-ASR-Nano, Fun-ASR-MLT-Nano, SenseVoiceSmall, Paraformer-zh, Whisper-large-v3
  • port 8000
Live demo ↗ (opens in a new tab)Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#471 out of 100

whisperlivekit

Real-time streaming speech-to-text server with diarization and translation

11.2k starsApache-2.0last commit Oct 2026

WhisperLiveKit is a Python server that transcribes microphone or browser audio in real time using simultaneous-decoding policies (SimulStreaming, LocalAgreement) instead of re-running Whisper on fixed chunks. It exposes a native WebSocket at /asr, an OpenAI-compatible transcription endpoint and a Deepgram-compatible WebSocket, and ships a web UI and a Chrome extension. Backends include Whisper, MLX Whisper, Voxtral, Qwen3-ASR, Canary and FunASR SenseVoice.

Strengths

  • Serves OpenAI-style /v1/audio/transcriptions, Deepgram-style WebSocket and native /asr WebSocket
  • Many backends: Whisper, MLX Whisper, Voxtral, Qwen3-ASR, Canary, SenseVoiceSmall
  • Optional Sortformer speaker diarization and translation to and from 200 languages
  • One shared engine serves multiple concurrent sessions with per-session language

Weaknesses

  • Heavy extras (voxtral-hf, vLLM stacks) conflict and need separate environments
  • Qwen3 causal mode is English-only; vLLM causal and Metal paths are experimental
  • Qwen3 streaming plans one realtime session per GPU and needs an explicit language
  • Canary on CPU is slow; GPU strongly recommended for larger backends
  • GPU optional
  • Docker + Compose
  • Needs PyTorch, NeMo (diarization, Canary), vLLM (optional Qwen3 backend)
  • Models: Whisper, MLX Whisper, Voxtral Mini, Qwen3-ASR, Canary-1b-v2
  • port 8000
Documentation ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#569 out of 100

moonshine

On-device speech-to-text and voice agent toolkit with streaming models

11.4k starscustom licenselast commit Aug 2026

Moonshine Voice is a C++ library with bindings for Python, JavaScript/WASM, iOS, Android, macOS, Linux, Windows and Raspberry Pi. It runs streaming speech-to-text models trained by the project, from tiny 1MB models up to ones the README says beat Whisper Large V3 on the Open ASR leaderboard. It also covers text to speech and conversational agents, all on-device without accounts or API keys.

Strengths

  • Runs fully on-device, no account or API keys required
  • One library across Python, WASM, mobile, desktop and Raspberry Pi
  • Streaming design does work while the user is still speaking
  • Model sizes range from about 1MB up to Whisper Large V3-class accuracy

Weaknesses

  • Legacy non-English non-streaming models use a non-commercial community license
  • Repo license detected as NOASSERTION although README states MIT
  • No Docker image or live demo link in the README
  • no GPU
  • Models: Moonshine speech-to-text models, Moonshine tiny 1MB models
Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#667 out of 100

vibevoice

Microsoft's open speech models for long-form ASR and streaming TTS

Live demo ↗ (opens in a new tab)55.1k starsMITlast commit Oct 2026

VibeVoice is a family of speech models built on 7.5 Hz continuous tokenizers and next-token diffusion. The ASR models transcribe up to 60 minutes of audio in one pass with speaker, timestamp and hotword support, plus a streaming variant and a CPU build. The TTS side includes a 0.5B real-time model with about 300 ms first-audio latency. The 1.5B long-form TTS code was removed from the repository after misuse.

Strengths

  • ASR handles 60 minutes in one pass, returning speaker, timestamps and text together
  • Streaming ASR, vLLM serving and ASR fine-tuning code are documented
  • BitNet CPU build shrinks ASR from 4.62 GB to 1.58 GB, RTF under 1 on 3+ threads
  • Realtime TTS is 0.5B with about 300 ms first audible latency and streaming text input

Weaknesses

  • VibeVoice-TTS code was removed; the 1.5B TTS model is listed as Disabled
  • README says it is for research only, not recommended for commercial use
  • Realtime TTS long-form output is about 10 minutes
  • README gives no RAM or VRAM requirements for the full models
  • GPU optional
  • Needs Hugging Face model weights, vLLM (optional, for serving)
  • Models: VibeVoice-ASR-7B, VibeVoice-ASR-Streaming, VibeVoice-ASR-BitNet, VibeVoice-Realtime-0.5B, VibeVoice-TTS-1.5B
Live demo ↗ (opens in a new tab)Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#765 out of 100

faster-whisper

Python Whisper speech-to-text library built on CTranslate2

26.4k starsMITlast commit Oct 2026

Reimplementation of OpenAI's Whisper on the CTranslate2 inference engine, installed with pip and used from Python. The README reports up to 4x faster transcription than openai/whisper with lower memory, with int8 quantization on CPU and GPU. It supports batched transcription, word-level timestamps, Silero VAD filtering, and Distil-Whisper checkpoints.

Strengths

  • README benchmark: 13 min audio in 1m03s fp16 on RTX 3070 Ti, 17s with batch_size=8
  • int8 quantization on CPU and GPU cuts memory (2926MB vs 4525MB VRAM for large-v2)
  • No system FFmpeg needed; audio decoded through PyAV bundled libraries
  • Converts any Transformers-compatible Whisper model, including fine-tuned ones, with ct2-transformers-converter

Weaknesses

  • Library only; no built-in server, UI or CLI (these come from community projects)
  • GPU needs separately installed cuBLAS for CUDA 12 and cuDNN 9
  • Older CUDA 11 or cuDNN 8 setups require downgrading ctranslate2
  • whisper.cpp was faster on the README's CPU fp32 benchmark (2m05s vs 2m37s)
  • GPU optional
  • Docker
  • Needs CTranslate2, PyAV, cuBLAS for CUDA 12 (GPU), cuDNN 9 (GPU), Silero VAD
  • Models: Whisper (CTranslate2 format), Distil-Whisper, Hugging Face Transformers Whisper conversion
Repository on GitHub ↗ (opens in a new tab)
#861 out of 100

whisperx

Batched Whisper transcription with word-level timestamps and speaker diarization

Live demo ↗ (opens in a new tab)25k starsBSD-2-Clauselast commit Oct 2026

WhisperX transcribes audio with Whisper through a faster-whisper backend, using VAD-based batching. It then aligns words to the audio with language-specific wav2vec2 models and labels speakers with pyannote diarization. It runs as a CLI or Python library, and the README claims 70x realtime with large-v2.

Strengths

  • Word-level timestamps via wav2vec2 forced alignment, not Whisper's utterance-level timing
  • Batched inference; README claims 70x realtime with large-v2 on GPU
  • large-v2 with beam_size=5 needs under 8GB GPU memory per README
  • Runs on CPU with --compute_type int8, including macOS

Weaknesses

  • Diarization needs a Hugging Face token and accepting the pyannote model agreement
  • Words like "2014." or "£13.60" cannot be aligned and get no timing
  • Overlapping speech is handled poorly; README says diarization is far from perfect
  • Each language needs its own wav2vec2 alignment model; unsupported ones need manual setup
  • GPU optional
  • Needs CUDA toolkit 12.8 (GPU only), ffmpeg, Hugging Face token (diarization), pyannote-audio, faster-whisper
  • Models: Whisper (faster-whisper/CTranslate2), wav2vec2 alignment models, pyannote speaker-diarization-community-1, silero-vad
Live demo ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#956 out of 100

sensevoice

Multilingual speech recognition model with emotion and audio-event tags

Live demo ↗ (opens in a new tab)9.7k starsMITlast commit Sep 2026

SenseVoice is a speech model that does ASR, spoken language ID, speech emotion recognition and audio event detection in one non-autoregressive model. The released SenseVoiceSmall checkpoint covers Mandarin, Cantonese, English, Japanese and Korean. It runs through FunASR in Python, as ONNX or Libtorch exports, as a FastAPI service, or as a GGUF binary on CPU.

Strengths

  • One model outputs transcript, language ID, emotion tag and audio-event tag
  • README reports over 5x faster than Whisper-Small in its benchmark setup
  • Multiple runtimes: FunASR Python, ONNX, Libtorch, FastAPI, and a GGUF binary without Python
  • Includes finetuning scripts and a bounded-memory long-audio script that needs no VAD

Weaknesses

  • Released checkpoint supports only 5 languages, not the 50+ cited for the research work
  • Speaker diarization needs separate FSMN-VAD and CAM++ models; accuracy not validated
  • Direct inference limited to 30 seconds; longer audio needs VAD or windowing
  • Event detection is weaker than specialized AED models, per the README
  • GPU optional
  • Docker + Compose
  • Needs funasr, ffmpeg, FSMN-VAD, CAM++ (diarization only)
  • Models: SenseVoiceSmall, ONNX, Libtorch, GGUF
  • port 50000
Live demo ↗ (opens in a new tab)Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#1055 out of 100

whisper

Multilingual speech recognition and translation model with Python and CLI interfaces

111.7k starsMITlast commit Aug 2026

Whisper is a Transformer sequence-to-sequence model for speech recognition, speech-to-English translation, and language identification. It ships as a pip package with a `whisper` command and a Python API, in six sizes from tiny (39M parameters) to large (1550M), plus a turbo variant. Code and weights are MIT licensed.

Strengths

  • Six model sizes (39M to 1550M parameters) trade accuracy for speed and VRAM
  • Handles transcription, translation to English and language identification in one model
  • Simple `whisper audio.mp3 --model turbo` CLI and a short Python API
  • Code and weights both under MIT license

Weaknesses

  • No built-in HTTP server or web UI; you wrap it yourself
  • The turbo model is not trained for translation and returns the original language
  • Requires ffmpeg, and possibly Rust for tiktoken if no prebuilt wheel exists
  • Accuracy varies widely by language; large model needs about 10 GB VRAM
  • GPU optional
  • Needs ffmpeg, PyTorch, tiktoken, Rust (if no prebuilt tiktoken wheel)
  • Models: tiny, base, small, medium, large
Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)