#6666th of 81 in VoiceFor voice, speech and music
whisper-diarization
Whisper transcription with per-sentence speaker labels using NeMo diarization
- Stars
- 5.7k
- License
- BSD-2-Clause
- Last commit
- Aug 2026
- Language
- Jupyter Notebook
5,452 on 21 Apr 20265,673 on 10 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
A Python pipeline that transcribes audio with Whisper and labels the speaker of each sentence. It extracts vocals with Demucs, aligns timestamps with ctc-forced-aligner, then uses NeMo MarbleNet for VAD and TitaNet for speaker embeddings. It runs from the command line via diarize.py, and a Colab notebook is provided.
Who it is for: Engineers who need speaker-labelled transcripts from audio files
Strengths
- Combines Whisper, Demucs vocal separation, MarbleNet VAD and TitaNet embeddings in one script
- Forced alignment and punctuation realignment reduce speaker errors from timestamp drift
- Colab notebook available for trying it without a local setup
- CLI flags for model, language, device, batch size and disabling stem separation
Weaknesses
- Overlapping speech is not handled, per the README's known limitations
- Whisper and NeMo parameters are hardcoded in diarize.py and helpers.py
- diarize_parallel.py is experimental and wants 10 GB or more of VRAM
- Needs ffmpeg and Cython installed before the requirements will install
What it needs
- GPU optional
- Needs Python >= 3.10, ffmpeg, Cython, NVIDIA NeMo, Demucs, ctc-forced-aligner
- Models: Whisper (default medium.en), Faster Whisper, NeMo MarbleNet, NeMo TitaNet
Quick answers
Is whisper-diarization open source?
Yes. Its code is on GitHub under the BSD-2-Clause license.
Does whisper-diarization need a GPU?
No. It runs on a CPU and can use a GPU when you have one.
Also in Voice
See all 81| Rank | Project | Score |
|---|---|---|
| 1 | voicestudioDesktop app for voice cloning, dubbing, dictation and audiobooks | 98 out of 100 |
| 2 | handyOffline push-to-talk dictation app that pastes transcripts into any text field | 91 out of 100 |
| 3 | whisper.cppC/C++ speech-to-text engine running OpenAI Whisper models locally | 90 out of 100 |
| #4 | xiaozhi-esp32ESP32 firmware for a voice chatbot with MCP device and cloud tool control | 83 out of 100 |
| #5 | buzzOffline desktop app for transcribing and translating audio with Whisper | 83 out of 100 |