LoginTry it Free

What Is Speaker Diarization?

Speaker diarization is the step in speech recognition that works out who spoke when. It splits a recording into segments by voice and labels them Speaker 1, Speaker 2 and so on, so an interview or meeting transcript shows each person's turns instead of one block of text.

Example
Without diarization:
So where do we start the launch plan starts with pricing okay.

With diarization:
Speaker 1 · 00:00:02  So where do we start?
Speaker 2 · 00:00:04  The launch plan starts with pricing.
Speaker 1 · 00:00:07  Okay.

How it works

The audio is cut into short pieces, each piece is turned into a voice fingerprint, and similar fingerprints are grouped into speakers. The transcript is then aligned with those groups, word by word.

Diarization is not identification

Diarization knows that two voices are different, not whose they are. Names come from you: rename Speaker 1 to Anna once and every line of hers follows.

When it struggles

People talking over each other, very short replies like yeah, similar voices, and one microphone far from some speakers. Telling the system how many people are in the recording helps it group voices correctly.

Questions

How many speakers can diarization handle?

Several, but accuracy drops as the number grows and turns get shorter. Two to five speakers in a clear recording work best.

Does it work in every language?

Yes. Diarization listens to voices, not words, so it works the same in any language.

Why are two people merged into one speaker?

Their voices are similar or they share one distant microphone. Set the speaker count, or fix the labels in the editor.

Read more

Related terms

  • Verbatim transcription

    Verbatim transcription writes down speech exactly as it was said: every um and uh, repeated words, false starts, stutters and often non-verbal sounds like [laughs]. Its opposite, clean verbatim, keeps the speaker's words and meaning but removes fillers and stumbles so the text reads smoothly.

  • Timecode

    Timecode is a label for every frame of video, written as hours:minutes:seconds:frames — 01:02:03:12 is frame 12 of the third second. Editors use it to find the same moment in every copy of a project. Subtitle files usually count time in milliseconds instead, which is where mix-ups start.

  • SDH subtitles

    SDH stands for Subtitles for the Deaf and Hard of Hearing. Besides the dialogue, SDH subtitles name off-screen speakers and describe sounds that matter — [door slams], [phone buzzing], [tense music] — so the video makes sense without audio. Streaming services label them, for example, English [SDH].

Updated: