LoginTry it Free

What Is Voice Cloning?

Voice cloning is creating a synthetic voice that sounds like a specific real person, from recordings of their speech. The clone can then say new sentences the person never recorded, including in other languages, which is how dubbing tools keep a speaker’s own voice in a translation.

Example
Sample:  42 seconds of Anna speaking English in her video
Text:    “Empezamos con tres clientes y una sala alquilada.”
Result:  the Spanish line, spoken in a voice that sounds like Anna

How it works

A speech model trained on many voices learns what makes one voice different: pitch, timbre, accent and rhythm. Given a sample of a new speaker, it captures those traits and applies them to any text it speaks. The words come from the text; only the sound of the voice comes from the sample.

Instant and professional clones

An instant clone is made in seconds from a short sample, often under a minute or two, and sounds close to the person on neutral speech. A professional clone is trained on much more audio, typically half an hour to several hours of clean recordings, and holds up better in emotion, shouting or whispering.

Consent and the law

A voice identifies a person. Under the GDPR, voice data processed to identify someone is biometric data, and services that clone voices, including this one, treat the samples that way: they need a lawful basis, normally explicit consent. Illinois’ BIPA covers voiceprints, Tennessee’s ELVIS Act protects a person’s voice from unauthorised imitation, and the EU AI Act requires synthetic audio to be disclosed as such. Cloning someone without permission can break several of these at once.

What it is used for

Dubbing a creator’s videos in their own voice, fixing a word in a recording without a re-take, narration for people who have lost their voice, and audiobooks read by the author. The same technology makes voice scams possible, which is why reputable tools ask for confirmation that you may use the voice.

Questions

How much audio do you need to clone a voice?

For an instant clone, from about 20 seconds to a couple of minutes of clear speech. Professional clones are trained on 30 minutes or more.

Is voice cloning legal?

Cloning your own voice, or one you have permission to use, is legal in most places. Cloning someone without consent can breach privacy, biometric and publicity laws, and using it to deceive is fraud.

Can a cloned voice speak another language?

Yes. Multilingual speech models can make the clone speak languages the person doesn’t speak, keeping the sound of their voice, though their accent may change.

Can people tell a cloned voice from the real one?

On short, neutral sentences often not. Over a longer piece, flat emotion, odd emphasis or mispronounced names usually give it away.

Read more

Related terms

  • Dubbing

    Dubbing is replacing the speech in a film, series or video with new recorded speech, usually a translation into another language, so viewers hear the story in their own language instead of reading subtitles. The new lines are timed to the picture, and the music and sound effects of the original are kept.

  • Voice-over translation

    Voice-over translation lays a spoken translation over the original audio, which stays audible at a lower volume underneath. Unlike dubbing, it does not try to replace the original voices or match lip movements: viewers hear that someone else is translating.

  • Dubbing script

    A dubbing script is the translated dialogue of a film or video, rewritten so that each line can be spoken in the time the original line takes on screen. It lists every line with the character who says it and its in and out timecodes. The adapted text is what voice actors, or an AI voice, read in the studio.

Updated: