How Does Voice Synthesis Work? Examples and Industry Applications

Voice synthesis is not just a buzzword. It is a powerful technology with the potential to drastically change the way we produce and consume content, whether it’s video, audio, or in any other form.

More exactly, voice synthesis (also known as voice cloning, speech synthesis, and voice conversion technology) is the artificial simulation of human speech using a software called speech synthesizer. In simple words, through voice synthesis software, a person’s voice can be replaced with someone else’s.

It’s based on two kinds of technologies, text-to-speech (TTS) and speech-to-speech (SST). SST is more modern and preferred over TTS because it provides a higher level of authenticity and control of emotions.

Diving deeper into a definition of voice synthesis, we could say it’s a branch of synthetic media, a wide term referring to media content generated or modified through Artificial Intelligence (Machine Learning and Deep Learning). Synthetic media also includes video synthesis, image synthesis, music synthesis, or interactive media synthesis. It’s important to mention that voice synthesis was created to help people with visual and speech disabilities or those who struggle with reading.

But how did voice synthesis appear? In 1779, Russian professor Christian Kratzenstein built some acoustic resonators that mimicked the human vocal tract when activated by vibrating reeds. Then, in 1838, Willis discovered the connection between the organization of the vocal tract and specific vowel sounds. In 1939, at New York World’s Fair, Homer Dudley presented the first electrical speech synthesizer, VODER (Voice Operating Demonstrator). In 1968, Japanese Noriko Umeda built the first full text-to-speech system for English.

Afterward, speech synthesis evolved significantly. Nowadays, this technology is used for a variety of industries. For example, Respeecher was founded with the mission to clone human speech and swap voices to provide content creators throughout the world access to an effective and flexible way of creating audio content.

Main industry applications of voice synthesis

We created a list for you with the main applications of voice synthesis technology, since there are plenty of business verticals that could benefit from it:

1. The film and TV industries

From dubbing an actor’s voice in post-production to bringing back the voice of an actor who passed away, voice cloning is a crucial technology for filmmakers and TV producers.

More specifically, they will:

• no longer need to waste time and resources with a top actor by bringing him back to the studio to record again some lines;
• be able to add by themselves some lines after filming, without calling an actor back into the studio;
• be able to resurrect past voices and to bring them into the project;
• be able to record any voice in any language in case they have an overseas audience;
• be able to replicate children’s voices without the actual need of working with kids.

As you can see, the biggest advantage here is the flexibility brought for all the parties involved.

2. The gaming industry

Voice conversion is a huge help also for game developers. They can make changes at any point in the game development process because they’ll be able to create the audio content they need.

In case of larger games, with a lot of voices, Respeecher enables creators to record more voices per actor, making this process a lot easier. So, using voice synthesis, video games could be created faster and cost-effectively.

3. The advertising industry

Any company needs creative ads to increase its level of brand awareness and to stand out from the crowd. Voice synthesis enables an advertiser to replicate any voice to obtain the perfect commercial and to target specific audiences.

For example, with Respeecher, advertisers, but also filmmakers and TV producers could use singing voice synthesis, a technology that supports more emotions and can sing. Learn more from this demo:

4. Audiobooks and podcasts

Voice cloning technology is suitable also for recording audiobooks and podcasts using a famous voice, without making an actor spend hours in the recording studio, or to add a historical voice to the project, from an actor who is no longer with us.

Also, with Respeecher, anybody could narrate their book, without being professionally trained.

5. Other business industries

There is no doubt that in the future voice conversion will be also used in many other business industries, such as call-centers for people for whom English is not a primary language, or for enabling a robotic operator to sound more natural and human.

And let’s not forget about the benefits of this technology in the healthcare industry: it can help people with speaking problems due to severe illnesses (such as strokes, neuromuscular problems, and other medical conditions) to replicate their voices and speak naturally.

Therefore, an important aspect to mention here is that voice cloning has the potential to change even the quality of some people’s lives.

Conclusion

As any other technology, if used with bad intentions, voice synthesis could have harmful applications, such as creating fake news, frauds, scams, or making people believe someone said something they didn’t.

This is exactly why we created a set of principles, meant to support us in creating ethical voice synthesis, as part of Respeecher’s mission.

We consider it very important to educate people about the existence of voice synthesis and how to identify fake news. We think all the companies that produce this kind of technology should respect a strict ethics code.

At Respeecher, our goals are to:

• Educate the public about the capabilities of synthetic speech technology;
• Develop automatic detection algorithms that can detect synthetic speech even if it has not been watermarked by us;
• Work with gatekeepers of content such as Facebook and YouTube to limit the harm of voice cloning.

This article was initially published by Respeecher as a guest post on Business2News.