What is OpenAI Voice Engine for Creating Custom Voices

OpenAI has launched a voice engine that uses text input and a single 15-second audio sample to generate natural-sounding speech that closely resembles the original speaker. The voice cloning tool model is available for preview and usage for a group of testers who agreed to follow safety guidelines. Anticipating the misuse of the synthetic voice technology, OpenAI plans to release it in an informed manner for a broader audience. 

The Voice Engine project was first launched in 2022 and can create emotive and realistic voices using only a single 15-second sample. Based on the results from the test on a small group of people, OpenAI sees Voice Engine potential to be used for good across various industries.

According to the official OpenAI Voice Engine document, it was developed in late 2022 and has been used to power the preset voices available in the text-to-speech API as well as ChatGPT Voice and Read Aloud. At the same time, we are taking a cautious and informed approach to a broader release due to the potential for synthetic voice misuse. We hope to start a dialogue on the responsible deployment of synthetic voices, and how society can adapt to these new capabilities. Based on these conversations and the results of these small-scale tests, we will make a more informed decision about whether and how to deploy this technology at scale.

Also Read: How Artists and Filmmakers Use Sora Video Tool: Watch Top Sora-Created Videos Released by OpenAI

How do I use the OpenAI Voice Engine?

• Providing reading assistance: to non-readers and children through natural-sounding, emotive voices representing a wider range of speakers than what’s possible with preset voices.
• Translating content: like videos and podcasts, so creators and businesses can reach more people around the world, fluently and in their voices. One early adopter of this is HeyGen, an AI visual storytelling platform that works with enterprise customers to create custom, human-like avatars for a variety of content, from product marketing to sales demos.
• Reaching global communities: by improving essential service delivery in remote settings. Dimagi is building tools for community health workers to provide a variety of essential services, such as counseling for breastfeeding mothers. To help these workers develop their skills, Dimagi uses Voice Engine and GPT-4 to give interactive feedback in each worker’s primary language, including Swahili or more informal languages like Sheng, a code-mixed language popular in Kenya.
• Supporting people who are non-verbal: such as therapeutic applications for individuals with conditions that affect speech and educational enhancements for those with learning needs. Livox, an AI alternative communication app, powers Augmentative & Alternative Communication (AAC) devices that enable people with disabilities to communicate. By using Voice Engine, they can offer people who are non-verbal unique and non-robotic voices across many languages.
• Helping patients recover their voice: for those suffering from sudden or degenerative speech conditions.

Also Read: OpenAI Entering AI Chip Manufacturing To Challenge Nvidia’s Dominance

What Are The Safety Measures While Using Voice Engine?

• Phasing out voice-based authentication as a security measure for accessing bank accounts and other sensitive information.
• Watermarking to trace the origin of any audio generated by the Voice Engine
• Exploring policies to protect the use of individuals’ voices in AI
• Proactive monitoring of how audio generated by the Voice Engine is being used
• Educating the public on the capabilities and limitations of AI technologies, including the possibility of deceptive AI content
• Clear disclosure to their audience that the voices they’re hearing are AI-generated
• Accelerating the development and adoption of techniques for tracking the origin of audiovisual content, so it’s always clear when you’re interacting with a real person or with an AI

Also Read: Why Tyler Perry Puts $800M Studio Expansion on hold after seeing OpenAI’s Sora 

How Does OpenAI Voice Engine Works: Reference Audios

1. Current voice

2. Reference audio

3. Generated audio

OpenAI is working closely with researchers, policymakers, and other people to see the challenges and possible outcomes of implementing Voice Engine to broader users.