Deep Voice 3

Deep Voice 3, developed by Baidu, represents a significant leap forward in text-to-speech (TTS) technology, employing a fully-convolutional neural network architecture that focuses on scaling speech synthesis with convolutional sequence learning. This system demonstrates an exceptional balance of naturalness in speech synthesis, matching the quality of state-of-the-art neural TTS systems, while achieving up to ten times faster training speeds. Deep Voice 3's design allows for the handling of large datasets, training on over eight hundred hours of audio from more than two thousand speakers, making it highly versatile and scalable across different languages and voices (source).

Key features of Deep Voice 3 include its innovative use of residual convolutional layers to encode text into key and value vectors for an attention-based decoder. This decoder then predicts the mel-scale log magnitude spectrograms, corresponding to the output audio, with the aid of a converter network that predicts vocoder parameters for waveform synthesis. The system's architecture emphasizes the importance of text preprocessing, including normalization and the use of special characters to indicate pauses, which significantly improves speech quality by reducing mispronunciations and enhancing the natural flow of speech (source).

Furthermore, Deep Voice 3 distinguishes itself with its approach to handling multi-speaker scenarios through trainable speaker embeddings, and the flexibility to train models on either phoneme-only, character-only, or mixed character-and-phoneme inputs. This adaptability allows for improved pronunciation accuracy and the ability to correct mispronunciations using a phoneme dictionary, catering to the nuanced demands of real-world applications (source).

For more detailed insights into Deep Voice 3's architecture, including its encoder, decoder, and converter components, and its implications for the future of text-to-speech technology, you can refer to the comprehensive study available on arXiv.