Voice conversion using deep neural networks

Abstract

This thesis focuses on techniques to improve the performance of voice conversion. Voice conversion modifies the recorded speech of a source speaker towards a given target speaker. The resultant speech is to sound like the target speaker with the language content unchanged. This technology has been applied to create personalized voice in text-to-speech or virtual avatar, speech-to-singing synthesis or spoofing attacks in speaker verification systems. To perform voice conversion, the usual approach is to create a conversion functions which is applied on the source speaker’s speech features such as timbre and prosodic features, to generate the corresponding target features. In this past decade, most of voice conversion researches had focused on spectral mapping, i.e. conversion of the features representing the timbre characteristics in a frame by frame manner. In chapter 3, we investigate a comprehensive approach to train the conversion function using DNN which considers both timbre and prosodic features simultaneously. For better modelling, we have used high-dimension spectral features. However, this further worsen the ability to robustly train a DNN which typically requires large amount of training data. To overcome the issue of limited training data, we propose a new pretraining process using autoencoder. The experimental results show the proposed comprehensive framework with pretraining performs better than conventional voice conversion systems including the state-of-the-art GMM-based system. The technique introduced in chapter 3 only learns a DNN system to convert between a pair of speaker. To reduce the need for parallel training data of new speaker pair, in chapter 4 we examine a novel DNN adaptation technology for voice conversion by including two bias vector representing both source and target speaker. By this configuration, new speaker pair conversion are archived. Our preliminary results show that conversion to new target speakers’ voices could be achieved.