Adversarial Speaker Disentanglement Using Unannotated External Data for Self-supervised Representation-based Voice Conversion- Demo

ABSTRACT

Nowadays, recognition-synthesis-based methods have been quite popular with voice conversion (VC). By introducing linguistics features with good disentangling characters extracted from an automatic speech recognition (ASR) model, the VC performance achieved considerable breakthroughs. Recently, self-supervised learning (SSL) methods trained with a large-scale unannotated speech corpus have been applied to downstream tasks focusing on the content information, which is suitable for VC tasks. However, a huge amount of speaker information in SSL representations degrades timbre similarity and the quality of converted speech significantly. To address this problem, we proposed a high-similarity any-to-one voice conversion method with the input of SSL representations. We incorporated adversarial training mechanisms in the synthesis module using external unannotated corpora. Feed-forward transformer-based acoustic model and Hifi-GAN vocoder were used to generate the high-quality waveform. Two auxiliary discriminators were trained to distinguish whether a sequence of mel-spectrograms has been converted by the acoustic model and whether a sequence of content embeddings contains speaker information from external corpora. Experimental results show that our proposed method achieves comparable similarity and higher naturalness than the supervised method, which needs a huge amount of annotated corpora for training and is applicable to improve similarity for VC methods with other SSL representations as input.

Conversion Tasks

Our goal is to converte the timbre to target speaker while preserve the fine-grained prosody information contaioned in source waveform, which lead to high naturalness.

No annotated data were used in the whole training steps.

• Proposed - the proposed adversarial discriminator method.
• Baseline - the proposed method without Embedding Discriminator and Conversion Discriminator.

Below are a few audios samples from the target CSMSC spkr.

Below are a few demo audios.

Hubert Soft Unit experiment

Proposed method and Baseline method using Hubert Soft Unit as SSL representations.

Convect Features experiment

Proposed method and Baseline method using Convect Feature as SSL representations.

Hubert Raw experiment

Proposed method and Baseline method using Hubert Raw as SSL representations.