Coqui AI TTS Russian Model Training Walkthrough

[iprovalo]

I ran a few training experiments on a Russian language ljspeech dataset (M-AI-Labs) using Coqui AI TTS. So far I have been training Glow TTS with MB Melgan and Hifigan vocoders. I will be adding the Tacotron and Fast Speech to the list later. While this is still a work-in-progress, here are some preliminary results which I would like to share for the comments and suggestions.

I documented the configs, as well as some key charts and metrics from the tensorboard along with some observations for nine jobs - two vocoders, and seven models.

Google Doc version, open for the comments.

My open questions:

1. Why is mailabs dataset style not working as well as ljspeech style?
2. Why is ljspeech dataset alignment not looking as smooth or diagonal as mailabs one? Is it the padding?
3. Curious why the batch size increase causes training slowdown. Should it be the opposite?
4. Why does the average loss in Run 9 for Hifigan look spiky?

Thank you!

[iprovalo]

Related to #1703

[iprovalo]

Here is a google drive folder with the samples:

• Tensorboard:
  tensorboard_glow_tts_run_12.wav
  tensorboard_mb_melgan_run_13.wav

• Inference based on the based best model:
  glow_tts_ru_ruslan.wav
  glow_tts_ru_ruslan_no_vocoder.wav

[MaksimSerg]

tensorboard_mb_melgan_run_13.wav - это Glow-tts + multiband_melgan ?

[iprovalo]

No, this one is just mb_melgan, no model.

[kin0303]

@iprovalo

Hi, I want to ask. How do you train the vocoder if you use multi-speaker? is it training using one of the speakers?

[iprovalo]

@BlackMamba1122 yes, that's how I would do it (pick one speaker). I am trying that for the Spanish dataset at the moment.

Adding @erogol to comment.

[madey83]

@iprovalo

durring your test did you train Hifigan vocoders on russian dataset or you used what is available on github page: https://github.com/jik876/hifi-gan

What was your final config to get the best audio? Can you share it?