[Bug] Bad result of YourTTS trainning

[ggbangyes]

Describe the bug

Training the YourTTS model https://github.com/coqui-ai/TTS/blob/dev/recipes/vctk/yourtts/train_yourtts.py
on VCTK dataset, but getting a bad speech generation result.
generated file even don't speak English.
The model seems like do not converge.

I use one single RTX3090 train for 20 hours.

�

To Reproduce

https://github.com/coqui-ai/TTS/blob/dev/recipes/vctk/yourtts/train_yourtts.py

Expected behavior

get a relatively good-quality of voice

Logs

[1m   --> STEP: 1039/1243 -- GLOBAL_STEP: 34600�[0m
     | > loss_disc: 2.54361  (2.52598)
     | > loss_disc_real_0: 0.21437  (0.18234)
     | > loss_disc_real_1: 0.18707  (0.21857)
     | > loss_disc_real_2: 0.19539  (0.22308)
     | > loss_disc_real_3: 0.20699  (0.23064)
     | > loss_disc_real_4: 0.21294  (0.22657)
     | > loss_disc_real_5: 0.18105  (0.21795)
     | > loss_0: 2.54361  (2.52598)
     | > grad_norm_0: 48.09296  (43.48993)
     | > loss_gen: 2.17479  (2.24237)
     | > loss_kl: 2.19664  (2.13727)
     | > loss_feat: 4.84878  (4.85987)
     | > loss_mel: 19.80973  (19.19988)
     | > loss_duration: 1.70732  (1.70623)
     | > loss_1: 30.73726  (30.14562)
     | > grad_norm_1: 224.25027  (240.00607)
     | > current_lr_0: 0.00020 
     | > current_lr_1: 0.00020 
     | > step_time: 1.62350  (1.68084)
     | > loader_time: 0.01860  (0.01940)

Environment

"CUDA": {
        "GPU": [
            "NVIDIA GeForce RTX 3090",
            "NVIDIA GeForce RTX 3090"
        ],
        "available": true,
        "version": "11.7"
    },
    "Packages": {
        "PyTorch_debug": false,
        "PyTorch_version": "2.0.1+cu117",
        "TTS": "0.14.3",
        "numpy": "1.21.6"
    },
    "System": {
        "OS": "Linux",
        "architecture": [
            "64bit",
            "ELF"
        ],
        "processor": "x86_64",
        "python": "3.9.13",
        "version": "#161~18.04.1-Ubuntu SMP Fri Feb 10 15:55:22 UTC 2023"
    }
}

Additional context

No response

[Bestom927]

me having same question too, still unclear now?

[pengzhendong]

Same question.

NVIDIA GeForce RTX 3090 50w steps:

tts.tts_to_file(text="It took me quite a long time to debug.", speaker="VCTK_p277", file_path=wav)

download.mp4

[nicemanis]

Judging from the audio sample, I would say there is an issue with the sampling rate.

[Value-Jack]

have you solve this problem finally?

[Value-Jack]

Describe the bug

Training the YourTTS model https://github.com/coqui-ai/TTS/blob/dev/recipes/vctk/yourtts/train_yourtts.py on VCTK dataset, but getting a bad speech generation result. generated file even don't speak English. The model seems like do not converge.

I use one single RTX3090 train for 20 hours.

�

To Reproduce

https://github.com/coqui-ai/TTS/blob/dev/recipes/vctk/yourtts/train_yourtts.py

Expected behavior

get a relatively good-quality of voice

Logs

[1m   --> STEP: 1039/1243 -- GLOBAL_STEP: 34600�[0m
     | > loss_disc: 2.54361  (2.52598)
     | > loss_disc_real_0: 0.21437  (0.18234)
     | > loss_disc_real_1: 0.18707  (0.21857)
     | > loss_disc_real_2: 0.19539  (0.22308)
     | > loss_disc_real_3: 0.20699  (0.23064)
     | > loss_disc_real_4: 0.21294  (0.22657)
     | > loss_disc_real_5: 0.18105  (0.21795)
     | > loss_0: 2.54361  (2.52598)
     | > grad_norm_0: 48.09296  (43.48993)
     | > loss_gen: 2.17479  (2.24237)
     | > loss_kl: 2.19664  (2.13727)
     | > loss_feat: 4.84878  (4.85987)
     | > loss_mel: 19.80973  (19.19988)
     | > loss_duration: 1.70732  (1.70623)
     | > loss_1: 30.73726  (30.14562)
     | > grad_norm_1: 224.25027  (240.00607)
     | > current_lr_0: 0.00020 
     | > current_lr_1: 0.00020 
     | > step_time: 1.62350  (1.68084)
     | > loader_time: 0.01860  (0.01940)

Environment

"CUDA": {
        "GPU": [
            "NVIDIA GeForce RTX 3090",
            "NVIDIA GeForce RTX 3090"
        ],
        "available": true,
        "version": "11.7"
    },
    "Packages": {
        "PyTorch_debug": false,
        "PyTorch_version": "2.0.1+cu117",
        "TTS": "0.14.3",
        "numpy": "1.21.6"
    },
    "System": {
        "OS": "Linux",
        "architecture": [
            "64bit",
            "ELF"
        ],
        "processor": "x86_64",
        "python": "3.9.13",
        "version": "#161~18.04.1-Ubuntu SMP Fri Feb 10 15:55:22 UTC 2023"
    }
}

Additional context

No response

have you solved this problem finally?