WaveFlow: A Compact Flow-based Model for Raw Audio

Update: Pretrained weights are now available. See links below.

This is an unofficial PyTorch implementation of WaveFlow (Ping et al, ICML 2020) model.

The aim for this repo is to provide easy-to-use PyTorch version of WaveFlow as a drop-in alternative to various neural vocoder models used with NVIDIA's Tacotron2 audio processing backend.

Please refer to the official implementation written in PaddlePaddle for the official results.

Setup

Clone this repo and install requirements

git clone https://github.com/L0SG/WaveFlow.git
cd WaveFlow
pip install -r requirements.txt

Install Apex for mixed-precision training

Train your model

Download LJ Speech Data. In this example it's in data/
Make a list of the file names to use for training/testing.
```
ls data/*.wav | tail -n+10 > train_files.txt
ls data/*.wav | head -n10 > test_files.txt
```
-n+10 and -n10 indicates that this example reserves the first 10 audio clips for model testing.
Edit the configuration file and train the model.

Below are the example commands using waveflow-h16-r64-bipartize.json
```
nano configs/waveflow-h16-r64-bipartize.json
python train.py -c configs/waveflow-h16-r64-bipartize.json
```
Single-node multi-GPU training is automatically enabled with DataParallel (instead of DistributedDataParallel for simplicity).

For mixed precision training, set "fp16_run": true on the configuration file.

You can load the trained weights from saved checkpoints by providing the path to checkpoint_path variable in the config file.

checkpoint_path accepts either explicit path, or the parent directory if resuming from averaged weights over multiple checkpoints.

Examples

insert checkpoint_path: "experiments/waveflow-h16-r64-bipartize/waveflow_5000" in the config file then run
```
python train.py -c configs/waveflow-h16-r64-bipartize.json
```
for loading averaged weights over 10 recent checkpoints, insert checkpoint_path: "experiments/waveflow-h16-r64-bipartize" in the config file then run
```
python train.py -a 10 -c configs/waveflow-h16-r64-bipartize.json
```
you can reset the optimizer and training scheduler (and keep the weights) by providing --warm_start
```
python train.py --warm_start -c configs/waveflow-h16-r64-bipartize.json
```
Synthesize waveform from the trained model.

insert checkpoint_path in the config file and use --synthesize to train.py. The model generates waveform by looping over test_files.txt.
```
python train.py --synthesize -c configs/waveflow-h16-r64-bipartize.json
```
if fp16_run: true, the model uses FP16 (half-precision) arithmetic for faster performance (on GPUs equipped with Tensor Cores).

Pretrained Weights

We provide pretrained weights via Google Drive. The models are trained for 5 M steps, then we averaged weights over 20 last checkpoints with -a 20. Audio quality almost matches the original paper.