Skip to content

v22.2.0
Compare
Choose a tag to compare
@bmaltais bmaltais released this 16 Nov 00:06
· 710 commits to master since this release
v22.2.0
6adfb3f
  • 2023/11/15 (v22.2.0)
  • sd-scripts code base update:

    • sdxl_train.py now supports different learning rates for each Text Encoder.

      • Example:
        • --learning_rate 1e-6: train U-Net only
        • --train_text_encoder --learning_rate 1e-6: train U-Net and two Text Encoders with the same learning rate (same as the previous version)
        • --train_text_encoder --learning_rate 1e-6 --learning_rate_te1 1e-6 --learning_rate_te2 1e-6: train U-Net and two Text Encoders with the different learning rates
        • --train_text_encoder --learning_rate 0 --learning_rate_te1 1e-6 --learning_rate_te2 1e-6: train two Text Encoders only
        • --train_text_encoder --learning_rate 1e-6 --learning_rate_te1 1e-6 --learning_rate_te2 0: train U-Net and one Text Encoder only
        • --train_text_encoder --learning_rate 0 --learning_rate_te1 0 --learning_rate_te2 1e-6: train one Text Encoder only

    • train_db.py and fine_tune.py now support different learning rates for Text Encoder. Specify with --learning_rate_te option.

      • To train Text Encoder with fine_tune.py, specify --train_text_encoder option too. train_db.py trains Text Encoder by default.

    • Fixed the bug that Text Encoder is not trained when block lr is specified in sdxl_train.py.

    • Debiased Estimation loss is added to each training script. Thanks to sdbds!

      • Specify --debiased_estimation_loss option to enable it. See PR #889 for details.

    • Training of Text Encoder is improved in train_network.py and sdxl_train_network.py. Thanks to KohakuBlueleaf! PR #895

    • The moving average of the loss is now displayed in the progress bar in each training script. Thanks to shirayu! PR #899

    • PagedAdamW32bit optimizer is supported. Specify --optimizer_type=PagedAdamW32bit. Thanks to xzuyn! PR #900

    • Other bug fixes and improvements.

  • kohya_ss gui updates:
    • Implement GUI support for SDXL finetune TE1 and TE2 training LR parameters and for non SDXL finetune TE training parameter
    • Implement GUI support for Dreambooth TE LR parameter
    • Implement Debiased Estimation loss at the botom of the Advanced Parameters tab.

Full Changelog: v22.1.1...v22.2.0

Assets 2
Loading