v23.1.1

What's Changed

• Add Huber loss support by @bmaltais in #2221

Full Changelog: v23.1.0...v23.1.1

v23.1.0

• Update sd-scripts to 0.8.7

  • The default value of huber_schedule in Scheduled Huber Loss is changed from exponential to snr, which is expected to give better results.

  • Highlights

    • The dependent libraries are updated. Please see Upgrade and update the libraries.
      • Especially imagesize is newly added, so if you cannot update the libraries immediately, please install with pip install imagesize==1.4.1 separately.
      • bitsandbytes==0.43.0, prodigyopt==1.0, lion-pytorch==0.0.6 are included in the requirements.txt.
        • bitsandbytes no longer requires complex procedures as it now officially supports Windows.
      • Also, the PyTorch version is updated to 2.1.2 (PyTorch does not need to be updated immediately). In the upgrade procedure, PyTorch is not updated, so please manually install or update torch, torchvision, xformers if necessary (see Upgrade PyTorch).
    • When logging to wandb is enabled, the entire command line is exposed. Therefore, it is recommended to write wandb API key and HuggingFace token in the configuration file (.toml). Thanks to bghira for raising the issue.
      • A warning is displayed at the start of training if such information is included in the command line.
      • Also, if there is an absolute path, the path may be exposed, so it is recommended to specify a relative path or write it in the configuration file. In such cases, an INFO log is displayed.
      • See #1123 and PR #1240 for details.
    • Colab seems to stop with log output. Try specifying --console_log_simple option in the training script to disable rich logging.
    • Other improvements include the addition of masked loss, scheduled Huber Loss, DeepSpeed support, dataset settings improvements, and image tagging improvements. See below for details.

  • Training scripts

    • train_network.py and sdxl_train_network.py are modified to record some dataset settings in the metadata of the trained model (caption_prefix, caption_suffix, keep_tokens_separator, secondary_separator, enable_wildcard).
    • Fixed a bug that U-Net and Text Encoders are included in the state in train_network.py and sdxl_train_network.py. The saving and loading of the state are faster, the file size is smaller, and the memory usage when loading is reduced.
    • DeepSpeed is supported. PR #1101 and #1139 Thanks to BootsofLagrangian! See PR #1101 for details.
    • The masked loss is supported in each training script. PR #1207 See Masked loss for details.
    • Scheduled Huber Loss has been introduced to each training scripts. PR #1228 Thanks to kabachuha for the PR and cheald, drhead, and others for the discussion! See the PR and Scheduled Huber Loss for details.
    • The options --noise_offset_random_strength and --ip_noise_gamma_random_strength are added to each training script. These options can be used to vary the noise offset and ip noise gamma in the range of 0 to the specified value. PR #1177 Thanks to KohakuBlueleaf!
    • The options --save_state_on_train_end are added to each training script. PR #1168 Thanks to gesen2egee!
    • The options --sample_every_n_epochs and --sample_every_n_steps in each training script now display a warning and ignore them when a number less than or equal to 0 is specified. Thanks to S-Del for raising the issue.

  • Dataset settings

    • The English version of the dataset settings documentation is added. PR #1175 Thanks to darkstorm2150!
    • The .toml file for the dataset config is now read in UTF-8 encoding. PR #1167 Thanks to Horizon1704!
    • Fixed a bug that the last subset settings are applied to all images when multiple subsets of regularization images are specified in the dataset settings. The settings for each subset are correctly applied to each image. PR #1205 Thanks to feffy380!
    • Some features are added to the dataset subset settings.
      • secondary_separator is added to specify the tag separator that is not the target of shuffling or dropping.
        • Specify secondary_separator=";;;". When you specify secondary_separator, the part is not shuffled or dropped.
      • enable_wildcard is added. When set to true, the wildcard notation {aaa|bbb|ccc} can be used. The multi-line caption is also enabled.
      • keep_tokens_separator is updated to be used twice in the caption. When you specify keep_tokens_separator="|||", the part divided by the second ||| is not shuffled or dropped and remains at the end.
      • The existing features caption_prefix and caption_suffix can be used together. caption_prefix and caption_suffix are processed first, and then enable_wildcard, keep_tokens_separator, shuffling and dropping, and secondary_separator are processed in order.
      • See Dataset config for details.
    • The dataset with DreamBooth method supports caching image information (size, caption). PR #1178 and #1206 Thanks to KohakuBlueleaf! See DreamBooth method specific options for details.

  • Image tagging

    • The support for v3 repositories is added to tag_image_by_wd14_tagger.py (--onnx option only). PR #1192 Thanks to sdbds!
      • Onnx may need to be updated. Onnx is not installed by default, so please install or update it with pip install onnx==1.15.0 onnxruntime-gpu==1.17.1 etc. Please also check the comments in requirements.txt.
    • The model is now saved in the subdirectory as --repo_id in tag_image_by_wd14_tagger.py . This caches multiple repo_id models. Please delete unnecessary files under --model_dir.
    • Some options are added to tag_image_by_wd14_tagger.py.
      • Some are added in PR #1216 Thanks to Disty0!
      • Output rating tags --use_rating_tags and --use_rating_tags_as_last_tag
      • Output character tags first --character_tags_first
      • Expand character tags and series --character_tag_expand
      • Specify tags to output first --always_first_tags
      • Replace tags --tag_replacement
      • See Tagging documentation for details.
    • Fixed an error when specifying --beam_search and a value of 2 or more for --num_beams in make_captions.py.

  • About Masked loss
    The masked loss is supported in each training script. To enable the masked loss, specify the --masked_loss option.

    The feature is not fully tested, so there may be bugs. If you find any issues, please open an Issue.

    ControlNet dataset is used to specify the mask. The mask images should be the RGB images. The pixel value 255 in R channel is treated as the mask (the loss is calculated only for the pixels with the mask), and 0 is treated as the non-mask. The pixel values 0-255 are converted to 0-1 (i.e., the pixel value 128 is treated as the half weight of the loss). See details for the dataset specification in the LLLite documentation.

  • About Scheduled Huber Loss
    Scheduled Huber Loss has been introduced to each training scripts. This is a method to improve robustness against outliers or anomalies (data corruption) in the training data.

    With the traditional MSE (L2) loss function, the impact of outliers could be significant, potentially leading to a degradation in the quality of generated images. On the other hand, while the Huber loss function can suppress the influence of outliers, it tends to compromise the reproduction of fine details in images.

    To address this, the proposed method employs a clever application of the Huber loss function. By scheduling the use of Huber loss in the early stages of training (when noise is high) and MSE in the later stages, it strikes a balance between outlier robustness and fine detail reproduction.

    Experimental results have confirmed that this method achieves higher accuracy on data containing outliers compared to pure Huber loss or MSE. The increase in computational cost is minimal.

    The newly added arguments loss_type, huber_schedule, and huber_c allow for the selection of the loss function type (Huber, smooth L1, MSE), scheduling method (exponential, constant, SNR), and Huber's parameter. This enables optimization based on the characteristics of the dataset.

    See PR #1228 for details.

    • loss_type: Specify the loss function type. Choose huber for Huber loss, smooth_l1 for smooth L1 loss, and l2 for MSE loss. The default is l2, which is the same as before.
    • huber_schedule: Specify the scheduling method. Choose exponential, constant, or snr. The default is snr.
    • huber_c: Specify the Huber's parameter. The default is 0.1.

    Please read Releases for recent updates.`

• Added GUI support for the new parameters listed above.
  *...

v23.0.15

What's Changed

• Add support for toml dataset configuration fole to all trainers
• Add new setup menu option to install Triton 2.1.0 for Windows
• Add support for LyCORIS BOFT and DoRA and QLyCORIS options for LoHA, LoKr and LoCon
• Fix issue with vae path validation
• Other fixes

Full Changelog: v23.0.14...v23.0.15

v23.0.14

What's Changed

• Add missed train_norm network args for diag-oft by @gesen2egee in #2124
• fixed bitsandbytes 0.41.2 installation not working on setup_windows by @MontriSugoiDev in #2121
• Fix caption issue in #2130

New Contributors

• @gesen2egee made their first contribution in #2124
• @MontriSugoiDev made their first contribution in #2121

Full Changelog: v23.0.13...v23.0.14

v23.0.13

What's Changed

• Fix issue with image sample generation

Full Changelog: v23.0.12...v23.0.13

v23.0.12

What's Changed

• Enhanced Logging and Tracking Capabilities

  • Added support for configuring advanced logging and tracking:
    • wandb_run_name: Set a custom name for your Weights & Biases runs to easily identify and organize your experiments.
    • log_tracker_name and log_tracker_config: Integrate custom logging trackers with your projects. Specify the tracker name and provide its configuration to enable detailed monitoring and logging of your runs.

• Custom Path Defaults

  • You can now specify custom paths more easily:
    • Simply copy the config example.toml file located in the root directory of the repository to config.toml.
    • Edit the config.toml file to adjust paths and settings according to your preferences.

Software Updates

• sd-scripts updated to v0.8.5

  • Bug Fixes:
    • Corrected an issue where the value of timestep embedding was incorrect during SDXL training. This fix ensures accurate training progress and results.
    • Addressed a related inference issue with the generation script, improving the reliability of SDXL model outputs.
  • Note: The exact impact of this bug is currently unknown, but it's recommended to update to v0.8.5 for anyone engaged in SDXL training to ensure optimal performance and results.

• Upgrade of lycoris_lora Python Module

  • Updated the lycoris_lora module to version 2.2.0.post3. This update may include bug fixes, performance improvements, and new features.

Recommendations for Users

• To benefit from the latest features and improvements, users are encouraged to update their installations and configurations accordingly.

Other updates

• chore(docker): adjust Dockerfile layers to improve build cache design by @jim60105 in #2102
• Code improvements by @bmaltais in #2107
• Revert tensorboard and tensorflow versions for RunPod since the new versions don't work correctly on RunPod by @ashleykleynhans in #2111
• Update zh-TW locale to 23.0.11. by @hinablue in #2113
• v23.0.12 by @bmaltais in #2119

Full Changelog: v23.0.11...v23.0.12

v23.0.11

What's Changed

• Increase icon size
• More setup fixes

Full Changelog: v23.0.10...v23.0.11

v23.0.10

What's Changed

• Fix setup issue for linux by @bmaltais in #2090

Full Changelog: v23.0.9...v23.0.10

v23.0.9

What's Changed

• Reworked how setup can be run to improve Stability Matrix support
• Add support for huggingface based vea path

Full Changelog: v23.0.8...v23.0.9

v23.0.8

What's Changed

• Add the ability to create outout and logs folder if it does not exist

Full Changelog: v23.0.7...v23.0.8