Releases: bmaltais/kohya_ss
v21.0.1
v20.8.2
- 2023/02/26 (v20.8.2):
- Fix issue #231
- Change default for seed to random
- Add support for --share argument to
kohya_gui.py
andgui.ps1
- Implement 8bit adam login to help with the legacy
Use 8bit adam
checkbox that is now superceided by theOptimizer
dropdown selection. This field will be eventually removed. Kept for now for backward compatibility.
v20.8.1
- 2023/02/23 (v20.8.1):
- Fix instability training issue in
train_network.py
.fp16
training is probably not affected by this issue.- Training with
float
for SD2.x models will work now. Also training with bf16 might be improved. - This issue seems to have occurred in PR#190.
- Add some metadata to LoRA model. Thanks to space-nuko!
- Raise an error if optimizer options conflict (e.g.
--optimizer_type
and--use_8bit_adam
.) - Support ControlNet in
gen_img_diffusers.py
(no documentation yet.)
- Fix instability training issue in
v20.8.0
- 2023/02/22 (v20.8.0):
- Add gui support for optimizers:
AdamW, AdamW8bit, Lion, SGDNesterov, SGDNesterov8bit, DAdaptation, AdaFactor
- Add gui support for
--noise_offset
- Refactor optmizer options. Thanks to mgz-dev!
- Add
--optimizer_type
option for each training script. Please see help. Japanese documentation is here. --use_8bit_adam
and--use_lion_optimizer
options also work and will override the options above for backward compatibility.
- Add
- Add SGDNesterov and its 8bit.
- Add D-Adaptation optimizer. Thanks to BootsofLagrangian and all!
- Please install D-Adaptation optimizer with
pip install dadaptation
(it is not in requirements.txt currently.) - Please see kohya-ss/sd-scripts#181 for details.
- Please install D-Adaptation optimizer with
- Add AdaFactor optimizer. Thanks to Toshiaki!
- Extra lr scheduler settings (num_cycles etc.) are working in training scripts other than
train_network.py
. - Add
--max_grad_norm
option for each training script for gradient clipping.0.0
disables clipping. - Symbolic link can be loaded in each training script. Thanks to TkskKurumi!
- Add gui support for optimizers:
v20.7.4
- 2023/02/19 (v20.7.4):
- Add
--use_lion_optimizer
to each training script to use Lion optimizer.- Please install Lion optimizer with
pip install lion-pytorch
(it is not inrequirements.txt
currently.)
- Please install Lion optimizer with
- Add
--lowram
option totrain_network.py
. Load models to VRAM instead of VRAM (for machines which have bigger VRAM than RAM such as Colab and Kaggle). Thanks to Isotr0py!- Default behavior (without lowram) has reverted to the same as before 14 Feb.
- Fixed git commit hash to be set correctly regardless of the working directory. Thanks to vladmandic!
- Add
- 2023/02/15 (v20.7.3):
- Update upgrade.ps1 script
- Integrate new kohya sd-script
- Noise offset is recorded to the metadata. Thanks to space-nuko!
- Show the moving average loss to prevent loss jumping in
train_network.py
andtrain_db.py
. Thanks to shirayu! - Add support with multi-gpu trainining for
train_network.py
. Thanks to Isotr0py! - Add
--verbose
option forresize_lora.py
. For details, see this PR. Thanks to mgz-dev! - Git commit hash is added to the metadata for LoRA. Thanks to space-nuko!
- Add
--noise_offset
option for each training scripts.- Implementation of https://www.crosslabs.org//blog/diffusion-with-offset-noise
- This option may improve ability to generate darker/lighter images. May work with LoRA.
v20.7.3
v20.7.2
- 2023/02/11 (v20.7.2):
lora_interrogator.py
is added innetworks
folder. Seepython networks\lora_interrogator.py -h
for usage.- For LoRAs where the activation word is unknown, this script compares the output of Text Encoder after applying LoRA to that of unapplied to find out which token is affected by LoRA. Hopefully you can figure out the activation word. LoRA trained with captions does not seem to be able to interrogate.
- Batch size can be large (like 64 or 128).
train_textual_inversion.py
now supports multiple init words.- Following feature is reverted to be the same as before. Sorry for confusion:
Now the number of data in each batch is limited to the number of actual images (not duplicated). Because a certain bucket may contain smaller number of actual images, so the batch may contain same (duplicated) images.
- Add new tool to sort, group and average crop image in a dataset
v20.7.1
2023/02/09 (v20.7.1)
- Caption dropout is supported in train_db.py
, fine_tune.py
and train_network.py
. Thanks to forestsource!
- --caption_dropout_rate
option specifies the dropout rate for captions (01.0, 0.1 means 10% chance for dropout). If dropout occurs, the image is trained with the empty caption. Default is 0 (no dropout).1.0, 0.1 means 10% chance for dropout). If dropout occurs, the tag is removed from the caption. If
- --caption_dropout_every_n_epochs
option specifies how many epochs to drop captions. If 3
is specified, in epoch 3, 6, 9 ..., images are trained with all captions empty. Default is None (no dropout).
- --caption_tag_dropout_rate
option specified the dropout rate for tags (comma separated tokens) (0--keep_tokens
option is set, these tokens (tags) are not dropped. Default is 0 (no droupout).
- The bulk image downsampling script is added. Documentation is here (in Jpanaese). Thanks to bmaltais!
- Typo check is added. Thanks to shirayu!
- Add option to autolaunch the GUI in a browser and set the server_port. USe either gui.ps1 --inbrowser --server_port 3456
or gui.cmd -inbrowser -server_port 3456
v20.7.0
- 2023/02/06 (v20.7.0)
--bucket_reso_steps
and--bucket_no_upscale
options are added to training scripts (fine tuning, DreamBooth, LoRA and Textual Inversion) andprepare_buckets_latents.py
.--bucket_reso_steps
takes the steps for buckets in aspect ratio bucketing. Default is 64, same as before.- Any value greater than or equal to 1 can be specified; 64 is highly recommended and a value divisible by 8 is recommended.
- If less than 64 is specified, padding will occur within U-Net. The result is unknown.
- If you specify a value that is not divisible by 8, it will be truncated to divisible by 8 inside VAE, because the size of the latent is 1/8 of the image size.
- If
--bucket_no_upscale
option is specified, images smaller than the bucket size will be processed without upscaling.- Internally, a bucket smaller than the image size is created (for example, if the image is 300x300 and
bucket_reso_steps=64
, the bucket is 256x256). The image will be trimmed. - Implementation of #130.
- Images with an area larger than the maximum size specified by
--resolution
are downsampled to the max bucket size.
- Internally, a bucket smaller than the image size is created (for example, if the image is 300x300 and
- Now the number of data in each batch is limited to the number of actual images (not duplicated). Because a certain bucket may contain smaller number of actual images, so the batch may contain same (duplicated) images.
--random_crop
now also works with buckets enabled.- Instead of always cropping the center of the image, the image is shifted left, right, up, and down to be used as the training data. This is expected to train to the edges of the image.
- Implementation of discussion #34.