Dev pure (#2039)

* Remove library folder, update setup_windows

* Cleanup sd-scripts files

* set sd-scripts release via file

* More cleanup

* Point to sd-scripts python files

* Fix blip captioning working directory

* Fix typo

* Fix issue with verify lora

* Improve git cloning of sd-scripts repo

* Integrate cloning in linux setup script

* Remove unnecessary folder

* Update linux setup

* Removing unnecessary code

* Fix validation issue under linux

* Update tensorboard version linux

* Updating linux requirements

* Update README

* Added submodule for sd-scripts

* Update for proper submodule handling

* Simplify submodule init

* Update comment

* Improve WSL2 LD_LIBRARY_PATH handling

* Update tensorboard handling of logging_dir

* chore(docker): update TensorFlow and TensorBoard versions

- Upgrade TensorFlow version in Dockerfile and requirements to 2.15.0.post1
- Update TensorBoard version in requirements to 2.15.2

Signed-off-by: 陳鈞 <[email protected]>

* chore(docker): Add a mount for `sd-scripts` in the Dockerfile during pip install

- Add a mount step for `sd-scripts` in the Dockerfile during pip install.

Signed-off-by: 陳鈞 <[email protected]>

* doc(docker): Update readme about docker

* Fix issue with TE LR

* Fix issue with TE LR

* Update sd-scripts to Feb 27 2024 commit

* Update version of bnb for windows

* Update windows setup

* Update windows setup

* Update bnb linux to match windows

* remove height:auto

* update UI

 * unify Quick Pick + custom model selection into gr.Dropdown()
 * add list_dirs(), list_files()
 * use gr.Accordion()
 * always show model + training image folder + output name + minimal default settings
 * do not show hidden files
 * clean up list_*()

* use PYTHONPATH env variable

* add "tool" classname to gr.Button() for sd-webui compatible

* update *_caption_gui.py using create_refresh_buttons()

* update Utilities->Group Images

* update Utilities->Convert model

* update dataset gui

* update Dreambooth/LoRA folders preparation

* update *lora*_gui UI using list_files(), create_refresh_button()

* more fixes for PYTHONPATH

* Minor edits to basic caption and common gui

* Optimising some code

* Fix path issue

* Fix issues with list_images_dir

* Update finetuning image folder label

* Minor updates to UI

---------

Signed-off-by: 陳鈞 <[email protected]>
Co-authored-by: 陳鈞 <[email protected]>
Co-authored-by: Won-Kyu Park <[email protected]>

.augmentignore

@@ -0,0 +1,15 @@
+.env
+.cache
+.vscode
+__pycache__
+bitsandbytes_windows
+cudnn_windows
+data
+dataset
+docs
+examples
+outputs
+SmilingWolf
+test
+v2_inference
+venv

.gitignore

@@ -1,5 +1,6 @@
 # Python
 venv
+venv2
 __pycache__
 *.egg-info
 build
@@ -44,4 +45,4 @@ requirements_tmp_for_setup.txt
 0.13.3
 
 *.npz
-presets/*/user_presets/*
+presets/*/user_presets/*

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "sd-scripts"]
+    path = sd-scripts
+    url = https://github.com/kohya-ss/sd-scripts.git

.release

@@ -1 +1 @@
-v22.7.0
+v23.0.0

.vscode/settings.json

@@ -1,5 +1,6 @@
 {
     "python.linting.enabled": true,
     "python.formatting.provider": "yapf",
-    "DockerRun.DisableDockerrc": true
+    "DockerRun.DisableDockerrc": true,
+    "augment.enableAutomaticCompletions": false
 }

Dockerfile

@@ -30,7 +30,7 @@ RUN --mount=type=cache,id=pip-$TARGETARCH$TARGETVARIANT,sharing=locked,target=/r
     torch==2.1.2 torchvision==0.16.2 \
     xformers==0.0.23.post1 \
     # Why [and-cuda]: https://github.com/tensorflow/tensorflow/issues/61468#issuecomment-1759462485
-    tensorflow[and-cuda]==2.14.0 \
+    tensorflow[and-cuda]==2.15.0.post1 \
     ninja \
     pip setuptools wheel
 
@@ -39,6 +39,7 @@ RUN --mount=type=cache,id=pip-$TARGETARCH$TARGETVARIANT,sharing=locked,target=/r
     --mount=source=requirements_linux_docker.txt,target=requirements_linux_docker.txt \
     --mount=source=requirements.txt,target=requirements.txt \
     --mount=source=setup/docker_setup.py,target=setup.py \
+    --mount=source=sd-scripts,target=sd-scripts,rw \
     pip install -r requirements_linux_docker.txt -r requirements.txt
 
 # Replace pillow with pillow-simd (Only for x86)