Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Experimental Feature] FP8 weight dtype for base model when running train_network (or sdxl_train_network) #1057

Merged
merged 31 commits into from
Jan 20, 2024
Merged

[Experimental Feature] FP8 weight dtype for base model when running train_network (or sdxl_train_network) #1057

merged 31 commits into from
Jan 20, 2024

Commits on Oct 24, 2023

  1. Add fp8 support

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    ed883f2 View commit details
    Browse the repository at this point in the history

  2. remove some debug prints

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    c0c7109 View commit details
    Browse the repository at this point in the history

  3. Better implementation for te

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    9e2b99f View commit details
    Browse the repository at this point in the history

  4. Fix some misunderstanding

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    628b44d View commit details
    Browse the repository at this point in the history

  5. as same as unet, add explicit convert

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    65da2cc View commit details
    Browse the repository at this point in the history

  6. better impl for convert TE to fp8

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    1623d6c View commit details
    Browse the repository at this point in the history

  7. fp8 for not only unet

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    e2f32f5 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2023

  1. Better cache TE and TE lr

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    ed82a1a View commit details
    Browse the repository at this point in the history

  2. match arg name

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    f582142 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. Fix with list

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    f52cbc2 View commit details
    Browse the repository at this point in the history

  2. Add timeout settings

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    34f1cd6 View commit details
    Browse the repository at this point in the history

  3. Fix arg style

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    19d6617 View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2023

  1. Add custom seperator

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    e355764 View commit details
    Browse the repository at this point in the history

  2. Fix typo

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    3f670d2 View commit details
    Browse the repository at this point in the history

  3. Fix typo again

    @KohakuBlueleaf
    KohakuBlueleaf committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    35c32a6 View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2023

  1. Merge remote-tracking branch 'upstream/dev' into fp8-experiments

    @KohakuBlueleaf
    KohakuBlueleaf committed Nov 4, 2023
    Configuration menu
    Copy the full SHA
    3f0c40a View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2023

  1. Fix dtype error

    @KohakuBlueleaf
    KohakuBlueleaf committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    9be4f44 View commit details
    Browse the repository at this point in the history

  2. Fix gradient problem

    @KohakuBlueleaf
    KohakuBlueleaf committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    b003446 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2023

  1. Fix req grad

    @KohakuBlueleaf
    KohakuBlueleaf committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    c0f8d28 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/dev' into fp8-experiments

    @KohakuBlueleaf
    KohakuBlueleaf committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    9bb4fcb View commit details
    Browse the repository at this point in the history

  3. fix merge

    @KohakuBlueleaf
    KohakuBlueleaf committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    67a3ad8 View commit details
    Browse the repository at this point in the history

  4. Fix merge

    @KohakuBlueleaf
    KohakuBlueleaf committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    54f5f46 View commit details
    Browse the repository at this point in the history

Commits on Dec 16, 2023

  1. Merge remote-tracking branch 'upstream/dev' into fp8-experiments

    @KohakuBlueleaf
    KohakuBlueleaf committed Dec 16, 2023
    Configuration menu
    Copy the full SHA
    fa82f2a View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2023

  1. Merge pull request #1 from kohya-ss/dev 

    Dev
    @KohakuBlueleaf
    KohakuBlueleaf authored Dec 21, 2023
    Configuration menu
    Copy the full SHA
    3f0414c View commit details
    Browse the repository at this point in the history

Commits on Jan 6, 2024

  1. Merge branch 'kohya-ss:main' into fp8-experiments

    @KohakuBlueleaf
    KohakuBlueleaf authored Jan 6, 2024
    Configuration menu
    Copy the full SHA
    4664c2d View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/dev' into fp8-experiments

    @KohakuBlueleaf
    KohakuBlueleaf committed Jan 6, 2024
    Configuration menu
    Copy the full SHA
    12151c9 View commit details
    Browse the repository at this point in the history

  3. Resolve merge

    @KohakuBlueleaf
    KohakuBlueleaf committed Jan 6, 2024
    Configuration menu
    Copy the full SHA
    b4b872e View commit details
    Browse the repository at this point in the history

  4. arrangement and document

    @KohakuBlueleaf
    KohakuBlueleaf committed Jan 6, 2024
    Configuration menu
    Copy the full SHA
    9229282 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2024

  1. Merge remote-tracking branch 'upstream/dev' into fp8-experiments

    @KohakuBlueleaf
    KohakuBlueleaf committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    3438703 View commit details
    Browse the repository at this point in the history

  2. Resolve merge error

    @KohakuBlueleaf
    KohakuBlueleaf committed Jan 17, 2024
    Configuration menu
    Copy the full SHA
    c5b9187 View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2024

  1. Add assert for mixed precision

    @KohakuBlueleaf
    KohakuBlueleaf committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    100f852 View commit details
    Browse the repository at this point in the history