Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add experimental INT8 quantized training #644

Merged
merged 45 commits into from
Aug 16, 2024
Merged

Add experimental INT8 quantized training #644

merged 45 commits into from
Aug 16, 2024

Commits on Aug 9, 2024

  1. initial commit

    @gau-nernst
    gau-nernst committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    3d42329 View commit details
    Browse the repository at this point in the history

  2. add tests

    @gau-nernst
    gau-nernst committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    eca170a View commit details
    Browse the repository at this point in the history

  3. add training

    @gau-nernst
    gau-nernst committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    dd162a8 View commit details
    Browse the repository at this point in the history

  4. support py3.9

    @gau-nernst
    gau-nernst committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    b286f5d View commit details
    Browse the repository at this point in the history

  5. skip test for torch<2.3

    @gau-nernst
    gau-nernst committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    8a84aca View commit details
    Browse the repository at this point in the history

  6. fix pytest

    @gau-nernst
    gau-nernst committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    ea47c7d View commit details
    Browse the repository at this point in the history

  7. fix adamw

    @gau-nernst
    gau-nernst committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    f20486b View commit details
    Browse the repository at this point in the history

  8. add some FSDP ops

    @gau-nernst
    gau-nernst committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    3415244 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2024

  1. add more fsdp ops

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    5d0e658 View commit details
    Browse the repository at this point in the history

  2. more ops

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    d753476 View commit details
    Browse the repository at this point in the history

  3. add benchmark script

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    9c77800 View commit details
    Browse the repository at this point in the history

  4. some organisation

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    158eb61 View commit details
    Browse the repository at this point in the history

  5. add FSDP test

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    db0290f View commit details
    Browse the repository at this point in the history

  6. clean up

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    1c32b78 View commit details
    Browse the repository at this point in the history

  7. update FSDP test

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    ff69121 View commit details
    Browse the repository at this point in the history

  8. add compile test (things are crashing)

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    45342ba View commit details
    Browse the repository at this point in the history

  9. fix bias

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    f1587a2 View commit details
    Browse the repository at this point in the history

  10. substantial update to tests

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    7f9102a View commit details
    Browse the repository at this point in the history

  11. fix compile for FSDP

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    0428330 View commit details
    Browse the repository at this point in the history

  12. update readme. rename file

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    001422c View commit details
    Browse the repository at this point in the history

  13. speed up CI

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    2eb2787 View commit details
    Browse the repository at this point in the history

  14. fix typo

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    d39caba View commit details
    Browse the repository at this point in the history

  15. fix typo

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    de6aa25 View commit details
    Browse the repository at this point in the history

  16. typos. unset some dynamo flags

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    adbe47d View commit details
    Browse the repository at this point in the history

  17. update readme

    @gau-nernst
    gau-nernst committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    3fdf776 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2024

  1. remove requires_grad, since it is unnecessary

    @gau-nernst
    gau-nernst committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    ea0ee4f View commit details
    Browse the repository at this point in the history

  2. remove note

    @gau-nernst
    gau-nernst committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    36d0e1a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'pytorch:main' into qt_int8

    @gau-nernst
    gau-nernst authored Aug 11, 2024
    Configuration menu
    Copy the full SHA
    2360a97 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. Merge branch 'main' into qt_int8

    @gau-nernst
    gau-nernst committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    9e19104 View commit details
    Browse the repository at this point in the history

  2. don't set inductor flags

    @gau-nernst
    gau-nernst committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    6bc7621 View commit details
    Browse the repository at this point in the history

  3. rename

    @gau-nernst
    gau-nernst committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    6646c0b View commit details
    Browse the repository at this point in the history

  4. update README

    @gau-nernst
    gau-nernst committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    00e25cf View commit details
    Browse the repository at this point in the history

  5. rename optimizer

    @gau-nernst
    gau-nernst committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    927a6d1 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2024

  1. Merge branch 'main' into qt_int8

    @gau-nernst
    gau-nernst committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    8377707 View commit details
    Browse the repository at this point in the history

  2. update benchmark script

    @gau-nernst
    gau-nernst committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    de49e8b View commit details
    Browse the repository at this point in the history

  3. make compile explicit

    @gau-nernst
    gau-nernst committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    f80ac97 View commit details
    Browse the repository at this point in the history

  4. update docs

    @gau-nernst
    gau-nernst committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    e375c3d View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. Merge branch 'main' into qt_int8

    @gau-nernst
    gau-nernst committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    6396a95 View commit details
    Browse the repository at this point in the history

  2. use torch.optim.Adam to avoid FSDP optim compile bug

    @gau-nernst
    gau-nernst committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    662c61f View commit details
    Browse the repository at this point in the history

  3. update docs

    @gau-nernst
    gau-nernst committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    cc90298 View commit details
    Browse the repository at this point in the history

  4. update doc

    @gau-nernst
    gau-nernst committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    f1c588b View commit details
    Browse the repository at this point in the history

  5. update docs

    @gau-nernst
    gau-nernst committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    f444fa6 View commit details
    Browse the repository at this point in the history

  6. fix CI test

    @gau-nernst
    gau-nernst committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    640ec2d View commit details
    Browse the repository at this point in the history

  7. skip test

    @gau-nernst
    gau-nernst committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    dad6560 View commit details
    Browse the repository at this point in the history

  8. fix compiled test

    @gau-nernst
    gau-nernst committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    4924e8d View commit details
    Browse the repository at this point in the history