Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add torch.compile + FSDP2 float8 all-gather in CI #468

Merged
merged 34 commits into from
Jul 19, 2024
Merged

add torch.compile + FSDP2 float8 all-gather in CI #468

merged 34 commits into from
Jul 19, 2024

Commits on Jun 12, 2024

  1. float8 tmp save 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    8d00b73 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. Merge branch 'main' into fsdp2

    @weifengpy
    weifengpy authored Jun 19, 2024
    Configuration menu
    Copy the full SHA
    68d9f61 View commit details
    Browse the repository at this point in the history

  2. run 8b eager successfully 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    4cd5f74 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. enable compile 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    05a4a06 View commit details
    Browse the repository at this point in the history

  2. benchmark 

    Summaiy:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    f48a82e View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. 1d setup 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    14aabfb View commit details
    Browse the repository at this point in the history

  2. 2d setup 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    b88aee9 View commit details
    Browse the repository at this point in the history

  3. 2d setup 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    2b4e0c2 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. Merge branch 'main' into fsdp2

    @weifengpy
    weifengpy authored Jul 3, 2024
    Configuration menu
    Copy the full SHA
    ad63aba View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. Merge branch 'main' into fsdp2

    @weifengpy
    weifengpy authored Jul 11, 2024
    Configuration menu
    Copy the full SHA
    71d4dc6 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. fp8 all-gather FSDP 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    23536e9 View commit details
    Browse the repository at this point in the history

  2. linter 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    bdb0fd0 View commit details
    Browse the repository at this point in the history

  3. add unit test and restore original toml 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    ef0e843 View commit details
    Browse the repository at this point in the history

  4. add unit test for float8 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    c294f6a View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. better doc with original dtype all-gather and value error on fp8 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    b58b07b View commit details
    Browse the repository at this point in the history

  2. improve config msg 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    7df10ae View commit details
    Browse the repository at this point in the history

  3. Merge branch 'pytorch:main' into fsdp2

    @weifengpy
    weifengpy authored Jul 15, 2024
    Configuration menu
    Copy the full SHA
    f674012 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. rename config to enable_fp8_linear and improve comments 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    7dd788c View commit details
    Browse the repository at this point in the history

  2. rename to enable_fp8_linear 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    faefe27 View commit details
    Browse the repository at this point in the history

  3. add 2D test 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    7aad066 View commit details
    Browse the repository at this point in the history

  4. import Optional and NotImplement for delayed scaling 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    5040c31 View commit details
    Browse the repository at this point in the history

  5. remove TP fp8 all-gather from CI 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    cee653e View commit details
    Browse the repository at this point in the history

  6. fix linter 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    e164285 View commit details
    Browse the repository at this point in the history

  7. remove redudant check 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    22c71ea View commit details
    Browse the repository at this point in the history

  8. install float8_experimental in CI 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    595f83d View commit details
    Browse the repository at this point in the history

  9. Merge branch 'pytorch:main' into fsdp2

    @weifengpy
    weifengpy authored Jul 16, 2024
    Configuration menu
    Copy the full SHA
    68e9f19 View commit details
    Browse the repository at this point in the history

  10. import float8_experimental inside enable_fp8_linear 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    9de67ff View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. import float8_experimental only when needed 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    367507f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'pytorch:main' into fsdp2

    @weifengpy
    weifengpy authored Jul 17, 2024
    Configuration menu
    Copy the full SHA
    de3de0e View commit details
    Browse the repository at this point in the history

  3. skip CI on non-H100 GPUs 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    1ed8dab View commit details
    Browse the repository at this point in the history

  4. warning about sm90 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    2be380d View commit details
    Browse the repository at this point in the history

  5. Merge branch 'pytorch:main' into fsdp2

    @weifengpy
    weifengpy authored Jul 17, 2024
    Configuration menu
    Copy the full SHA
    eb7d801 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2024

  1. Merge branch 'pytorch:main' into fsdp2

    @weifengpy
    weifengpy authored Jul 18, 2024
    Configuration menu
    Copy the full SHA
    1952e4b View commit details
    Browse the repository at this point in the history

  2. add torch.compile + FSDP2 float8 all-gather to CI 

    Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
    @weifengpy
    weifengpy committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    121acae View commit details
    Browse the repository at this point in the history