Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add the DistributedFusedLamb optimizer #39148

Merged
merged 24 commits into from
Feb 18, 2022
Merged

Add the DistributedFusedLamb optimizer #39148

merged 24 commits into from
Feb 18, 2022

Commits on Jan 22, 2022

  1. add DistributedFusedLamb op

    @sneaxiy
    sneaxiy committed Jan 22, 2022
    Configuration menu
    Copy the full SHA
    fdb70be View commit details
    Browse the repository at this point in the history

  2. polish code

    @sneaxiy
    sneaxiy committed Jan 22, 2022
    Configuration menu
    Copy the full SHA
    4751b5a View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2022

  1. Merge upstream/develop

    @sneaxiy
    sneaxiy committed Jan 23, 2022
    Configuration menu
    Copy the full SHA
    a07c50b View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2022

  1. fix compile error

    @sneaxiy
    sneaxiy committed Jan 24, 2022
    Configuration menu
    Copy the full SHA
    3cc786a View commit details
    Browse the repository at this point in the history

  2. Merge upstream/develop

    @sneaxiy
    sneaxiy committed Jan 24, 2022
    Configuration menu
    Copy the full SHA
    8e48865 View commit details
    Browse the repository at this point in the history

  3. compatible with pten changement

    @sneaxiy
    sneaxiy committed Jan 24, 2022
    Configuration menu
    Copy the full SHA
    dc5e176 View commit details
    Browse the repository at this point in the history

  4. Merge develop to solve conflict

    @sneaxiy
    sneaxiy committed Jan 24, 2022
    Configuration menu
    Copy the full SHA
    1f78e2b View commit details
    Browse the repository at this point in the history

  5. fix rocm compile error

    @sneaxiy
    sneaxiy committed Jan 24, 2022
    Configuration menu
    Copy the full SHA
    ffcac8e View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2022

  1. improve converage

    @sneaxiy
    sneaxiy committed Jan 25, 2022
    Configuration menu
    Copy the full SHA
    b9a7f57 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2022

  1. Merge upstream/develop

    @sneaxiy
    sneaxiy committed Feb 5, 2022
    Configuration menu
    Copy the full SHA
    5a2b664 View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2022

  1. Merge upstream/develop

    @sneaxiy
    sneaxiy committed Feb 8, 2022
    Configuration menu
    Copy the full SHA
    cbc7671 View commit details
    Browse the repository at this point in the history

  2. update upstream/develop

    @sneaxiy
    sneaxiy committed Feb 8, 2022
    Configuration menu
    Copy the full SHA
    71b91d4 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2022

  1. Merge upstream/develop to solve conflict

    @sneaxiy
    sneaxiy committed Feb 9, 2022
    Configuration menu
    Copy the full SHA
    f693843 View commit details
    Browse the repository at this point in the history

Commits on Feb 10, 2022

  1. Merge branch 'develop' into add_dist_fused_lamb

    @sneaxiy
    sneaxiy authored Feb 10, 2022
    Configuration menu
    Copy the full SHA
    9bab15c View commit details
    Browse the repository at this point in the history

Commits on Feb 11, 2022

  1. Merge upstream/develop

    @sneaxiy
    sneaxiy committed Feb 11, 2022
    Configuration menu
    Copy the full SHA
    55a1a4e View commit details
    Browse the repository at this point in the history

  2. fix cast_with_ptr.h

    @sneaxiy
    sneaxiy committed Feb 11, 2022
    Configuration menu
    Copy the full SHA
    ffd14f4 View commit details
    Browse the repository at this point in the history

  3. add FLAGS_distributed_lamb_divide_nranks_when_allreduce=1

    @sneaxiy
    sneaxiy committed Feb 11, 2022
    Configuration menu
    Copy the full SHA
    6528c34 View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2022

  1. fix clip before allreduce

    @sneaxiy
    sneaxiy committed Feb 12, 2022
    Configuration menu
    Copy the full SHA
    5c82d3b View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2022

  1. Merge upstream/develop

    @sneaxiy
    sneaxiy committed Feb 14, 2022
    Configuration menu
    Copy the full SHA
    52f1b9f View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2022

  1. add use_master_param_norm

    @sneaxiy
    sneaxiy committed Feb 15, 2022
    Configuration menu
    Copy the full SHA
    93859d8 View commit details
    Browse the repository at this point in the history

  2. code polish

    @sneaxiy
    sneaxiy committed Feb 15, 2022
    Configuration menu
    Copy the full SHA
    06944e8 View commit details
    Browse the repository at this point in the history

  3. Merge upstream/develop

    @sneaxiy
    sneaxiy committed Feb 15, 2022
    Configuration menu
    Copy the full SHA
    1806633 View commit details
    Browse the repository at this point in the history

  4. fix bug

    @sneaxiy
    sneaxiy committed Feb 15, 2022
    Configuration menu
    Copy the full SHA
    d5f3392 View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2022

  1. fix ROCM ci

    @sneaxiy
    sneaxiy committed Feb 16, 2022
    1 Configuration menu
    Copy the full SHA
    dc8c2eb View commit details
    Browse the repository at this point in the history