Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhancing SFT Training Efficiency Using Packing and FlashAttention2 with Position IDs #31629

Merged
merged 13 commits into from
Jul 23, 2024
Merged

Enhancing SFT Training Efficiency Using Packing and FlashAttention2 with Position IDs #31629

merged 13 commits into from
Jul 23, 2024

Commits on Jul 23, 2024

  1. add DataCollatorBatchFlattening

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    df3c9b2 View commit details
    Browse the repository at this point in the history

  2. Update data_collator.py

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    dfe08de View commit details
    Browse the repository at this point in the history

  3. change name

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    8120b3a View commit details
    Browse the repository at this point in the history

  4. new FA2 flow if position_ids is provided

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    0598510 View commit details
    Browse the repository at this point in the history

  5. add comments

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    1ff2343 View commit details
    Browse the repository at this point in the history

  6. minor fix

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    f97ab71 View commit details
    Browse the repository at this point in the history

  7. minor fix data collator

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    08e1f2c View commit details
    Browse the repository at this point in the history

  8. add test cases for models

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    48ce3d2 View commit details
    Browse the repository at this point in the history

  9. add test case for data collator

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    6c6b168 View commit details
    Browse the repository at this point in the history

  10. remove extra code

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    00e7abf View commit details
    Browse the repository at this point in the history

  11. formating for ruff check and check_repo.py

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    3a87293 View commit details
    Browse the repository at this point in the history

  12. ruff format 

    ruff format tests src utils
    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    c60b720 View commit details
    Browse the repository at this point in the history

  13. custom_init_isort.py

    @RhuiDih
    RhuiDih committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    9030559 View commit details
    Browse the repository at this point in the history