Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[core ] Integrate Flash attention 2 in most used models #25598

Merged
merged 99 commits into from
Sep 22, 2023
Merged

[core ] Integrate Flash attention 2 in most used models #25598

merged 99 commits into from
Sep 22, 2023

Commits on Aug 18, 2023

  1. v1

    @younesbelkada
    younesbelkada committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    8bb77a1 View commit details
    Browse the repository at this point in the history

  2. oops

    @younesbelkada
    younesbelkada committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    2e18421 View commit details
    Browse the repository at this point in the history

  3. working v1

    @younesbelkada
    younesbelkada committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    fe5795e View commit details
    Browse the repository at this point in the history

  4. fixup

    @younesbelkada
    younesbelkada committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    4bd15e2 View commit details
    Browse the repository at this point in the history

  5. add some TODOs

    @younesbelkada
    younesbelkada committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    49fe318 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'add-flash-attn-2' of https://github.com/younesbelkada/t… 

    …ransformers into HEAD
    @younesbelkada
    younesbelkada committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    f5d440b View commit details
    Browse the repository at this point in the history

  7. fixup

    @younesbelkada
    younesbelkada committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    50491e8 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2023

  1. Merge remote-tracking branch 'upstream/main' into add-flash-attn-2

    @younesbelkada
    younesbelkada committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    7df78c0 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2023

  1. padding support + try with module replacement

    @younesbelkada
    younesbelkada committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    0e30d13 View commit details
    Browse the repository at this point in the history

  2. nit

    @younesbelkada
    younesbelkada committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    ad8b905 View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2023

  1. alternative design

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    3c31f10 View commit details
    Browse the repository at this point in the history

  2. oops

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    2628bf3 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/main' into HEAD

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    56d0b49 View commit details
    Browse the repository at this point in the history

  4. add use_cache support for llama

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    20d1b37 View commit details
    Browse the repository at this point in the history

  5. v1 falcon

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    a82f1ca View commit details
    Browse the repository at this point in the history

  6. nit

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    c72e8ff View commit details
    Browse the repository at this point in the history

  7. a bit of refactor

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    66823f9 View commit details
    Browse the repository at this point in the history

  8. nit

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    41f8f3d View commit details
    Browse the repository at this point in the history

  9. nits nits

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    a64a1a9 View commit details
    Browse the repository at this point in the history

  10. add v1 padding support falcon (even though it seemed to work before)

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    67e3fc2 View commit details
    Browse the repository at this point in the history

  11. nit

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    8444ab6 View commit details
    Browse the repository at this point in the history

  12. falcon works

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    8b1c2df View commit details
    Browse the repository at this point in the history

  13. fixup

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    c3ebcd2 View commit details
    Browse the repository at this point in the history

  14. v1 tests

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    1c212d8 View commit details
    Browse the repository at this point in the history

  15. nit

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    4618701 View commit details
    Browse the repository at this point in the history

  16. fix generation llama flash

    @fxmarty
    fxmarty committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    85ec946 View commit details
    Browse the repository at this point in the history

  17. Merge branch 'add-flash-attn-2' of https://github.com/younesbelkada/t… 

    …ransformers into add-flash-attn-2
    @fxmarty
    fxmarty committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    2248f20 View commit details
    Browse the repository at this point in the history

  18. update tests

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    0881ced View commit details
    Browse the repository at this point in the history

  19. Merge branch 'add-flash-attn-2' of https://github.com/younesbelkada/t… 

    …ransformers into HEAD
    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    a8a1b2d View commit details
    Browse the repository at this point in the history

  20. fix tests + nits

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    2be3e03 View commit details
    Browse the repository at this point in the history

  21. fix copies

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    b6d3e58 View commit details
    Browse the repository at this point in the history

  22. fix nit

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    b47e85c View commit details
    Browse the repository at this point in the history

  23. test- padding mask

    @younesbelkada
    younesbelkada committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    db8bd64 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2023

  1. stype

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    58848ab View commit details
    Browse the repository at this point in the history

  2. add more mem efficient support

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    3f73557 View commit details
    Browse the repository at this point in the history

  3. Update src/transformers/modeling_utils.py 

    Co-authored-by: Patrick von Platen <[email protected]>
    @younesbelkada @patrickvonplaten
    younesbelkada and patrickvonplaten authored Sep 4, 2023
    Configuration menu
    Copy the full SHA
    baae736 View commit details
    Browse the repository at this point in the history

  4. fixup

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    55f6140 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'upstream/main' into add-flash-attn-2

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    10d5c1b View commit details
    Browse the repository at this point in the history

  6. nit

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    3fb221a View commit details
    Browse the repository at this point in the history

  7. fixup

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    a931aeb View commit details
    Browse the repository at this point in the history

  8. remove it from config when saving

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    68a1204 View commit details
    Browse the repository at this point in the history

  9. fixup

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    36e0d6e View commit details
    Browse the repository at this point in the history

  10. revert docstring

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    2beeb68 View commit details
    Browse the repository at this point in the history

  11. add more checks

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    7b5da2c View commit details
    Browse the repository at this point in the history

  12. use values

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    b99a582 View commit details
    Browse the repository at this point in the history

  13. oops

    @younesbelkada
    younesbelkada committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    adaed45 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2023

  1. new version

    @fxmarty
    fxmarty committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    7f06af6 View commit details
    Browse the repository at this point in the history

  2. fixup

    @younesbelkada
    younesbelkada committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    2d36c6f View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into add-flash-attn-2

    @younesbelkada
    younesbelkada authored Sep 5, 2023
    Configuration menu
    Copy the full SHA
    a663fa4 View commit details
    Browse the repository at this point in the history

  4. add same trick for falcon

    @younesbelkada
    younesbelkada committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    9d3693f View commit details
    Browse the repository at this point in the history

  5. nit

    @younesbelkada
    younesbelkada committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    65ae59c View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2023

  1. add another test

    @younesbelkada
    younesbelkada committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    43185b5 View commit details
    Browse the repository at this point in the history

  2. change tests

    @younesbelkada
    younesbelkada committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    c61157e View commit details
    Browse the repository at this point in the history

  3. fix issues with GC and also falcon

    @younesbelkada
    younesbelkada committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    2f17792 View commit details
    Browse the repository at this point in the history

  4. fixup

    @younesbelkada
    younesbelkada committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    65c3861 View commit details
    Browse the repository at this point in the history

  5. oops

    @younesbelkada
    younesbelkada committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    165a503 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. Update src/transformers/models/falcon/modeling_falcon.py 

    Co-authored-by: Arthur <[email protected]>
    @younesbelkada @ArthurZucker
    younesbelkada and ArthurZucker authored Sep 13, 2023
    Configuration menu
    Copy the full SHA
    5abc702 View commit details
    Browse the repository at this point in the history

  2. add init_rope

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    5069e4a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'add-flash-attn-2' of https://github.com/younesbelkada/t… 

    …ransformers into add-flash-attn-2
    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    11400d8 View commit details
    Browse the repository at this point in the history

  4. updates

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    ace7939 View commit details
    Browse the repository at this point in the history

  5. fix copies

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    fe9b16d View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into add-flash-attn-2

    @younesbelkada
    younesbelkada authored Sep 13, 2023
    Configuration menu
    Copy the full SHA
    6174c06 View commit details
    Browse the repository at this point in the history

  7. fixup

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    acfc954 View commit details
    Browse the repository at this point in the history

  8. fixup

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    33a0f62 View commit details
    Browse the repository at this point in the history

  9. more clarification

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    ee8ba20 View commit details
    Browse the repository at this point in the history

  10. fixup

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    e28fb0b View commit details
    Browse the repository at this point in the history

  11. right padding tests

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    025727c View commit details
    Browse the repository at this point in the history

  12. add docs

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    8f7e400 View commit details
    Browse the repository at this point in the history

  13. add FA in docker image

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    3259392 View commit details
    Browse the repository at this point in the history

  14. more clarifications

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    57a077b View commit details
    Browse the repository at this point in the history

  15. add some figures

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    e62b0b8 View commit details
    Browse the repository at this point in the history

  16. add todo

    @younesbelkada
    younesbelkada committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    7419438 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2023

  1. rectify comment

    @younesbelkada
    younesbelkada committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    3ba5e98 View commit details
    Browse the repository at this point in the history

  2. Change to FA2

    @younesbelkada
    younesbelkada committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    585e463 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2023

  1. Update docs/source/en/perf_infer_gpu_one.md 

    Co-authored-by: Arthur <[email protected]>
    @younesbelkada @ArthurZucker
    younesbelkada and ArthurZucker authored Sep 19, 2023
    Configuration menu
    Copy the full SHA
    ec0f8b9 View commit details
    Browse the repository at this point in the history

  2. split in two lines

    @younesbelkada
    younesbelkada committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    3e5ea35 View commit details
    Browse the repository at this point in the history

  3. change test name

    @younesbelkada
    younesbelkada committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    4bb1bc5 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/main' into add-flash-attn-2

    @younesbelkada
    younesbelkada committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    3ea4633 View commit details
    Browse the repository at this point in the history

  5. add more tests

    @younesbelkada
    younesbelkada committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    b67c21e View commit details
    Browse the repository at this point in the history

  6. some clean up

    @younesbelkada
    younesbelkada committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    5b73557 View commit details
    Browse the repository at this point in the history

  7. remove rearrange deps

    @younesbelkada
    younesbelkada committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    48e3bcf View commit details
    Browse the repository at this point in the history

  8. add more docs

    @younesbelkada
    younesbelkada committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    0461384 View commit details
    Browse the repository at this point in the history

  9. revert changes on dockerfile

    @younesbelkada
    younesbelkada committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    8d72a66 View commit details
    Browse the repository at this point in the history

  10. Revert "revert changes on dockerfile" 

    This reverts commit 8d72a66.
    @younesbelkada
    younesbelkada committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    73b2f07 View commit details
    Browse the repository at this point in the history

  11. revert changes on dockerfile

    @younesbelkada
    younesbelkada committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    fb7654c View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. Apply suggestions from code review 

    Co-authored-by: Lysandre Debut <[email protected]>
    @younesbelkada @LysandreJik
    younesbelkada and LysandreJik authored Sep 20, 2023
    Configuration menu
    Copy the full SHA
    a737bde View commit details
    Browse the repository at this point in the history

  2. address some comments

    @younesbelkada
    younesbelkada committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    80951ae View commit details
    Browse the repository at this point in the history

  3. docs

    @younesbelkada
    younesbelkada committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    6f7ff42 View commit details
    Browse the repository at this point in the history

  4. use inheritance

    @younesbelkada
    younesbelkada committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    257a633 View commit details
    Browse the repository at this point in the history

  5. Update src/transformers/testing_utils.py 

    Co-authored-by: Lysandre Debut <[email protected]>
    @younesbelkada @LysandreJik
    younesbelkada and LysandreJik authored Sep 20, 2023
    Configuration menu
    Copy the full SHA
    360da70 View commit details
    Browse the repository at this point in the history

  6. fixup

    @younesbelkada
    younesbelkada committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    1d91bc4 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into add-flash-attn-2

    @younesbelkada
    younesbelkada authored Sep 20, 2023
    Configuration menu
    Copy the full SHA
    8ecab97 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. Apply suggestions from code review 

    Co-authored-by: Arthur <[email protected]>
    @younesbelkada @ArthurZucker
    younesbelkada and ArthurZucker authored Sep 21, 2023
    Configuration menu
    Copy the full SHA
    7c5720f View commit details
    Browse the repository at this point in the history

  2. Update src/transformers/modeling_utils.py

    @younesbelkada
    younesbelkada authored Sep 21, 2023
    Configuration menu
    Copy the full SHA
    28b82e2 View commit details
    Browse the repository at this point in the history

  3. final comments

    @younesbelkada
    younesbelkada committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    84b5793 View commit details
    Browse the repository at this point in the history

  4. clean up

    @younesbelkada
    younesbelkada committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    949172f View commit details
    Browse the repository at this point in the history

  5. style

    @younesbelkada
    younesbelkada committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    825c7e0 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2023

  1. add cast + warning for PEFT models

    @younesbelkada
    younesbelkada committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    1af232c View commit details
    Browse the repository at this point in the history

  2. fixup

    @younesbelkada
    younesbelkada committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    d7f16c5 View commit details
    Browse the repository at this point in the history