Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Kernel] Flashinfer for prefill & decode, with Cudagraph support for decode #4628

Merged
merged 47 commits into from
Jun 28, 2024
Merged

[Kernel] Flashinfer for prefill & decode, with Cudagraph support for decode #4628

merged 47 commits into from
Jun 28, 2024

Commits on May 6, 2024

  1. flashinfer for prefill

    LiuXiaoxuanPKU committed May 6, 2024
    Configuration menu
    Copy the full SHA
    0eb1ab1 View commit details
    Browse the repository at this point in the history

  2. minor

    LiuXiaoxuanPKU committed May 6, 2024
    Configuration menu
    Copy the full SHA
    4590b46 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. fix docker

    LiuXiaoxuanPKU committed May 7, 2024
    Configuration menu
    Copy the full SHA
    3bfbdf7 View commit details
    Browse the repository at this point in the history

  2. work for prefix caching

    LiuXiaoxuanPKU committed May 7, 2024
    Configuration menu
    Copy the full SHA
    993a4ae View commit details
    Browse the repository at this point in the history

  3. dedup test

    LiuXiaoxuanPKU committed May 7, 2024
    Configuration menu
    Copy the full SHA
    b4d9dae View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. Merge branch 'main' into flashinfer-prefill

    LiuXiaoxuanPKU committed May 28, 2024
    Configuration menu
    Copy the full SHA
    eb2d18e View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. format

    LiuXiaoxuanPKU committed May 29, 2024
    Configuration menu
    Copy the full SHA
    5e3d11d View commit details
    Browse the repository at this point in the history

  2. fix test

    LiuXiaoxuanPKU committed May 29, 2024
    Configuration menu
    Copy the full SHA
    89f0e2c View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. remove flashinfer from ci

    LiuXiaoxuanPKU committed May 30, 2024
    Configuration menu
    Copy the full SHA
    72e704b View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2024

  1. wip, cuda graph for decode

    LiuXiaoxuanPKU committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    f9770ed View commit details
    Browse the repository at this point in the history

  2. wip

    LiuXiaoxuanPKU committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    f1849f7 View commit details
    Browse the repository at this point in the history

  3. pass tests

    LiuXiaoxuanPKU committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    88425a3 View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2024

  1. wip

    LiuXiaoxuanPKU committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    74a8eeb View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. pass simple tests, need more fix for correctness

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    dcbbfd6 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2024

  1. optimizer prepare input

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    4302848 View commit details
    Browse the repository at this point in the history

  2. padding

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    d739312 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into flashinfer-prefill

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    e5017e2 View commit details
    Browse the repository at this point in the history

  4. style

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    5ad175a View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. share workspace buffer to reduce cudagraph extra memory cost

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    543dc3b View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2024

  1. address comments

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    11b7347 View commit details
    Browse the repository at this point in the history

  2. fix

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    b5db4be View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. fix comments

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    f53d03e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into flashinfer-prefill

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    6fb1b6d View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. support TP > 1

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    e05ff79 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. try CI

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    8f685dd View commit details
    Browse the repository at this point in the history

  2. minor

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    0f8e7a1 View commit details
    Browse the repository at this point in the history

  3. format

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    cf275a1 View commit details
    Browse the repository at this point in the history

  4. minor

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    0ab32ee View commit details
    Browse the repository at this point in the history

  5. try CI

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    c421f1f View commit details
    Browse the repository at this point in the history

  6. flash attention dependency

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    815efc2 View commit details
    Browse the repository at this point in the history

  7. minor

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    901b369 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. flash attn

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    b2d9895 View commit details
    Browse the repository at this point in the history

  2. format

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    df16a6b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into flashinfer-prefill

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    64a24cb View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2024

  1. fix ci

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    dc4e7ef View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2024

  1. Merge branch 'main' into flashinfer-prefill

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    9774919 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. use llama3-8b in test and add warning

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    aeb0df6 View commit details
    Browse the repository at this point in the history

  2. fix

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    8a72dcf View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. remove amd tests

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    aaddbad View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into flashinfer-prefill

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    4aa2069 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. fix

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    e61bd38 View commit details
    Browse the repository at this point in the history

  2. minor

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    b2484df View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. fix

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    0f4f796 View commit details
    Browse the repository at this point in the history

  2. change buffer init

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    3dca2f0 View commit details
    Browse the repository at this point in the history

  3. fix ci

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    7853235 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into flashinfer-prefill

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    8316bc3 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into flashinfer-prefill

    @LiuXiaoxuanPKU
    LiuXiaoxuanPKU committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    d5348f1 View commit details
    Browse the repository at this point in the history