Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a inplace concat custom op based on CUDA VMM API (resubmitted) #9320

Open
wants to merge 24 commits into
base: develop
Choose a base branch
from
Open

Add a inplace concat custom op based on CUDA VMM API (resubmitted) #9320

wants to merge 24 commits into from

Commits on Aug 26, 2024

  1. add a PIR pass to remove the pd_op.assign_out_ op at the end of while… 

    … loop, avoiding redundant kv cache copy
    @lszxb
    lszxb committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    b1ebe45 View commit details
    Browse the repository at this point in the history

  2. avoid cuda sync in postprocess of LLM decoding

    @lszxb
    lszxb committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    16b554b View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. Revert workarounds about full_with_tensor as it will fix by paddle fr… 

    …amework
    @lszxb
    lszxb committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    5ece22d View commit details
    Browse the repository at this point in the history

  2. add vtensor custom operator and support for llama

    @lszxb
    lszxb committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    31047f7 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2024

  1. add transform pass for vtensor and update vtensor impl

    @lszxb
    lszxb committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    1992697 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into fix_remove_assign_out_in_while_loop

    @lszxb
    lszxb committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    21807cd View commit details
    Browse the repository at this point in the history

  3. update remove_assign_out_pass, now it only trigger with cf.yield

    @lszxb
    lszxb committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    20aa163 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'fix_remove_assign_out_in_while_loop' into add_vtensor_op

    @lszxb
    lszxb committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    b645e16 View commit details
    Browse the repository at this point in the history

  5. comment out debug info in apply_vtensor_concat_pass

    @lszxb
    lszxb committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    9abb6a7 View commit details
    Browse the repository at this point in the history

  6. add apply_vtensor_concat_pass to setup.py

    @lszxb
    lszxb committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    dfe0ce3 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. update vtensor op: add sync before unmap

    @lszxb
    lszxb committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    c174b8f View commit details
    Browse the repository at this point in the history

  2. enable apply_vtensor_concat_pass in predictor.py

    @lszxb
    lszxb committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    b13e79d View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. update remove_assign_out_pass: add more constraints to improve robust… 

    …ness
    @lszxb
    lszxb committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    7058e45 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'fix_remove_assign_out_in_while_loop' into add_vtensor_op

    @lszxb
    lszxb committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    9bca022 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2024

  1. Merge branch 'develop' into fix_remove_assign_out_in_while_loop

    @lszxb
    lszxb committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    a95f16e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into fix_avoid_cuda_sync

    @lszxb
    lszxb committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    53e229a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'fix_remove_assign_out_in_while_loop' into add_vtensor_op

    @lszxb
    lszxb committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    b3b9ec3 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'fix_avoid_cuda_sync' into all_fix

    @lszxb
    lszxb committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    85e0654 View commit details
    Browse the repository at this point in the history

  5. Add copyright for new files

    @lszxb
    lszxb committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    64b75da View commit details
    Browse the repository at this point in the history

  6. Remove comments

    @lszxb
    lszxb committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    018e166 View commit details
    Browse the repository at this point in the history

  7. Format codes

    @lszxb
    lszxb committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    ca1aaf7 View commit details
    Browse the repository at this point in the history

  8. disable vtensor on hip

    @lszxb
    lszxb committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    31b2dcd View commit details
    Browse the repository at this point in the history

  9. format python code by autopep8

    @lszxb
    lszxb committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    9fdda07 View commit details
    Browse the repository at this point in the history

  10. Lint code

    @lszxb
    lszxb committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    d5a9393 View commit details
    Browse the repository at this point in the history