Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add dynamic batching embedding/reranking #774

Merged
merged 43 commits into from
Nov 6, 2024
Merged

add dynamic batching embedding/reranking #774

merged 43 commits into from
Nov 6, 2024

Commits on Oct 9, 2024

  1. draft static batching embedding/reranking on single gaudi card

    @Spycsh
    Spycsh committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    43da415 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    36d3d95 View commit details
    Browse the repository at this point in the history

  3. fix

    @Spycsh
    Spycsh committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    e72938d View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. resolve segfault, deadlock and other issues

    @Spycsh
    Spycsh committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    8bd8dbd View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    e036caf View commit details
    Browse the repository at this point in the history

  3. narrow down default timeout

    @Spycsh
    Spycsh committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    3464641 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. Merge branch 'main' into static_batching

    @Spycsh
    Spycsh authored Oct 11, 2024
    Configuration menu
    Copy the full SHA
    18fa6c8 View commit details
    Browse the repository at this point in the history

  2. add doockerfile

    @Spycsh
    Spycsh committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    353815a View commit details
    Browse the repository at this point in the history

  3. fix hpu local microservice start

    @Spycsh
    Spycsh committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    24f3c3d View commit details
    Browse the repository at this point in the history

  4. openai format

    @Spycsh
    Spycsh committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    efc37d8 View commit details
    Browse the repository at this point in the history

  5. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    e20ac3b View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2024

  1. configurable timeout

    @Spycsh
    Spycsh committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    48a4a52 View commit details
    Browse the repository at this point in the history

  2. lower timeout

    @Spycsh
    Spycsh committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    8183f3b View commit details
    Browse the repository at this point in the history

  3. fix

    @Spycsh
    Spycsh committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    2ebc439 View commit details
    Browse the repository at this point in the history

  4. lower default timeout

    @Spycsh
    Spycsh committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    43633df View commit details
    Browse the repository at this point in the history

  5. bf16

    @Spycsh
    Spycsh committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    b51d219 View commit details
    Browse the repository at this point in the history

  6. log, pad max_len

    @Spycsh
    Spycsh committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    08d5c05 View commit details
    Browse the repository at this point in the history

  7. autocast, 128

    @Spycsh
    Spycsh committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    7329404 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. fix acc issue

    @Spycsh
    Spycsh committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    6396e54 View commit details
    Browse the repository at this point in the history

  2. perf fallback with no acc drop

    @Spycsh
    Spycsh committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    9df1e8b View commit details
    Browse the repository at this point in the history

  3. revert no-padding ones

    @Spycsh
    Spycsh committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    8ae56eb View commit details
    Browse the repository at this point in the history

  4. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    8ff9a59 View commit details
    Browse the repository at this point in the history

  5. fix hpu graph wrapper

    @Spycsh
    Spycsh committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    08b11ec View commit details
    Browse the repository at this point in the history

  6. Merge branch 'static_batching' of https://github.com/Spycsh/GenAIComps … 

    …into static_batching
    @Spycsh
    Spycsh committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    33698f9 View commit details
    Browse the repository at this point in the history

  7. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    6492e70 View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2024

  1. add padding batch

    @Spycsh
    Spycsh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    8f8861c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'static_batching' of https://github.com/Spycsh/GenAIComps … 

    …into static_batching
    @Spycsh
    Spycsh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    268600a View commit details
    Browse the repository at this point in the history

  3. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    4411462 View commit details
    Browse the repository at this point in the history

  4. habana 1.18

    @Spycsh
    Spycsh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    51c3306 View commit details
    Browse the repository at this point in the history

  5. static -> dynamic

    @Spycsh
    Spycsh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    357f302 View commit details
    Browse the repository at this point in the history

  6. add UT, add param in_single_process

    @Spycsh
    Spycsh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    c3aaed7 View commit details
    Browse the repository at this point in the history

  7. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    a556581 View commit details
    Browse the repository at this point in the history

  8. add docker file

    @Spycsh
    Spycsh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    fda14f5 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'static_batching' of https://github.com/Spycsh/GenAIComps … 

    …into static_batching
    @Spycsh
    Spycsh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    b7d8a5e View commit details
    Browse the repository at this point in the history

  10. Merge branch 'main' into static_batching

    @Spycsh
    Spycsh authored Oct 15, 2024
    Configuration menu
    Copy the full SHA
    7ac3324 View commit details
    Browse the repository at this point in the history

  11. fix case doc empty, and pass model id from env

    @Spycsh
    Spycsh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    d9ac817 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'static_batching' of https://github.com/Spycsh/GenAIComps … 

    …into static_batching
    @Spycsh
    Spycsh committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    a337c2d View commit details
    Browse the repository at this point in the history

  13. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    ab21b1d View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. Merge branch 'main' into static_batching

    @chensuyue
    chensuyue authored Oct 17, 2024
    Configuration menu
    Copy the full SHA
    d5afdc6 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. Merge branch 'main' into static_batching

    @Spycsh
    Spycsh authored Oct 18, 2024
    Configuration menu
    Copy the full SHA
    f42e7d8 View commit details
    Browse the repository at this point in the history

  2. CI

    @Spycsh
    Spycsh committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    4fd090d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'static_batching' of https://github.com/Spycsh/GenAIComps … 

    …into static_batching
    @Spycsh
    Spycsh committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    3fbe230 View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2024

  1. Merge branch 'main' into static_batching

    @ZePan110
    ZePan110 authored Nov 4, 2024
    Configuration menu
    Copy the full SHA
    8526275 View commit details
    Browse the repository at this point in the history