Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI/Build] Add nightly benchmarking for tgi, tensorrt-llm and lmdeploy #5362

Merged
merged 152 commits into from
Jul 11, 2024
Merged

[CI/Build] Add nightly benchmarking for tgi, tensorrt-llm and lmdeploy #5362

merged 152 commits into from
Jul 11, 2024

Commits on Jun 9, 2024

  1. Kuntai: add tgi and trt benchmarking script (initial version)

    @KuntaiDu
    KuntaiDu committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    f813e2e View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. update initial benchmarking script for lmdeploy

    @KuntaiDu
    KuntaiDu committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    d6cba46 View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Merge branch 'vllm-project:main' into kuntai-benchmark-dev

    @KuntaiDu
    KuntaiDu committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    2cc1023 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. Add download tokenizer script for lmdeploy

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    5d8292b View commit details
    Browse the repository at this point in the history

  2. add one-click runnable script for lmdeploy, parse tests from json file

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    a2dd7c9 View commit details
    Browse the repository at this point in the history

  3. add nightly test json file

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    8416ce6 View commit details
    Browse the repository at this point in the history

  4. bug fix on tokenizer directory

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    df4ba8f View commit details
    Browse the repository at this point in the history

  5. bug fix on getting model

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    b974495 View commit details
    Browse the repository at this point in the history

  6. update test cases

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    80d1c77 View commit details
    Browse the repository at this point in the history

  7. update parameter name

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    b3f3b0e View commit details
    Browse the repository at this point in the history

  8. typo fix

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    9483acf View commit details
    Browse the repository at this point in the history

  9. add wait_for_server

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    d72ae51 View commit details
    Browse the repository at this point in the history

  10. update summarization script

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    0e819f0 View commit details
    Browse the repository at this point in the history

  11. use pkill tp kill lmdeploy

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    9181a1d View commit details
    Browse the repository at this point in the history

  12. update script for tgi

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    6e1936c View commit details
    Browse the repository at this point in the history

  13. add install jq

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    c6aded9 View commit details
    Browse the repository at this point in the history

  14. reduce 7b llama tp to 1

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    ccbcd18 View commit details
    Browse the repository at this point in the history

  15. update lmdeploy tp

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    38cc38a View commit details
    Browse the repository at this point in the history

  16. bug fix

    @KuntaiDu
    KuntaiDu committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    832891e View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. update tensorrt script

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    5877806 View commit details
    Browse the repository at this point in the history

  2. update nightly suite

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    9972aba View commit details
    Browse the repository at this point in the history

  3. add double quote

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    6493679 View commit details
    Browse the repository at this point in the history

  4. add trt llm version

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    e62cae6 View commit details
    Browse the repository at this point in the history

  5. update trt

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    9ce3589 View commit details
    Browse the repository at this point in the history

  6. update on how to kill the server

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    f634dee View commit details
    Browse the repository at this point in the history

  7. update vllm nightly test

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    b47e30b View commit details
    Browse the repository at this point in the history

  8. disalbe vllm server log

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    ec8b295 View commit details
    Browse the repository at this point in the history

  9. adjust how to kill processes in vllm

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    792ef7f View commit details
    Browse the repository at this point in the history

  10. update nightly tests

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    4f67c96 View commit details
    Browse the repository at this point in the history

  11. update summary results

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    f0fe30c View commit details
    Browse the repository at this point in the history

  12. update summary results

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    d097843 View commit details
    Browse the repository at this point in the history

  13. add upload_to_buildkite utility

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    2c30f38 View commit details
    Browse the repository at this point in the history

  14. update kickoff pipeline to initiate nightly benchmark

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    7304668 View commit details
    Browse the repository at this point in the history

  15. update kickoff pipeline

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    f811ef0 View commit details
    Browse the repository at this point in the history

  16. update the label name

    @KuntaiDu
    KuntaiDu committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    1876048 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. bug fix: exit benchmarking script after finish benchmarking one appli… 

    …cation
    @KuntaiDu
    KuntaiDu committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    0a4518d View commit details
    Browse the repository at this point in the history

  2. make yapf, ruff and isort happy

    @KuntaiDu
    KuntaiDu committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    f47db88 View commit details
    Browse the repository at this point in the history

  3. give nightly pipeline higher priority

    @KuntaiDu
    KuntaiDu committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    8f4da1b View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2024

  1. fix new bugs in latest lmdeploy docker

    @KuntaiDu
    KuntaiDu committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    2cbdac3 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. try llama 70B with tp 4

    @KuntaiDu
    KuntaiDu committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    c24b963 View commit details
    Browse the repository at this point in the history

  2. rebuild

    @KuntaiDu
    KuntaiDu committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    f79b6c4 View commit details
    Browse the repository at this point in the history

  3. use mixtral model to prevent disk quota exceeded

    @KuntaiDu
    KuntaiDu committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    dfb77f4 View commit details
    Browse the repository at this point in the history

  4. remove wait

    @KuntaiDu
    KuntaiDu committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    d2e4171 View commit details
    Browse the repository at this point in the history

  5. temporarily remove trt pipeline --- disk quota exceeded

    @KuntaiDu
    KuntaiDu committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    dc52195 View commit details
    Browse the repository at this point in the history

  6. fall back to 70B and test the storage required

    @KuntaiDu
    KuntaiDu committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    8ffd8b1 View commit details
    Browse the repository at this point in the history

  7. use llama-2 as I do not have llama3 access...)

    @KuntaiDu
    KuntaiDu committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    313f54f View commit details
    Browse the repository at this point in the history

  8. fix model name

    @KuntaiDu
    KuntaiDu committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    62b2407 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. try llama 70B

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    785d246 View commit details
    Browse the repository at this point in the history

  2. check file system size

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    7fd891e View commit details
    Browse the repository at this point in the history

  3. update code for removing cache

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    52cf795 View commit details
    Browse the repository at this point in the history

  4. merge common parameters

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    b8d1c94 View commit details
    Browse the repository at this point in the history

  5. fix typo

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    14fb650 View commit details
    Browse the repository at this point in the history

  6. reduce qps to 8, just for testing

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    3aba28a View commit details
    Browse the repository at this point in the history

  7. append to the same context

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    733ac33 View commit details
    Browse the repository at this point in the history

  8. optimize for buildkite annotation

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    5e1ec4b View commit details
    Browse the repository at this point in the history

  9. add double enter for md table

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    2423106 View commit details
    Browse the repository at this point in the history

  10. format adjust for markdown presentation

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    d3f9701 View commit details
    Browse the repository at this point in the history

  11. move header to the description file

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    c4d651b View commit details
    Browse the repository at this point in the history

  12. separate annotation to a new step

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    25c5a2f View commit details
    Browse the repository at this point in the history

  13. add extra step to annotate pipeline

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    5183fea View commit details
    Browse the repository at this point in the history

  14. add wait

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    f0684af View commit details
    Browse the repository at this point in the history

  15. bring back the full test case

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    6c7ddf8 View commit details
    Browse the repository at this point in the history

  16. fix syntax error

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    13f5d99 View commit details
    Browse the repository at this point in the history

  17. break when the server failed to start --- so that the buildkite uploa… 

    …ding still works
    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    11079c7 View commit details
    Browse the repository at this point in the history

  18. make yapf happy

    @KuntaiDu
    KuntaiDu committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    0ed8131 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. test vllm code

    @KuntaiDu
    KuntaiDu committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    fae306e View commit details
    Browse the repository at this point in the history

  2. check if mounting is successfull

    @KuntaiDu
    KuntaiDu committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    5098e10 View commit details
    Browse the repository at this point in the history

  3. add pwd

    @KuntaiDu
    KuntaiDu committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    b0e7667 View commit details
    Browse the repository at this point in the history

  4. add ls

    @KuntaiDu
    KuntaiDu committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    4034f5f View commit details
    Browse the repository at this point in the history

  5. update to read code from the docker, instead of running wget

    @KuntaiDu
    KuntaiDu committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    4ab2eca View commit details
    Browse the repository at this point in the history

  6. remove raid

    @KuntaiDu
    KuntaiDu committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    d6a34d3 View commit details
    Browse the repository at this point in the history

  7. try Roger's fix

    @KuntaiDu
    KuntaiDu committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    c57ac0a View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. remove nvme raid

    @KuntaiDu
    KuntaiDu committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    d75d45b View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. raise the priority of benchmarking development jobs

    @KuntaiDu
    KuntaiDu committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    5dc8c8c View commit details
    Browse the repository at this point in the history

  2. reduce the # of test from 1000 to 500, for faster testing

    @KuntaiDu
    KuntaiDu committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    8b91927 View commit details
    Browse the repository at this point in the history

  3. trt won't run all the test. Just run llama-3 70B. Fix this bug tomorrow

    @KuntaiDu
    KuntaiDu committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    8539874 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2024

  1. debug tensorrt

    @KuntaiDu
    KuntaiDu committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    144328b View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. bug fix: avoid reassigning params during the for loop

    @KuntaiDu
    KuntaiDu committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    64e9518 View commit details
    Browse the repository at this point in the history

  2. bring lmdeploy back for testing

    @KuntaiDu
    KuntaiDu committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    a94c140 View commit details
    Browse the repository at this point in the history

  3. change test name

    @KuntaiDu
    KuntaiDu committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    1f0ccb0 View commit details
    Browse the repository at this point in the history

  4. separating run server command from the bash file

    @KuntaiDu
    KuntaiDu committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    2f53b96 View commit details
    Browse the repository at this point in the history

  5. clean up

    @KuntaiDu
    KuntaiDu committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    51d679e View commit details
    Browse the repository at this point in the history

  6. run lmdeploy server in a separate process

    @KuntaiDu
    KuntaiDu committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    21c986d View commit details
    Browse the repository at this point in the history

  7. bring back the full test suite

    @KuntaiDu
    KuntaiDu committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    96bc249 View commit details
    Browse the repository at this point in the history

  8. bug fix: need to use llama checkpoint converter for mixtral model

    @KuntaiDu
    KuntaiDu committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    6c566cb View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. reduce test case to only mixtral, debug lmdeploy + mixtral

    @KuntaiDu
    KuntaiDu committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    162700f View commit details
    Browse the repository at this point in the history

  2. developing fp8 + tensorrt-llm

    @KuntaiDu
    KuntaiDu committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    b0d74cd View commit details
    Browse the repository at this point in the history

  3. move fp8 quantization to common parameters

    @KuntaiDu
    KuntaiDu committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    f1a7955 View commit details
    Browse the repository at this point in the history

  4. add fp8 for vllm

    @KuntaiDu
    KuntaiDu committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    459fb2f View commit details
    Browse the repository at this point in the history

  5. remove unused parameters

    @KuntaiDu
    KuntaiDu committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    79b295c View commit details
    Browse the repository at this point in the history

  6. use llama2 for local debugging

    @KuntaiDu
    KuntaiDu committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    019802a View commit details
    Browse the repository at this point in the history

  7. move kv cache dtype inside vllm

    @KuntaiDu
    KuntaiDu committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    3d20f92 View commit details
    Browse the repository at this point in the history

  8. change model

    @KuntaiDu
    KuntaiDu committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    44e2d97 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. test fp8 performance

    @KuntaiDu
    KuntaiDu committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    b8dbd8a View commit details
    Browse the repository at this point in the history

  2. reduce calib size

    @KuntaiDu
    KuntaiDu committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    0313c19 View commit details
    Browse the repository at this point in the history

  3. freeze fp16 benchmark

    @KuntaiDu
    KuntaiDu committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    7b483a1 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'vllm-project:main' into kuntai-benchmark-dev

    @KuntaiDu
    KuntaiDu committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    2e577b3 View commit details
    Browse the repository at this point in the history

  5. add standard deviation for each metric -- to plot confidence interval

    @KuntaiDu
    KuntaiDu committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    c5e6662 View commit details
    Browse the repository at this point in the history

  6. remove annotation inside the job --- run the annotation at the last.

    @KuntaiDu
    KuntaiDu committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    22e78b5 View commit details
    Browse the repository at this point in the history

  7. reduce nightly pipeline length

    @KuntaiDu
    KuntaiDu committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    59072ed View commit details
    Browse the repository at this point in the history

  8. remove headers in result

    @KuntaiDu
    KuntaiDu committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    a3e4355 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2024

  1. add visualization step

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    e27677a View commit details
    Browse the repository at this point in the history

  2. support figure visualization

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    7c845ae View commit details
    Browse the repository at this point in the history

  3. adjust visualization

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    3a70a60 View commit details
    Browse the repository at this point in the history

  4. visual adjustment

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    8260d38 View commit details
    Browse the repository at this point in the history

  5. remove text annotation

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    4643749 View commit details
    Browse the repository at this point in the history

  6. add padding

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    3146a96 View commit details
    Browse the repository at this point in the history

  7. add hyperlink

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    6da59d1 View commit details
    Browse the repository at this point in the history

  8. bring back the full suite of test

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    0802f9f View commit details
    Browse the repository at this point in the history

  9. adjust test order

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    8e6fca2 View commit details
    Browse the repository at this point in the history

  10. bring back full benchmark suite

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    a77fcbd View commit details
    Browse the repository at this point in the history

  11. add more pad

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    8b51f45 View commit details
    Browse the repository at this point in the history

  12. mount huggingface cache

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    4427b06 View commit details
    Browse the repository at this point in the history

  13. mount huggingface cache

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    329efe6 View commit details
    Browse the repository at this point in the history

  14. add even more padding

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    a174d26 View commit details
    Browse the repository at this point in the history

  15. add illustration

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    3f49b0c View commit details
    Browse the repository at this point in the history

  16. make yapf and ruff happy

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    5c3a7d0 View commit details
    Browse the repository at this point in the history

  17. add datetime to filename and make yapf happy

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    ec6f42d View commit details
    Browse the repository at this point in the history

  18. debug mixtral and llama70B

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    0090577 View commit details
    Browse the repository at this point in the history

  19. pin lmdeploy transformers to 4.41.2

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    f76a04a View commit details
    Browse the repository at this point in the history

  20. skip the test case instead of exit the whoel test suite

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    859d6f3 View commit details
    Browse the repository at this point in the history

  21. update transformers for mixtral model

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    3ce4f5f View commit details
    Browse the repository at this point in the history

  22. move transformers update

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    646114d View commit details
    Browse the repository at this point in the history

  23. typo fix

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    b6058aa View commit details
    Browse the repository at this point in the history

  24. bring back the full test suite

    @KuntaiDu
    KuntaiDu committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    ac4d137 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2024

  1. remove wrongfully-added results

    @KuntaiDu
    KuntaiDu committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    11731f4 View commit details
    Browse the repository at this point in the history

  2. adjust plotting & provide more details in nightly description

    @KuntaiDu
    KuntaiDu committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    b012f71 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2024

  1. adjust figure -- add grid, bar plot, color, +throughput

    @KuntaiDu
    KuntaiDu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    4def302 View commit details
    Browse the repository at this point in the history

  2. typo fix

    @KuntaiDu
    KuntaiDu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    6b77d2b View commit details
    Browse the repository at this point in the history

  3. bug fix: set color using attribute

    @KuntaiDu
    KuntaiDu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    8c0259c View commit details
    Browse the repository at this point in the history

  4. mute curl output --- it's getting toooo long

    @KuntaiDu
    KuntaiDu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    2ee07df View commit details
    Browse the repository at this point in the history

  5. adjust coloring

    @KuntaiDu
    KuntaiDu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    9547066 View commit details
    Browse the repository at this point in the history

  6. increase font size, adjust coloring

    @KuntaiDu
    KuntaiDu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    a3085a1 View commit details
    Browse the repository at this point in the history

  7. adjust font size

    @KuntaiDu
    KuntaiDu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    0a554ae View commit details
    Browse the repository at this point in the history

  8. adjust spacing

    @KuntaiDu
    KuntaiDu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    c6c9292 View commit details
    Browse the repository at this point in the history

  9. increase font size

    @KuntaiDu
    KuntaiDu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    4788d27 View commit details
    Browse the repository at this point in the history

  10. increase cap size

    @KuntaiDu
    KuntaiDu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    ccc160c View commit details
    Browse the repository at this point in the history

  11. make yapf and ruff happy

    @KuntaiDu
    KuntaiDu committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    b6c5572 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. allow running performance benchmark & nightly benchmark simultaneously

    @KuntaiDu
    KuntaiDu committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    13d8c04 View commit details
    Browse the repository at this point in the history

  2. adjust the annotation context for nightly benchmark so that it does n… 

    …ot overlap with performance benchmark
    @KuntaiDu
    KuntaiDu committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    4d77e8f View commit details
    Browse the repository at this point in the history

  3. cut redundant lines in nightly-pipeline.yaml using yaml anchor

    @KuntaiDu
    KuntaiDu committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    da41c53 View commit details
    Browse the repository at this point in the history

  4. add dpi=400

    @KuntaiDu
    KuntaiDu committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    c108084 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. adjust pipeline upload order

    @KuntaiDu
    KuntaiDu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    57e6783 View commit details
    Browse the repository at this point in the history

  2. merge two pipelines using yq

    @KuntaiDu
    KuntaiDu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    b057b4b View commit details
    Browse the repository at this point in the history

  3. adjust merging logic

    @KuntaiDu
    KuntaiDu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    1053900 View commit details
    Browse the repository at this point in the history

  4. put blocking step as the first step

    @KuntaiDu
    KuntaiDu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    5ef7e8a View commit details
    Browse the repository at this point in the history

  5. this file has been moved to vllm-project/buildkite-ci. Remove it.

    @KuntaiDu
    KuntaiDu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    bbe115d View commit details
    Browse the repository at this point in the history

  6. add warning message

    @KuntaiDu
    KuntaiDu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    fb1e392 View commit details
    Browse the repository at this point in the history

  7. add a wait at the end, essential when merging multiple yaml files

    @KuntaiDu
    KuntaiDu committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    50ed6b7 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. adjust pipeline.yaml

    @KuntaiDu
    KuntaiDu committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    9758f94 View commit details
    Browse the repository at this point in the history

  2. adjust pipeline.yaml

    @KuntaiDu
    KuntaiDu committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    8608d17 View commit details
    Browse the repository at this point in the history

  3. fix pipeline yaml

    @KuntaiDu
    KuntaiDu committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    37c4c11 View commit details
    Browse the repository at this point in the history