Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update tgi with text-generation-inference:2.1.0 #273

Merged
merged 24 commits into from
Jul 9, 2024
Merged

update tgi with text-generation-inference:2.1.0 #273

merged 24 commits into from
Jul 9, 2024

Conversation

chensuyue
Copy link
Collaborator

@chensuyue chensuyue commented Jul 4, 2024

Description

update tgi with text-generation-inference:2.1.0

Issues

#230

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)

Dependencies

List the newly introduced 3rd party dependency if exists.

Tests

Describe the tests that you ran to verify your changes.

@chensuyue chensuyue requested a review from letonghan July 4, 2024 05:55
@chensuyue
Copy link
Collaborator Author

@letonghan the test with Phi-3 failed, please check.

@letonghan
Copy link
Collaborator

@letonghan the test with Phi-3 failed, please check.

It's mainly caused by the error below. The parameter --max-input-tokens should be smaller than the value of sliding_window=2047.
image

Signed-off-by: chensuyue <[email protected]>
@chensuyue chensuyue merged commit f236949 into main Jul 9, 2024
6 of 8 checks passed
@chensuyue chensuyue deleted the suyue/tgi branch July 9, 2024 14:28
yogeshmpandey pushed a commit to yogeshmpandey/GenAIComps that referenced this pull request Jul 10, 2024
dwhitena pushed a commit to predictionguard/GenAIComps that referenced this pull request Jul 24, 2024
lkk12014402 pushed a commit that referenced this pull request Aug 8, 2024
* remove useless schedule

Signed-off-by: Spycsh <[email protected]>

* remove blank line

Signed-off-by: Spycsh <[email protected]>

---------

Signed-off-by: Spycsh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants