Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade outlines to 0.1.1 #2658

Closed
wants to merge 6 commits into from
Closed

Conversation

aW3st
Copy link

@aW3st aW3st commented Oct 16, 2024

What does this PR do?

Upgrades Outlines package in the server to 0.1.1.

Outlines has released a number of fixes and improvements since the current version (0.0.34). Some highlights:
Compatibility with VLLMs
Support min/max number of digits for numbers in JSON Schema
Exclude escape character in JSON string fields (prevents invalid JSON generations)
Allow json ints to be negative

And more. This change should help everyone counting on reliable structured outputs.

Fixes # (issue)
Brings new improvements to regex parsing from Outlines

Before submitting

Who can review?

@Narsil I noticed you reviewed the last Outlines version bump, so tagging you for review.

@aW3st
Copy link
Author

aW3st commented Oct 24, 2024

@Narsil any thoughts on this? We're using TGI with guidance in production and encountering parsing errors. Upgrading Outlines should fix some of those errors.

If it makes the process easier I can open an issue as well.

@Narsil
Copy link
Collaborator

Narsil commented Oct 25, 2024

Thanks for this PR. Everything looks good, we upgraded other packets as well so the poetry.lock seems outdated now, could you rebase ?

I can take care of it if you want, I'll open a PR for the CI to run, and we'll see if this breaks anything.

@Narsil
Copy link
Collaborator

Narsil commented Oct 25, 2024

CI running here : ( I manually rebased) #2690

@Narsil
Copy link
Collaborator

Narsil commented Oct 25, 2024

'RegexGuide' object has no attribute 'allowed_token_ids'

And

FAILED integration-tests/models/test_grammar_response_format_llama.py::test_grammar_response_format_llama_json - KeyError: 'choices'

Full logs: https://github.com/huggingface/text-generation-inference/actions/runs/11512098640/job/32046723682?pr=2690

@aW3st
Copy link
Author

aW3st commented Oct 25, 2024

CI running here : ( I manually rebased) #2690
Thanks for this! looking into the bug.

@Narsil
Copy link
Collaborator

Narsil commented Oct 28, 2024

Closed in favor of : #2694

@Narsil Narsil closed this Oct 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants