Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ollama: support streaming #167

Closed
anakin87 opened this issue Jan 4, 2024 · 8 comments
Closed

Ollama: support streaming #167

anakin87 opened this issue Jan 4, 2024 · 8 comments
Assignees
Labels
feature request Ideas to improve an integration integration:ollama P3

Comments

@anakin87
Copy link
Member

anakin87 commented Jan 4, 2024

Ollama Generator should support streaming, as done in other Generators with streaming_callback: see for example OpenAIGenerator.

@anakin87 anakin87 added feature request Ideas to improve an integration integration:ollama labels Jan 4, 2024
@masci masci added the P3 label Jan 11, 2024
@sachinsachdeva
Copy link
Contributor

@anakin87 I can give this a go

@sachinsachdeva
Copy link
Contributor

Also seems like the official python client for ollama was published few days back ollama-python, I would suggest to check that out and use it in OllamaXXX components.

@sachinsachdeva
Copy link
Contributor

@AlistairLR112 ☝️

@AlistairLR112
Copy link
Contributor

Hey @sachinsachdeva , what do you think? I usually advise caution with developing against something that's tagged as experimental

@anakin87
Copy link
Member Author

In my opinion, in the future, the most appropriate choice might be to adopt the official python client.
For now, there is no hurry.

@sachinsachdeva
Copy link
Contributor

sachinsachdeva commented Jan 23, 2024

Hey @sachinsachdeva , what do you think? I usually advise caution with developing against something that's tagged as experimental

I think its match made in heaven, experimental library for Beta software 😈

@sachinsachdeva
Copy link
Contributor

Created #280

@vish9812
Copy link
Contributor

Please add this support to Chat generator as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Ideas to improve an integration integration:ollama P3
Projects
None yet
Development

No branches or pull requests

6 participants