feat: enable streaming for VertexAIGeminiChatGenerator #1014
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
integration:google-vertex
type:documentation
Amnah199
requested review from
anakin87 and
julian-risch
and removed request for
a team and
anakin87
|
@julian-risch I've added you for review since this is similar to the PR #1012. |
Amnah199
commented
Aug 22, 2024
...s/google_vertex/src/haystack_integrations/components/generators/google_vertex/chat/gemini.py
Show resolved
Hide resolved
julian-risch
requested changes
Aug 27, 2024
There was a problem hiding this comment.
Looks very good to me already! We should just add type annotations in two functions. Otherwise it'S ready to be merged. And I am looking forward to tests as tracked by #1019
...s/google_vertex/src/haystack_integrations/components/generators/google_vertex/chat/gemini.py
Outdated
Show resolved
Hide resolved
...s/google_vertex/src/haystack_integrations/components/generators/google_vertex/chat/gemini.py
Outdated
Show resolved
Hide resolved
Amnah199
commented
Aug 27, 2024
| responses.append(streaming_chunk.content) | ||
|
|
||
| combined_response = "".join(responses).lstrip() | ||
| return [ChatMessage.from_system(content=combined_response)] |
There was a problem hiding this comment.
This will be changed to assistant role in another PR. I am tracking it here.
Amnah199
added a commit
that referenced
this pull request
Oct 2, 2024
* Add streaming_callback in init() and run()
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related Issues
Proposed Changes:
Add
streaming_callbackparam toVertexAIGeminiChatGeneratorto enable streaming.How did you test it?
Manually testing as this component has no written tests.
Notes for the reviewer
This feature can also be implemented without using
StreamingChunkbut I decided to use it for consistency with other implementations we have in Bedrock, OpenAI etc.Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:.