You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
See this discussion: #358.
The problem is this:
You have streaming results
You want to still run a chain/state machine
You want to stream back intermediate results
For instance, say you have a voice call -- you want to move from one state to another. You need to stream it in on each step.
Or, say you have an agent that does some complicated stuff. You want to stream back intermediate results from each step to show how the agent is currently thinking. You'll want to stream, while iterating through...
Describe the solution you'd like
Sync
foraction, streaming_containerinapp.iterate_streams(
halt_after=[...],
halt_before=[...],
inputs={...}
):
do_something_with(streaming_container) # stream back in your fastAPI endpoint, say
Async
asyncforaction, streaming_containerinapp.iterate_streams(
halt_after=[...],
halt_before=[...],
inputs={...}
):
awaitdo_something_with(streaming_container) # stream back in your fastAPI endpoint, say
The nice thing is we should be able to call stream_results in a loop. The tough thing is it's a bit complex -- we could stream back all the individual results at once, but there's no guarentee the shape is the same.
Other things to think about:
should we stop at all results? Or just the streaming ones (and walk through the non-streaming ones...). Maybe have this be an option?
Should we automatically chain it? Or not? My guess is not.
Describe alternatives you've considered
See the possibility here: #358. It's all doable, but not particularly ergonomic. So the alternative is potentially to have this workaround. We should add this to the documentation too.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
See this discussion: #358.
The problem is this:
For instance, say you have a voice call -- you want to move from one state to another. You need to stream it in on each step.
Or, say you have an agent that does some complicated stuff. You want to stream back intermediate results from each step to show how the agent is currently thinking. You'll want to stream, while iterating through...
Describe the solution you'd like
Sync
Async
The nice thing is we should be able to call
stream_results
in a loop. The tough thing is it's a bit complex -- we could stream back all the individual results at once, but there's no guarentee the shape is the same.Other things to think about:
Describe alternatives you've considered
See the possibility here: #358. It's all doable, but not particularly ergonomic. So the alternative is potentially to have this workaround. We should add this to the documentation too.
The text was updated successfully, but these errors were encountered: