batched streaming

[aniketmaurya]

Before submitting

• Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

What does this PR do?

Enables batched streaming #45

• Update tests

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[codecov]

Codecov Report

Merging #55 (fc4927f) into main (38b5b09) will decrease coverage by 1%.
The diff coverage is 69%.

Additional details and impacted files

@@        Coverage Diff         @@
##           main   #55   +/-   ##
==================================
- Coverage    83%   82%   -1%     
==================================
  Files         6     6           
  Lines       381   426   +45     
==================================
+ Hits        318   351   +33     
- Misses       63    75   +12     

[lantiga]

Looks great so far

[aniketmaurya]

@lantiga we would also need to document on how are we going to batch requests with different argument like temperature etc.

[lantiga]

Hey @aniketmaurya this is looking good! Let's remove the [wip] and get it merged

[aniketmaurya]

@lantiga please feel free to approve and merge.