feat(langchain): add support for streamed calls

[sabrenner]

What does this PR do?

Adds support for .(a)stream(...) calls on LangChain LCEL chains, chat models, and completion (LLM) models. It accomplishes this by:

1. starting the span when the stream function is called
2. as the stream is consumed, add the chunk to a list of chunks
3. when the stream is consumed, finish the span, and concatenate the results into an output string

There is one caveat to this, where it's possible the last step in a chain's stream call is a JSONOutputParser. In this case, the stream is already concatenated for us, and we use that result instead.

A few additional notes:

• The async versions, astream, aren't actually async functions, they just return async generators. This is reflected in shared patching functions and test snapshots.
• This whole process is accomplished through a shared_stream, which returns a compatible iterable for sync and async stream managers. It utilizes on_span_started and on_span_finished functions to coordinate tags to add in the different cases, such as chain, chat, or llm.
• Since the stream methods do not invoke the underlying generate methods we trace, there's not easy path for code re-use. Thus, I tried to mimic the tags we add for the relevant chain.invoke, model.generate, and llm.invoke patched functions.

Note: The version of vcrpy which we pinned for reduced flakiness did not like streamed calls when using the LangChain library. As such, I introduced a new fixture that returns a stub for the HTTP transport the OpenAI client uses. Then, the client with that transport specified can be used on the langchain_openai instance. This approach uses text files with just the response data, which is why the "cassettes" added for the tests written aren't the usual yaml format.

Limitations

There are a couple of limitations that need to be noted:

1. Because of upstream issues with langchain not handling indices returned from streamed chunks from certain partner libraries (primarily OpenAI), we cannot properly aggregate streamed multiple choices by index onto tags.
2. We are not providing direct tracing support for astream_events in this PR. Although it uses astream under the hood, and it is tested that astream_events does not conflict with the support added in this PR, official tracing support will be added in a follow-up PR
3. Additionally, submitting streamed responses to LLM Observability will be in a direct follow-up PR as well.

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

---

[github-actions]

CODEOWNERS have been resolved as:

ddtrace/contrib/internal/langchain/utils.py                             @DataDog/ml-observability
releasenotes/notes/langchain-lcel-stream-calls-bff85c974a72cceb.yaml    @DataDog/apm-python
tests/contrib/langchain/cassettes/langchain_community/lcel_openai_chat_streamed_response.txt  @DataDog/ml-observability
tests/contrib/langchain/cassettes/langchain_community/lcel_openai_chat_streamed_response_json_output_parser.txt  @DataDog/ml-observability
tests/contrib/langchain/cassettes/langchain_community/lcel_openai_llm_streamed_response.txt  @DataDog/ml-observability
tests/snapshots/tests.contrib.langchain.test_langchain_community.test_streamed_chain.json  @DataDog/apm-python
tests/snapshots/tests.contrib.langchain.test_langchain_community.test_streamed_chat.json  @DataDog/apm-python
tests/snapshots/tests.contrib.langchain.test_langchain_community.test_streamed_json_output_parser.json  @DataDog/apm-python
tests/snapshots/tests.contrib.langchain.test_langchain_community.test_streamed_llm.json  @DataDog/apm-python
ddtrace/contrib/internal/langchain/patch.py                             @DataDog/ml-observability
tests/contrib/langchain/conftest.py                                     @DataDog/ml-observability
tests/contrib/langchain/test_langchain_community.py                     @DataDog/ml-observability

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: sabrenner/langchain-stream
Commit report: a96fff2
Test service: dd-trace-py

✅ 0 Failed, 592 Passed, 604 Skipped, 18m 53.33s Total duration (17m 40.25s time saved)

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-09-26 14:50:24

Comparing candidate commit a96fff2 in PR branch sabrenner/langchain-stream with baseline commit 26d9c60 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 371 metrics, 53 unstable metrics.

[Yun-Kim]

Great work! I added a few comments about styling (avoiding nesting when possible for both readability and ironically line count), and some clarification questions but this is great. Awesome job on setting up a test fixture instead of being bogged down by vcrpy!

[Yun-Kim]

Looks good, mostly clarification questions! For easier tracking, can we add a limitations section to this PR description (i.e. n>1 response tagging, astream_events())? Thanks!

[Yun-Kim]

Two last questions, I'll approve to unblock you but please address them (I'll defer to your judgement). Thanks!