-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat!: tracing with concurrency #8489
base: main
Are you sure you want to change the base?
feat!: tracing with concurrency #8489
Conversation
Can you please add a test that reproduces the issue you reported? Also there are some other tests failing, we should fix those too. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes sure. I edited one of the existing unit tests to check for this additional parameter but it is failing now. I am taking a closer look at it now.
I agree. This will surely cause code conflicts with the cycle PR. |
30d6d5b
to
a8d9007
Compare
Pull Request Test Coverage Report for Build 11495743757Details
💛 - Coveralls |
Related Issues
Proposed Changes:
Issue: I ran into some issues after deploying a pipeline as a service with tracing enabled. If there are concurrent calls to run the pipeline, the structure of spans kinda messed up due to the fact that the "haystack.component.run" spans are attached to the "haystack.pipeline.run" span. I experimented primarily with Langfuse, and this gave me a giant span with nested pipeline runs overlapping each other, which is definitely not ideal.
Purposed solution: I have come up with a solution by adding an optional
parent_span
parameter in the trace context. Furthermore, I tweaked the Langfuse tracer integration such that it creates a new span/generation on top of the parent span if it is provided, or create a root span if not.How did you test it?
Notes for the reviewer
Checklist
fix:
,feat:
,build:
,chore:
,ci:
,docs:
,style:
,refactor:
,perf:
,test:
.