Feature Request: solution to display inter-service time

[ecourreges-orange]

Requirement - what kind of business use case are you trying to solve?

One of the promises of distributed tracing is to be able to locate network issues and delays easier.
However, most of the time, network equipments don't create spans.
So right now I don't have a good way of measuring the network time apart from opening the 2 spans and doing "Span Start Time 2" - "Span Start Time 1"

Proposals

Virtual Span option

What do you think of a functionality on the trace visualization that would allow toggling and adding the display of two "virtual spans" in case of inter-service spans?

• one representing the time spent between the start of the parent and the start of the child span (supposedly represents the request time spent on network + tcp/ TLS handshake)
• one representing the time spent between the end of the child and the end of the parent (represents the time of sending the response)

Display the duration of a selection

A cheaper feature would be displaying the duration of a time selection in the upper time view.

Recreate inter-service spans with Kafka

Another way to do it would be with a kafka post-processing that actually creates "real" spans instead.
This would allow easy analytics as well.

Conclusion

Maybe some of this is planned in the UI rework?
I am a backend developer so will have a harder time contributing to this subject but you never know... if we agree on a solution, I could find time to develop this.
Thank you for your time.

[lookfwd]

"gap between traces". I've been discussing this in the past. I don't think this is UI only - since you would potentially like to be able to search by this as well. It's also very prominent for asynchronous calls, where your time is spend after the end of the initiating span and before the end of the callback. In that case all the processing is done on the gaps between spans.

[jkowall]

Had to chime in here, the time between spans could be several things mentioned here from network latency, encryption, queuing or other intermediaries which may not add spans, or other logical constructs. Additionally it could be cliuent side time between requests which might need JavaScript in the browser to execute, rendering, or other time spent on the client. If anything this should be handled in the instrumentation layer and not in the UI layer since we do not know where that time is being spent.

By using things like BPF we could analyze and understand some of the sources of latency, which is now being added to OpenTelemetry: open-telemetry/community#733 there are also more documents which have been generated about this technology and Otel.

A discussion and fixed on the collection of SSL and encryption tracing: open-telemetry/opentelemetry-specification#1652

Here is another recent discussion on client instrumentation: open-telemetry/opentelemetry-java-instrumentation#4617 and open-telemetry/opentelemetry-js#456

Closing this out as it's an Otel requirement IMO.