Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modify httpclient instrumentation to take advantage of sampling decision #1894

Merged
merged 3 commits into from
Mar 10, 2021
Merged

Modify httpclient instrumentation to take advantage of sampling decision #1894

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1c68f30
Modify httpclient instrumentation to take advantage of sampling decision
cijothomas Mar 10, 2021
53659c0
Merge branch 'main' into cijothomas/improveinstr1
cijothomas Mar 10, 2021
01df421
changelog
cijothomas Mar 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/OpenTelemetry.Instrumentation.Http/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Unreleased

* HttpClient (.NET Core) instrumentation performance optimization
by leveraging sampling decision and short circuiting path.
([#1894](https://github.com/open-telemetry/opentelemetry-dotnet/pull/1894))

## 1.0.0-rc2

Released 2021-Jan-29
Expand Down
50 changes: 34 additions & 16 deletions src/OpenTelemetry.Instrumentation.Http/Implementation/HttpHandlerDiagnosticListener.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,49 +67,67 @@ public HttpHandlerDiagnosticListener(HttpClientInstrumentationOptions options)

public override void OnStartActivity(Activity activity, object payload)
{
// The overall flow of what HttpClient library does is as below:
// Activity.Start()
// DiagnosticSource.WriteEvent("Start", payload)
// DiagnosticSource.WriteEvent("Stop", payload)
// Activity.Stop()

// This method is in the WriteEvent("Start", payload) path.
// By this time, samplers have already run and
// activity.IsAllDataRequested populated accordingly.

if (Sdk.SuppressInstrumentation)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this check is done to make sure to have no side-effect if under supression. Even without this, TracerProviderSdk drops the activity if under supressions, but this method does propagation. This check prevents every side effect including propagation if under suppression.

{
return;
}

if (!this.startRequestFetcher.TryFetch(payload, out HttpRequestMessage request) || request == null)
{
HttpInstrumentationEventSource.Log.NullPayload(nameof(HttpHandlerDiagnosticListener), nameof(this.OnStartActivity));
return;
}

// TODO: Investigate why this check is needed.
Copy link
Contributor

@utpilla utpilla Mar 10, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it was put for redirection cases where a redirect request might already have the context in the headers.

But I don't think the listener callback gets called for redirect cases. I tried this scenario: Call http://google.com from an HttpClient. This results in a 301 redirection to http:www.//google.com. The way HttpHandlerDiagnostics is currently configured, we only see one OnStartActivity and one OnStopActivity method call.

In the OnStartActivity method they request payload has the original Uri: http://google.com whereas in the OnStopActivity method the payload has the redirected Uri: http:www.//google.com. This would mean that we only enter OnStartActivity once for the original HttpRequestOut call. So this logic would not be used for redirects.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know there were cases when I was working on HttpWebRequest instrumentation where it would re-use the request, but it has been a while so my memory is foggy. I think SSL upgrade/downgrade was one of them? Pretty sure I added a test or two for it. That is for the .NET Framework stuff but posting here because there are probably many similarities with the .NET Core side.

if (Propagators.DefaultTextMapPropagator.Extract(default, request, HttpRequestMessageContextPropagation.HeaderValuesGetter) != default)
{
// this request is already instrumented, we should back off
activity.IsAllDataRequested = false;
return;
}

// Propagate context irrespective of sampling decision
var textMapPropagator = Propagators.DefaultTextMapPropagator;

if (!(this.httpClientSupportsW3C && textMapPropagator is TraceContextPropagator))
{
textMapPropagator.Inject(new PropagationContext(activity.Context, Baggage.Current), request, HttpRequestMessageContextPropagation.HeaderValueSetter);
}

try
// enrich Activity from payload only if sampling decision
// is favorable.
if (activity.IsAllDataRequested)
{
if (this.options.EventFilter(activity.OperationName, request) == false)
try
{
HttpInstrumentationEventSource.Log.RequestIsFilteredOut(activity.OperationName);
if (this.options.EventFilter(activity.OperationName, request) == false)
{
HttpInstrumentationEventSource.Log.RequestIsFilteredOut(activity.OperationName);
activity.IsAllDataRequested = false;
return;
}
}
catch (Exception ex)
{
HttpInstrumentationEventSource.Log.RequestFilterException(ex);
activity.IsAllDataRequested = false;
return;
}
}
catch (Exception ex)
{
HttpInstrumentationEventSource.Log.RequestFilterException(ex);
activity.IsAllDataRequested = false;
return;
}

activity.DisplayName = HttpTagHelper.GetOperationNameForHttpMethod(request.Method);
activity.DisplayName = HttpTagHelper.GetOperationNameForHttpMethod(request.Method);

ActivityInstrumentationHelper.SetActivitySourceProperty(activity, ActivitySource);
ActivityInstrumentationHelper.SetKindProperty(activity, ActivityKind.Client);
ActivityInstrumentationHelper.SetActivitySourceProperty(activity, ActivitySource);
ActivityInstrumentationHelper.SetKindProperty(activity, ActivityKind.Client);

if (activity.IsAllDataRequested)
{
activity.SetTag(SemanticConventions.AttributeHttpMethod, HttpTagHelper.GetNameForHttpMethod(request.Method));
activity.SetTag(SemanticConventions.AttributeHttpHost, HttpTagHelper.GetHostTagValueFromRequestUri(request.RequestUri));
activity.SetTag(SemanticConventions.AttributeHttpUrl, request.RequestUri.OriginalString);
Expand Down
1 change: 1 addition & 0 deletions test/OpenTelemetry.Instrumentation.Http.Tests/HttpClientTests.Basic.netcore31.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,7 @@ public async Task HttpClientInstrumentation_AddViaFactory_HttpInstrumentation_Co
[Fact]
public async Task HttpClientInstrumentationBacksOffIfAlreadyInstrumented()
{
// TODO: Investigate why this feature is required.
var processor = new Mock<BaseProcessor<Activity>>();

var request = new HttpRequestMessage
Expand Down