Add capture of http.route to DjangoInstrumentor middleware #1226

stschenk · 2020-10-08T22:30:07Z

Description DjangoInstrumentor middleware so that it captures the resolved function path for the route as the span attribute http.route. This will allow the function path to be shown used as an organizing operation name in AppInsights.

Extends

Fixes # (issue)

Type of change

Please delete options that are not relevant.

[ X ] Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

Extended instrumentation/opentelemetry-instrumentation-django/tests/test_middleware.py to cover this change

[ X ] Test instrumentation/opentelemetry-instrumentation-django/tests/test_middleware.py

Checklist:

[ x ] Followed the style guidelines of this project
[ x ] Changelogs have been updated
[ x ] Unit tests have been added
Documentation has been updated

linux-foundation-easycla · 2020-10-08T22:30:11Z

The committers are authorized under a signed CLA.

✅ stschenk (12da4f6, 7362f73)

…ion path

owais · 2020-10-09T00:17:13Z

.../opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/middleware.py

@@ -151,6 +169,7 @@ def process_response(self, request, response):
            self._environ_activation_key in request.META.keys()
            and self._environ_span_key in request.META.keys()
        ):
+            self._set_http_route(request.META[self._environ_span_key], request)


Would doing this in process_request be enough and take care of both success/error cases?

I have only been able to find this route as being available from request.resolver_match.route and request.resolver_match is none during process_request() but become available during process_response and process_exception. So I think this value cannot be set just in process_request().

owais · 2020-10-09T00:20:08Z

.../opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/middleware.py

+            else:
+                # A function-based view
+                func_path = func.__module__ + "." + func.__name__
+            span.set_attribute("http.route", func_path)


http.route is supposed to be a route (usually templated or named) from an HTTP router. In django's case, that would be the string representation of a URL from urls.py. Looks like we are using the Python function/handler name. I'm not sure how useful this tag is but irrespective of whether we add it or not, it shouldn't be named http.route as it means something else entirely unless I'm mistaken.
Spec: https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/trace/semantic_conventions/http.md#http-server-semantic-conventions

What we want is the http.route to be in the span attribute. But you are correct, the value should be populated from the request path (which will be a string from urls.py). I believe it should be within the request parameter.

See Opencensus

@owais / @lzchen Thank you for your comments.

I chose to use the class/function name that the request is being routed to so that a developer would be able to go directly from the http.route as reported in AppInsights( and other tools) to the code. But I do see that this is not in accordance with the spec so I will make the change so that http.route is set to the "matched route (path template)".

However, I do not agree that the "matched route (path template)" is the request.path as opencensus uses. This is because the request.path is the actual path of the request, not the route or path template. One path template, you could see multiple paths because the parameters in the path will vary. For instance, the following paths from request.path:
/api/1/financial-institutions/111/plaid_details/
/api/1/financial-institutions/444/plaid_details/
/api/1/financial-institutions/554/plaid_details/
would all map to one route:
^api/1/financial-institutions/(?P[^/.]+)/plaid_details/$'

I have only been able to find this route as being available from request.resolver_match.route and request.resolver_match is none during process_request() but become available during process_response and process_exception. So I think this value cannot be set just in process_request().

I will go ahead and make the change to setting http.route to request.resolver_match.route and push up the change.

Please let me know what you think. Thanks!

However, I do not agree that the "matched route (path template)" is the request.path as opencensus uses.

If you are trying to add this to enable your AppInsights scenario, I suggest doing it the same way OpenCensus is doing it. OpenCensus Python SDK is Microsoft's current solution for Python SDK to send telemetry to App Insights and so being consistent with it is important for back compat and to not break users. If you want to propose a new design, I think submitting a separate feature request (after merging this PR) would be good.

@lzchen - I have change the code so that it uses request.path for the http.route value. I think this make the middleware equivalent to Opencensus.

As you suggested, I will submit a separate feature request around using the route template instead of the url path

I think we should use templated path/route for this and stick with OpenTelemetry spec. If we decide to have same behavior of OC, we'll have same information duplicated in http.target and http.route attributes. I don't think (may be I'm wrong) the breakage here is big enough to warrant going against Otel semantic conventions. OpenCensus users can still find their old value for http.route in OpenTelemetry's http.target so no data is lost after the migration, only renamed. WDYT?

@owais
I thought request.path represented the templated path/route, but if it isn't then that is my misunderstanding. But yes, I agree with what you are saying.

Glad we are arriving at a consensus. I have switched the code back to using request.resolver_match.route which is the templated route. I also realized from looking at the OpenCensus implementation that though resolver_match is not set for process_request, it is available in process_view so I added that method.

Also http.path was being set in the OpenCensus implementation by not in the OpenTelemetry implementation so I also added that.

instrumentation/opentelemetry-instrumentation-django/CHANGELOG.md

lzchen · 2020-10-09T15:01:18Z

Address #1213

Co-authored-by: Leighton Chen <[email protected]>

owais · 2020-10-13T17:14:13Z

.../opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/middleware.py

+            span = request.META[self._environ_span_key]
+
+            if span.is_recording():
+                if getattr(request, "resolver_match") and getattr(


nit: hasattr might be better for this if we aren't using the return val directly.

I used getattr because I also don't want to capture the value if it is None.

match = getattr(request, "resolved_match") if match: route = getattr(match, "route") if route: set_attribute()

Perhaps use this? I know it's usually not a big deal but we never know where the library will be used and better to reduce attribute look ups if we can in case the code ends up on a very hot path.

makes sense... I have not thought about reducing attribute lookups before. Thanks!

owais · 2020-10-13T17:15:15Z

instrumentation/opentelemetry-instrumentation-django/tests/test_middleware.py

@@ -121,6 +122,7 @@ def test_traced_post(self):
        self.assertEqual(
            span.attributes["http.url"], "http://testserver/traced/"
        )
+        self.assertEqual(span.attributes["http.route"], "^traced/")


would be nice to add a templated route so tests cover that and catch any regressions in future.

owais · 2020-10-13T22:26:08Z

.../opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/middleware.py

@@ -117,6 +117,7 @@ def process_request(self, request):
            )
            for key, value in attributes.items():
                span.set_attribute(key, value)
+            span.set_attribute("http.path", request.path)


This should be http.target according to the spec. https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/trace/semantic_conventions/http.md#common-attributes

Sound good.

Isn't http.target already being populated in collect_request_attributes?

Good point @lzchen! I will remove the setting of the attribute from the django middleware.

owais

One last comment about using http.target instead of http.path

…//github.com/stschenk/opentelemetry-python into add_http.route_capture_to_DjangoInstrumentor

stschenk added 2 commits October 8, 2020 15:15

Add capture of http.route to DjangoInstrumentor middleware

12da4f6

Update changelog

7362f73

stschenk requested a review from a team October 8, 2020 22:30

stschenk added 2 commits October 8, 2020 15:37

Fix lint error

e3979de

Switch from using hidden _func_path property to calculating the funct…

f070259

…ion path

owais requested changes Oct 9, 2020

View reviewed changes

lzchen reviewed Oct 9, 2020

View reviewed changes

instrumentation/opentelemetry-instrumentation-django/CHANGELOG.md Outdated Show resolved Hide resolved

stschenk and others added 4 commits October 12, 2020 08:58

Update instrumentation/opentelemetry-instrumentation-django/CHANGELOG.md

9190a7d

Co-authored-by: Leighton Chen <[email protected]>

Switch to using request.resolver_match.route for http.route

5f7927b

Switch to using request.path for http.route

d919de2

capture request.resolver_match.route as http.route

3bd216d

owais reviewed Oct 13, 2020

View reviewed changes

add test for templated route

a5889bb

owais reviewed Oct 13, 2020

View reviewed changes

owais requested changes Oct 13, 2020

View reviewed changes

stschenk added 2 commits October 13, 2020 16:06

switch http.path to http.target

aa4e35b

fix lint issue

afadb66

owais approved these changes Oct 13, 2020

View reviewed changes

owais and others added 3 commits October 14, 2020 19:05

Merge branch 'master' into add_http.route_capture_to_DjangoInstrumentor

ad9dfa5

remove setting of http.target, it is handled elsewhere

7348240

Merge branch 'add_http.route_capture_to_DjangoInstrumentor' of https:…

9170599

…//github.com/stschenk/opentelemetry-python into add_http.route_capture_to_DjangoInstrumentor

lzchen approved these changes Oct 14, 2020

View reviewed changes

lzchen and others added 3 commits October 14, 2020 12:14

Merge branch 'master' into add_http.route_capture_to_DjangoInstrumentor

b65e384

Merge branch 'master' into add_http.route_capture_to_DjangoInstrumentor

3f93411

Merge branch 'master' into add_http.route_capture_to_DjangoInstrumentor

6a5c07f

lzchen merged commit b54540e into open-telemetry:master Oct 15, 2020

stschenk deleted the add_http.route_capture_to_DjangoInstrumentor branch October 15, 2020 20:23

srikanthccv pushed a commit to srikanthccv/opentelemetry-python that referenced this pull request Nov 1, 2020

chore(renovate): remove rebaseWhen config (open-telemetry#1226)

b7d08c7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add capture of http.route to DjangoInstrumentor middleware #1226

Add capture of http.route to DjangoInstrumentor middleware #1226

stschenk commented Oct 8, 2020

linux-foundation-easycla bot commented Oct 8, 2020 •

edited

Loading

owais Oct 9, 2020

stschenk Oct 12, 2020

owais Oct 9, 2020

lzchen Oct 9, 2020

lzchen Oct 9, 2020 •

edited

Loading

stschenk Oct 12, 2020 •

edited

Loading

lzchen Oct 12, 2020

stschenk Oct 12, 2020 •

edited

Loading

owais Oct 12, 2020

lzchen Oct 13, 2020

stschenk Oct 13, 2020

lzchen commented Oct 9, 2020 •

edited

Loading

owais Oct 13, 2020

stschenk Oct 13, 2020

owais Oct 13, 2020 •

edited

Loading

stschenk Oct 13, 2020

owais Oct 13, 2020

owais Oct 13, 2020

stschenk Oct 13, 2020

lzchen Oct 14, 2020

stschenk Oct 14, 2020

owais left a comment

Add capture of http.route to DjangoInstrumentor middleware #1226

Add capture of http.route to DjangoInstrumentor middleware #1226

Conversation

stschenk commented Oct 8, 2020

Description DjangoInstrumentor middleware so that it captures the resolved function path for the route as the span attribute http.route. This will allow the function path to be shown used as an organizing operation name in AppInsights.

Type of change

How Has This Been Tested?

Checklist:

linux-foundation-easycla bot commented Oct 8, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lzchen Oct 9, 2020 • edited Loading

Choose a reason for hiding this comment

stschenk Oct 12, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stschenk Oct 12, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lzchen commented Oct 9, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

owais Oct 13, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

owais left a comment

Choose a reason for hiding this comment

linux-foundation-easycla bot commented Oct 8, 2020 •

edited

Loading

lzchen Oct 9, 2020 •

edited

Loading

stschenk Oct 12, 2020 •

edited

Loading

stschenk Oct 12, 2020 •

edited

Loading

lzchen commented Oct 9, 2020 •

edited

Loading

owais Oct 13, 2020 •

edited

Loading