Skip to content

Commit

Permalink
ECS onboarding for HTTP attributes
Browse files Browse the repository at this point in the history
  • Loading branch information
lmolkova committed Apr 7, 2023
1 parent 26dbcc2 commit b3c6d06
Show file tree
Hide file tree
Showing 12 changed files with 184 additions and 124 deletions.
54 changes: 47 additions & 7 deletions semantic_conventions/http-common.yaml
Original file line number Diff line number Diff line change
@@ -1,15 +1,56 @@
groups:
- id: attributes.http.deprecated
type: attribute_group
brief: "Describes deprecated HTTP attributes."
prefix: http
attributes:
- id: method
type: string
brief: 'Deprecated, use `http.request.method` instead.'
stability: deprecated
examples: ["GET", "POST", "HEAD"]
- id: status_code
type: int
brief: 'Deprecated, use `http.response.status_code` instead.'
stability: deprecated
examples: [200]
- id: scheme
type: string
brief: 'Deprecated, use `url.scheme` instead.'
stability: deprecated
examples: ['http', 'https']
- id: url
type: string
brief: 'Deprecated, use `url.full` instead.'
stability: deprecated
examples: ['https://www.foo.bar/search?q=OpenTelemetry#SemConv']
- id: target
type: string
brief: 'Deprecated, use `url.path` and `url.query` instead.'
stability: deprecated
examples: ['/search?q=OpenTelemetry#SemConv']
- id: request_content_length
type: int
brief: 'Deprecated, use `http.request.body.bytes` instead.'
stability: deprecated
examples: 3495
- id: response_content_length
type: int
brief: 'Deprecated, use `http.response.body.bytes` instead.'
stability: deprecated
examples: 3495

- id: attributes.http.common
type: attribute_group
brief: "Describes HTTP attributes."
prefix: http
attributes:
- id: method
- id: request.method
type: string
requirement_level: required
brief: 'HTTP request method.'
examples: ["GET", "POST", "HEAD"]
- id: status_code
- id: response.status_code
type: int
requirement_level:
conditionally_required: If and only if one was received/sent.
Expand Down Expand Up @@ -47,17 +88,13 @@ groups:
note: >
When [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource) is absolute URI, `net.peer.name` MUST match
URI port identifier, otherwise it MUST match `Host` header port identifier.
- ref: url.full

- id: attributes.http.server
prefix: http
type: attribute_group
brief: 'HTTP Server spans attributes'
attributes:
- id: scheme
type: string
brief: 'The URI scheme identifying the used protocol.'
requirement_level: required
examples: ["http", "https"]
- id: route
type: string
requirement_level:
Expand Down Expand Up @@ -96,3 +133,6 @@ groups:
- Port identifier of the [request target](https://www.rfc-editor.org/rfc/rfc9110.html#target.resource)
if it's sent in absolute-form.
- Port identifier of the `Host` header
- ref: url.scheme
requirement_level: required
examples: ["http", "https"]
28 changes: 14 additions & 14 deletions semantic_conventions/metrics/http.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ groups:
extends: attributes.http.server
attributes:
# todo (lmolkova) build tools don't populate grandparent attributes
- ref: http.method
- ref: http.status_code
- ref: http.request.method
- ref: http.response.status_code
- ref: net.protocol.name
- ref: net.protocol.version

Expand All @@ -20,8 +20,8 @@ groups:
instrument: updowncounter
unit: "{request}"
attributes:
- ref: http.method
- ref: http.scheme
- ref: http.request.method
- ref: url.scheme
- ref: net.host.name
requirement_level: required
brief: >
Expand Down Expand Up @@ -60,8 +60,8 @@ groups:
# TODO (trask) below attributes are identical to above in metric.http.server.duration
attributes:
# todo (lmolkova) build tools don't populate grandparent attributes
- ref: http.method
- ref: http.status_code
- ref: http.request.method
- ref: http.response.status_code
- ref: net.protocol.name
- ref: net.protocol.version

Expand All @@ -74,8 +74,8 @@ groups:
extends: attributes.http.server
# TODO (trask) below attributes are identical to above in metric.http.server.duration
attributes:
- ref: http.method
- ref: http.status_code
- ref: http.request.method
- ref: http.response.status_code
- ref: net.protocol.name
- ref: net.protocol.version

Expand All @@ -87,8 +87,8 @@ groups:
unit: "ms"
extends: attributes.http.client
attributes:
- ref: http.method
- ref: http.status_code
- ref: http.request.method
- ref: http.response.status_code
- ref: net.protocol.name
- ref: net.protocol.version
- ref: net.sock.peer.addr
Expand All @@ -102,8 +102,8 @@ groups:
extends: attributes.http.client
# TODO (trask) below attributes are identical to above in metric.http.client.duration
attributes:
- ref: http.method
- ref: http.status_code
- ref: http.request.method
- ref: http.response.status_code
- ref: net.protocol.name
- ref: net.protocol.version
- ref: net.sock.peer.addr
Expand All @@ -117,8 +117,8 @@ groups:
extends: attributes.http.client
# TODO (trask) below attributes are identical to above in metric.http.client.duration
attributes:
- ref: http.method
- ref: http.status_code
- ref: http.request.method
- ref: http.response.status_code
- ref: net.protocol.name
- ref: net.protocol.version
- ref: net.sock.peer.addr
27 changes: 6 additions & 21 deletions semantic_conventions/trace/http.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,21 +8,21 @@ groups:
These conventions can be used for http and https schemes
and various HTTP versions like 1.1, 2 and SPDY.
attributes:
- id: request_content_length
- id: request.body.bytes
type: int
brief: >
The size of the request payload body in bytes. This is the number of bytes transferred excluding headers and
is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length)
header. For requests using transport encoding, this should be the compressed size.
examples: 3495
- id: response_content_length
- id: response.body.bytes
type: int
brief: >
The size of the response payload body in bytes. This is the number of bytes transferred excluding headers and
is often, but not always, present as the [Content-Length](https://www.rfc-editor.org/rfc/rfc9110.html#field.content-length)
header. For requests using transport encoding, this should be the compressed size.
examples: 3495
- ref: http.method
- ref: http.request.method
sampling_relevant: true
- ref: net.sock.peer.addr
- ref: net.sock.peer.port
Expand All @@ -38,17 +38,6 @@ groups:
span_kind: client
brief: 'Semantic Convention for HTTP Client'
attributes:
- id: url
type: string
requirement_level: required
brief: >
Full HTTP request URL in the form `scheme://host[:port]/path?query[#fragment]`.
Usually the fragment is not transmitted over HTTP, but if it is known, it should be included nevertheless.
note: >
`http.url` MUST NOT contain credentials passed via URL in form of `https://username:[email protected]/`.
In such case the attribute's value should be `https://www.example.com/`.
sampling_relevant: true
examples: ['https://www.foo.bar/search?q=OpenTelemetry#SemConv']
- id: resend_count
type: int
brief: >
Expand Down Expand Up @@ -91,12 +80,6 @@ groups:
span_kind: server
brief: 'Semantic Convention for HTTP Server'
attributes:
- id: target
type: string
brief: 'The full request target as passed in a HTTP request line or equivalent.'
requirement_level: required
sampling_relevant: true
examples: ['/users/12314/?q=ddds']
- id: client_ip
type: string
brief: >
Expand All @@ -115,7 +98,7 @@ groups:
one is at least somewhat confident that the address is not that of
the closest proxy.
examples: '83.164.160.102'
- ref: http.scheme
- ref: http.request.method
sampling_relevant: true
- ref: net.host.name
requirement_level: required
Expand Down Expand Up @@ -149,3 +132,5 @@ groups:
- ref: net.sock.host.addr
requirement_level: opt_in
- ref: net.sock.host.port
- ref: url.path
- ref: url.query
27 changes: 27 additions & 0 deletions semantic_conventions/url.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
groups:
- id: url
brief: "Describes URL."
type: attribute_group
prefix: url
attributes:
- id: scheme
type: string
brief: 'The URI scheme identifying the used protocol.'
examples: ["http", "https"]
- id: full
type: string
brief: >
Full HTTP request URL in the form `scheme://host[:port]/path?query[#fragment]`.
Usually the fragment is not transmitted over HTTP, but if it is known, it should be included nevertheless.
note: >
`url.full` MUST NOT contain credentials passed via URL in form of `https://username:[email protected]/`.
In such case the attribute's value should be `https://www.example.com/`.
examples: ['https://www.foo.bar/search?q=OpenTelemetry#SemConv']
- id: path
type: string
brief: 'The URI path'
examples: ['/search']
- id: query
type: string
brief: 'The URI query'
examples: ["?q=OpenTelemetry#SemConv"]
6 changes: 3 additions & 3 deletions specification/common/attribute-naming.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Names SHOULD follow these rules:
purpose should primarily drive the decision about forming nested namespaces.

- For each multi-word dot-delimited component of the attribute name separate the
words by underscores (i.e. use snake_case). For example `http.status_code`
words by underscores (i.e. use snake_case). For example `http.response.status_code`
denotes the status code in the http namespace.

- Names SHOULD NOT coincide with namespaces. For example if
Expand Down Expand Up @@ -96,8 +96,8 @@ denote old attribute names in rename operations).
- Semantic conventions exist for four areas: for Resource, Span, Log, and Metric
attribute names. In addition, for spans we have two more areas: Event and Link
attribute names. Identical namespaces or names in all these areas MUST have
identical meanings. For example the `http.method` span attribute name denotes
exactly the same concept as the `http.method` metric attribute, has the same
identical meanings. For example the `http.request.method` span attribute name denotes
exactly the same concept as the `http.request.method` metric attribute, has the same
data type and the same set of possible values (in both cases it records the
value of the HTTP protocol's request method as a string).

Expand Down
4 changes: 2 additions & 2 deletions specification/common/attribute-requirement-level.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ For example, [Database semantic convention](../trace/semantic_conventions/databa

## Required

All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, and can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.method` is an example of a Required attribute.
All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, and can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.request.method` is an example of a Required attribute.

_Note: Consumers of telemetry can detect if a telemetry item follows a specific semantic convention by checking for the presence of a `Required` attribute defined by such convention. For example, the presence of the `db.system` attribute on a span can be used as an indication that the span follows database semantics._

Expand Down Expand Up @@ -71,4 +71,4 @@ Here are several examples of expensive operations to be avoided by default:

- DNS lookups to populate `net.peer.name` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too.
- forcing an `http.route` calculation before the HTTP framework calculates it
- reading response stream to find `http.response_content_length` when `Content-Length` header is not available
- reading response stream to find `http.response.body.bytes` when `Content-Length` header is not available
2 changes: 1 addition & 1 deletion specification/compatibility/prometheus_and_openmetrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ attributes, and MUST NOT be added as metric attributes:
| ----------------------- | ----------- |
| `net.host.name` | The `<host>` portion of the target's URL that was scraped |
| `net.host.port` | The `<port>` portion of the target's URL that was scraped |
| `http.scheme` | `http` or `https` |
| `url.scheme` | `http` or `https` |

In addition to the attributes above, the
[target_info](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems)
Expand Down
6 changes: 3 additions & 3 deletions specification/logs/data-model-appendix.md
Original file line number Diff line number Diff line change
Expand Up @@ -409,19 +409,19 @@ When mapping from the unified model to HEC, we apply this additional mapping:
<td>%m</td>
<td>string</td>
<td>The request method.</td>
<td>Attributes["http.method"]</td>
<td>Attributes["http.request.method"]</td>
</tr>
<tr>
<td>%v,%p,%U,%q</td>
<td>string</td>
<td>Multiple fields that can be composed into URL.</td>
<td>Attributes["http.url"]</td>
<td>Attributes["url.full"]</td>
</tr>
<tr>
<td>%>s</td>
<td>string</td>
<td>Response status.</td>
<td>Attributes["http.status_code"]</td>
<td>Attributes["http.response.status_code"]</td>
</tr>
<tr>
<td>All other fields</td>
Expand Down
4 changes: 2 additions & 2 deletions specification/metrics/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -728,14 +728,14 @@ API](../overview.md#api) authors might consider:
```python
# Python

http_server_duration.Record(50, {"http.method": "POST", "http.scheme": "https"})
http_server_duration.Record(50, {"http.request.method": "POST", "url.scheme": "https"})
http_server_duration.Record(100, http_method="GET", http_scheme="http")
```

```csharp
// C#
httpServerDuration.Record(50, ("http.method", "POST"), ("http.scheme", "https"));
httpServerDuration.Record(50, ("http.request.method", "POST"), ("url.scheme", "https"));
httpServerDuration.Record(100, new HttpRequestAttributes { method = "GET", scheme = "http" });
```

Expand Down
Loading

0 comments on commit b3c6d06

Please sign in to comment.