From 16e22bf9daf3d063b041da3f8823618dbf2be5d3 Mon Sep 17 00:00:00 2001 From: Joao Grassi Date: Thu, 25 Aug 2022 13:06:01 +0200 Subject: [PATCH 1/6] Make context propagation requirements explict Co-authored-by: Georg Pirklbauer --- CHANGELOG.md | 2 + .../trace/semantic_conventions/messaging.md | 41 +++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index b539f9ab2c9..989affdc118 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,8 @@ release. ([#2675](https://github.com/open-telemetry/opentelemetry-specification/pull/2675)) - Add OpenSearch to db.system semantic conventions ([#2718](https://github.com/open-telemetry/opentelemetry-specification/pull/2718)). +- Make context propagation requirements explict for messaging semantic conventions + ([#xyz](https://github.com/open-telemetry/opentelemetry-specification/pull/xyz)). ### Compatibility diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 831165d9859..d3e79fb8e83 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -12,6 +12,7 @@ * [Conversations](#conversations) * [Temporary destinations](#temporary-destinations) - [Conventions](#conventions) + * [Context propagation](#context-propagation) * [Span name](#span-name) * [Span kind](#span-kind) * [Operation names](#operation-names) @@ -80,6 +81,46 @@ Often such destinations are unnamed or have an auto-generated name. Given these definitions, the remainder of this section describes the semantic conventions for Spans describing interactions with messaging systems. +### Context propagation + +A message may traverse many different components and layers in one or more intermediaries +when it is propagated from the producer to the consumer(s). To be able to correlate +consumer traces with producer traces using the existing context propagation mechanisms, +all components must propagate context down the chain. + +Due to the complex nature of messaging systems, it cannot be assumed +that all components are instrumented and propagate context accordingly. +For example, an application using a messaging system offered via +software as a service (SaaS) that does not propagate context will lead to incomplete traces. + +To be able to correlate consumer traces with producer traces without requiring +intermediary instrumentation, the context needs to be propagated on a +*per-message* basis instead of on a *per-request* basis. This allows all components +to have access to the same per-message context information, making it possible +to correlate all stages involved in processing a message with the message's +creation. + +A message *creation context* allows correlating producers with consumers +of a message and model the dependencies between them, +regardless of the underlying messaging transport mechanism and its instrumentation. + +The message creation context is created by the producer and should be propagated +to the consumer(s). + +Consumer traces cannot be directly correlated to producer traces if the message +creation context is not attached and propagated with the message. + +A producer SHOULD attach a message creation context to each message. +The message creation context SHOULD be attached in such a way that it is +not possible to be changed by intermediaries. + +> This document does not specify the exact mechanisms on how the creation context +> is attached/extracted to/from messages. Future versions of these conventions +> will give clear recommendations, following industry standards including, but not limited to +> [Trace Context: AMQP protocol](https://w3c.github.io/trace-context-amqp/) and +> [Trace Context: MQTT protocol](https://w3c.github.io/trace-context-mqtt/) +> once those standards reach a stable state. + ### Span name The span name SHOULD be set to the message destination name and the operation being performed in the following format: From 9da36c2427975720eefcb80cb3a7d4eec52eedd7 Mon Sep 17 00:00:00 2001 From: Joao Grassi Date: Thu, 25 Aug 2022 13:42:29 +0200 Subject: [PATCH 2/6] Update changelog entry --- CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 989affdc118..3271d5d5612 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,8 +70,8 @@ release. ([#2675](https://github.com/open-telemetry/opentelemetry-specification/pull/2675)) - Add OpenSearch to db.system semantic conventions ([#2718](https://github.com/open-telemetry/opentelemetry-specification/pull/2718)). -- Make context propagation requirements explict for messaging semantic conventions - ([#xyz](https://github.com/open-telemetry/opentelemetry-specification/pull/xyz)). +- Make context propagation requirements explicit for messaging semantic conventions + ([#2750](https://github.com/open-telemetry/opentelemetry-specification/pull/2750)). ### Compatibility From bca1e39ffa00a662d52dc4b9388be5900567c183 Mon Sep 17 00:00:00 2001 From: Joao Grassi Date: Fri, 23 Sep 2022 14:47:11 +0200 Subject: [PATCH 3/6] PR suggestions --- .../trace/semantic_conventions/messaging.md | 49 ++++++++++++------- 1 file changed, 32 insertions(+), 17 deletions(-) diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index d3e79fb8e83..3f74d95758f 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -31,6 +31,8 @@ ## Definitions +### Message + Although messaging systems are not as standardized as, e.g., HTTP, it is assumed that the following definitions are applicable to most of them that have similar concepts at all (names borrowed mostly from JMS): A *message* is an envelope with a potentially empty payload. @@ -44,6 +46,27 @@ With Apache Kafka, the physical broker a message is written to depends on the nu Messages can be delivered to 0, 1, or multiple consumers depending on the dispatching semantic of the protocol. +### Producer + +The "producer" is a specific instance, process or device that creates and +publishes a message. "Publishing" is the process of sending a message or batch +to the intermediary or consumer. + +### Consumer + +A "consumer" receives the message and acts upon it. It uses the context and +data to execute some logic, which might lead to the occurrence of new events. + +The consumer receives, processes, and settles a message. "Receiving" is the +process of obtaining a message from the intermediary, "processing" is the +process of acting on the information a message contains, "settling" is the +process of notifying an intermediary that a message was processed successfully. + +### Intermediary + +An "intermediary" receives a message to forward it to the next receiver, which +might be another intermediary or a consumer. + ### Destinations A destination is usually identified by some name unique within the messaging system instance, which might look like a URL or a simple one-word identifier. @@ -88,31 +111,23 @@ when it is propagated from the producer to the consumer(s). To be able to correl consumer traces with producer traces using the existing context propagation mechanisms, all components must propagate context down the chain. -Due to the complex nature of messaging systems, it cannot be assumed -that all components are instrumented and propagate context accordingly. -For example, an application using a messaging system offered via -software as a service (SaaS) that does not propagate context will lead to incomplete traces. - -To be able to correlate consumer traces with producer traces without requiring -intermediary instrumentation, the context needs to be propagated on a -*per-message* basis instead of on a *per-request* basis. This allows all components -to have access to the same per-message context information, making it possible -to correlate all stages involved in processing a message with the message's -creation. +Messaging systems themselves may trace messages as the messages travels from +producers to consumers. Such tracing would cover the transport layer but would +not help in correlating producers with consumers. To be able to directly +correlate producers with consumers, another context that is propagated with +the message is required. A message *creation context* allows correlating producers with consumers of a message and model the dependencies between them, regardless of the underlying messaging transport mechanism and its instrumentation. The message creation context is created by the producer and should be propagated -to the consumer(s). - -Consumer traces cannot be directly correlated to producer traces if the message -creation context is not attached and propagated with the message. +to the consumer(s). Consumer traces cannot be directly correlated with producer +traces if the message creation context is not attached and propagated with the message. A producer SHOULD attach a message creation context to each message. -The message creation context SHOULD be attached in such a way that it is -not possible to be changed by intermediaries. +If possible, the message creation context SHOULD be attached +in such a way that it cannot be changed by intermediaries. > This document does not specify the exact mechanisms on how the creation context > is attached/extracted to/from messages. Future versions of these conventions From 67eb951ff028341b4a964e1ba03e3f1aa354b950 Mon Sep 17 00:00:00 2001 From: Joao Grassi Date: Fri, 23 Sep 2022 14:54:03 +0200 Subject: [PATCH 4/6] Fix TOC --- specification/trace/semantic_conventions/messaging.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 3f74d95758f..f38d19fb61b 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -7,6 +7,10 @@ - [Definitions](#definitions) + * [Message](#message) + * [Producer](#producer) + * [Consumer](#consumer) + * [Intermediary](#intermediary) * [Destinations](#destinations) * [Message consumption](#message-consumption) * [Conversations](#conversations) From b97911d45553dc8254fcea8cc5c1880efbfb7e65 Mon Sep 17 00:00:00 2001 From: Joao Grassi Date: Mon, 26 Sep 2022 10:42:03 +0200 Subject: [PATCH 5/6] PR suggestions --- specification/trace/semantic_conventions/messaging.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index f38d19fb61b..0c7d8eec90f 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -54,7 +54,7 @@ Messages can be delivered to 0, 1, or multiple consumers depending on the dispat The "producer" is a specific instance, process or device that creates and publishes a message. "Publishing" is the process of sending a message or batch -to the intermediary or consumer. +of messages to the intermediary or consumer. ### Consumer From 874938bef8f97965b4753f6678764fa9e692e257 Mon Sep 17 00:00:00 2001 From: Joao Grassi Date: Fri, 30 Sep 2022 09:11:22 +0200 Subject: [PATCH 6/6] Update CHANGELOG.md Co-authored-by: Reiley Yang --- CHANGELOG.md | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d448ff1b6bb..0ebb780edc6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,7 +34,6 @@ release. ([#2706](https://github.com/open-telemetry/opentelemetry-specification/pull/2706)) - Add exceptions to the logs semantic conventions ([#2819](https://github.com/open-telemetry/opentelemetry-specification/pull/2819)) - - Make context propagation requirements explicit for messaging semantic conventions ([#2750](https://github.com/open-telemetry/opentelemetry-specification/pull/2750)).