From 3eccf821c79f5dd2ca2656c43376f55f3869510b Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Wed, 1 Jun 2022 12:30:34 -0700 Subject: [PATCH 01/24] Context propagation requirements for messaging semantic conventions --- ...emantic-conventions-context-propagation.md | 173 ++++++++++++++++++ 1 file changed, 173 insertions(+) create mode 100644 text/trace/messaging-semantic-conventions-context-propagation.md diff --git a/text/trace/messaging-semantic-conventions-context-propagation.md b/text/trace/messaging-semantic-conventions-context-propagation.md new file mode 100644 index 000000000..d0cb70306 --- /dev/null +++ b/text/trace/messaging-semantic-conventions-context-propagation.md @@ -0,0 +1,173 @@ +# Context propagation requirements for messaging semantic conventions + +The [existing messaging semantic conventions for tracing](https://github.com/open-telemetry/opentelemetry-specification/blob/a1a8676a43dce6a4e447f65518aef8e98784306c/specification/trace/semantic_conventions/messaging.md) +implicitly impose certain requirements on context propagation mechanisms used. +This document proposes a way to make these requirements explicit. + +This OTEP is based on [OTEP 0173](0173-messaging-semantic-conventions.md), +which defines basic terms and describes messaging scenarios that should be +supported by messaging semantic conventions. + +* [Terminology](#terminology) +* [Motivation](#motivation) + - [Example](#example) +* [Proposed addition to the messaging semantic conventions](#proposed-addition-to-the-messaging-semantic-conventions) + - [Context propagation](#context-propagation) + - [Requirements](#requirements) +* [Future possibilities](#future-possibilities) + - [Transport context propagation](#transport-context-propagation) + - [Standards for context propagation](#standards-for-context-propagation) + +## Terminology + +For terms used in this document, refer to [OTEP 173](#0173-messaging-semantic-conventions.md#terminology). + +## Motivation + +The current [messaging semantic conventions for tracing](https://github.com/open-telemetry/opentelemetry-specification/blob/a1a8676a43dce6a4e447f65518aef8e98784306c/specification/trace/semantic_conventions/messaging.md) +provide a list of [examples](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/messaging.md#examples), +which illustrate how producer and consumer spans can be correlated by +parent/child relationships or links. All the examples assume that context +information for a given message is propagated from the producer to the consumer. + +However, this is not a trivial assumption, and it is not easily accomodated by +existing established context propagation mechanisms. Those mechanism propagate +context on a per-request basis, whereas the messaging semantic conventions +assume that context is propagated on a per-message basis. This means, that +although several requests might be involved in the processing of a single +message (publishing a message, fetching a message, probable multiple times by +multiple consumers), it is assumed that all components have access to the same +per-message context information that allows correlating all the stages of +processing a message. + +To achieve this desired outcome, a context needs to be attached to a message, +and intermediaries must not alter the context attached to the message. _This +requirement should be documented, as it is an important factor in deciding how +to propagate context for message scenarios, and how to standardize context +propagation for existing message protocols._ + +The additions proposed in this document are not breaking or invalidating any of +the existing semantic conventions for messaging, but rather make an implicit +requirement explicit. + +### Example + +Many intermediaries (message brokers) offer REST APIs for publishing and +fetching messages. A producer can publish a message by sending an HTTP request +to the intermediary, a consumer can pull a message by sending an HTTP request +to the intermediary: + +``` + +----------+ + | Producer | + +----------+ + | + | HTTP POST (publishing a message) + v ++--------------+ +| Intermediary | ++--------------+ + ^ + | HTTP GET (fetching a message) + | + +----------+ + | Consumer | + +----------+ +``` + +In this scenario, it is not possible to correlate producer and consumer traces +for a given message by using HTTP context propagation mechanisms and without +special broker instrumentation. Both HTTP requests, the one that publishes and +the one that pulls the message, originate from different traces. There is no +way the context attached to the HTTP POST request publishing the message can be +forwarded by the HTTP GET request fetching a message. + +For correlating producer and consumer traces without special intermediary +instrumentation it is thus necessary to attach a producer context to the +message so it can be extracted and used by the consumer, regardless of the +contexts that are propagated on HTTP requests for publishing and fetching the +message. + +Although OpenTelemetry semantic conventions cannot specify the exact mechanisms +to achieve this for every intermediary and every protocol, this requirement +must be clearly formulated, so that it can be implemented by protocols and +instrumentations. + +## Proposed addition to the messaging semantic conventions + +### Context propagation + +A message may pass many different components and layers in one or more +intermediaries when it is propagated from the producer to the consumer. It +cannot be assumed, and in many cases, it is not even desired, that all those +components and layers are instrumented and propagate context according to +OpenTelemetry requirements. + +A _creation context_ allows correlating the producer with the consumers of a +message, regardless of intermediary instrumentation. The creation context is +created by the producer and must be propagated to the consumers. It must not be +altered by intermediaries. This context helps to model dependencies between +producers and consumers, regardless of the underlying messaging transport +mechanism and its instrumentation. + +Instrumentors are required to instrument producer and consumer applications +so that context is attached to messages and extracted from messages in a +coordinated way. Future versions of these conventions might recommend [context propagation according to certain industry standards](#standards-for-context-propagation). + +### Requirements + +A producer SHOULD attach a creation context to each message. The creation context +SHOULD be attached in a way so that it is not possible to be changed by intermediaries. + +## Future possibilities + +### Transport context propagation + +As illustrated in the [example](#example) above, a creation context can be +attached to a message, while different contexts are propagated via HTTP requests +publishing and fetching the message. When coming up with conventions and +guidance for intermediary instrumentation, it will be beneficial to clearly +outline those two layers of context propagation and build conventions for +intermediary instrumentation on top of this outline: + +1. The _creation context layer_ allows correlating the producer with the + consumers of a message, regardless of intermediary instrumentation. The + creation context is created by the producer and must be propagated to the + consumers. It must not be altered by intermediaries. + + This layer helps to model dependencies between producers and consumers, + regardless of the underlying messaging transport mechanism and its + instrumentation. +2. An additional _transport context layer_ allows correlating the producer and + the consumer with an intermediary. It also allows correlating multiple + intermediaries among each other. The transport context can be changed by + intermediaries, according to intermediary instrumentations. Intermediaries that + are not instrumented might simply drop the transport context. + + This layer helps to gain insights into details of the message transport. + +This would keep the existing correlation between producers and consumers intact +while allowing intermediaries to use the transport context to correlate +intermediary instrumentation with existing producer and consumer +instrumentations. + +### Standards for context propagation + +Currently, instrumentors have to decide how to attach and extract context from +messages to fulfil the [requirements for context propagation](#context-propagation). +While preserving the freedom for instrumentors to choose how to propagate +context, in the future these conventions should list recommended ways of how to +propagate context using popular messaging protocols. + +Currently, several attempts exist to standardize context propagation for different +messaging protocols and scenarios: + +* [AMQP](https://w3c.github.io/trace-context-amqp/) +* [MQTT](https://w3c.github.io/trace-context-mqtt/) +* [CloudEvents via HTTP](https://github.com/cloudevents/spec/blob/v1.0.1/extensions/distributed-tracing.md) + +Those standards are in draft states and/or are not widely adopted yet. It is +planned to drive those standards to a stable state and to make sure they cover +requirements put forth by these semantic conventions. Finally, these semantic +conventions should give a clear and stable recommendation for each protocol and +scenario. From 9c3be676e22ece5de96a735247c999c8dbe250a7 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Thu, 2 Jun 2022 07:39:04 -0700 Subject: [PATCH 02/24] Add PR number --- ...=> 0205-messaging-semantic-conventions-context-propagation.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename text/trace/{messaging-semantic-conventions-context-propagation.md => 0205-messaging-semantic-conventions-context-propagation.md} (100%) diff --git a/text/trace/messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md similarity index 100% rename from text/trace/messaging-semantic-conventions-context-propagation.md rename to text/trace/0205-messaging-semantic-conventions-context-propagation.md From 5c88d221724f5a20cb06c0b8864dda45491c6212 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Thu, 2 Jun 2022 07:40:58 -0700 Subject: [PATCH 03/24] Fix spelling --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index d0cb70306..b452840d2 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -30,7 +30,7 @@ which illustrate how producer and consumer spans can be correlated by parent/child relationships or links. All the examples assume that context information for a given message is propagated from the producer to the consumer. -However, this is not a trivial assumption, and it is not easily accomodated by +However, this is not a trivial assumption, and it is not easily accommodated by existing established context propagation mechanisms. Those mechanism propagate context on a per-request basis, whereas the messaging semantic conventions assume that context is propagated on a per-message basis. This means, that From a7cd0cc34d38a9b393b9ed9679325c1a23c3021c Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:22:35 -0700 Subject: [PATCH 04/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index b452840d2..ac030d45f 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -1,6 +1,6 @@ # Context propagation requirements for messaging semantic conventions -The [existing messaging semantic conventions for tracing](https://github.com/open-telemetry/opentelemetry-specification/blob/a1a8676a43dce6a4e447f65518aef8e98784306c/specification/trace/semantic_conventions/messaging.md) +The [existing messaging semantic conventions for tracing](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.11.0/specification/trace/semantic_conventions/messaging.md) implicitly impose certain requirements on context propagation mechanisms used. This document proposes a way to make these requirements explicit. From 77ffd8647cd92f5a97647a9e1de15c73f9ed1d11 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:22:51 -0700 Subject: [PATCH 05/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index ac030d45f..b9eecfefc 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -25,7 +25,7 @@ For terms used in this document, refer to [OTEP 173](#0173-messaging-semantic-co ## Motivation The current [messaging semantic conventions for tracing](https://github.com/open-telemetry/opentelemetry-specification/blob/a1a8676a43dce6a4e447f65518aef8e98784306c/specification/trace/semantic_conventions/messaging.md) -provide a list of [examples](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/messaging.md#examples), +provide a list of [examples](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.11.0/specification/trace/semantic_conventions/messaging.md#examples), which illustrate how producer and consumer spans can be correlated by parent/child relationships or links. All the examples assume that context information for a given message is propagated from the producer to the consumer. From 6196390ac4ebe490f7fc44a341ea8ec3505de850 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:23:40 -0700 Subject: [PATCH 06/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index b9eecfefc..2d1facd91 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -31,7 +31,7 @@ parent/child relationships or links. All the examples assume that context information for a given message is propagated from the producer to the consumer. However, this is not a trivial assumption, and it is not easily accommodated by -existing established context propagation mechanisms. Those mechanism propagate +existing established context propagation mechanisms. Those mechanisms propagate context on a per-request basis, whereas the messaging semantic conventions assume that context is propagated on a per-message basis. This means, that although several requests might be involved in the processing of a single From b8eccaadac1974ae7f258a99c331d2f236fd3f48 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:25:41 -0700 Subject: [PATCH 07/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 2d1facd91..9b6d90bf0 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -157,7 +157,7 @@ Currently, instrumentors have to decide how to attach and extract context from messages to fulfil the [requirements for context propagation](#context-propagation). While preserving the freedom for instrumentors to choose how to propagate context, in the future these conventions should list recommended ways of how to -propagate context using popular messaging protocols. +propagate context using well-established messaging protocols. Currently, several attempts exist to standardize context propagation for different messaging protocols and scenarios: From 3edfcc66d4b2d8f46c9ba1d4ebaceb9315d79b47 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:25:49 -0700 Subject: [PATCH 08/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 9b6d90bf0..877e60112 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -24,7 +24,7 @@ For terms used in this document, refer to [OTEP 173](#0173-messaging-semantic-co ## Motivation -The current [messaging semantic conventions for tracing](https://github.com/open-telemetry/opentelemetry-specification/blob/a1a8676a43dce6a4e447f65518aef8e98784306c/specification/trace/semantic_conventions/messaging.md) +The current [messaging semantic conventions for tracing](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.11.0/specification/trace/semantic_conventions/messaging.md) provide a list of [examples](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.11.0/specification/trace/semantic_conventions/messaging.md#examples), which illustrate how producer and consumer spans can be correlated by parent/child relationships or links. All the examples assume that context From 039546261b7900e717343717d4533ccc217d9659 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:28:20 -0700 Subject: [PATCH 09/24] PR comments --- ...ssaging-semantic-conventions-context-propagation.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 877e60112..31fb1e278 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -153,11 +153,11 @@ instrumentations. ### Standards for context propagation -Currently, instrumentors have to decide how to attach and extract context from -messages to fulfil the [requirements for context propagation](#context-propagation). -While preserving the freedom for instrumentors to choose how to propagate -context, in the future these conventions should list recommended ways of how to -propagate context using well-established messaging protocols. +Currently, instrumentation authors have to decide how to attach and extract +context from messages to fulfil the [requirements for context propagation](#context-propagation). +While preserving the freedom for instrumentation authors to choose how to +propagate context, in the future these conventions should list recommended ways +of how to propagate context using well-established messaging protocols. Currently, several attempts exist to standardize context propagation for different messaging protocols and scenarios: From 88fbcc8806c9698668594cc18fcaa9828d2702c8 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:28:51 -0700 Subject: [PATCH 10/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 31fb1e278..788a1969c 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -35,7 +35,7 @@ existing established context propagation mechanisms. Those mechanisms propagate context on a per-request basis, whereas the messaging semantic conventions assume that context is propagated on a per-message basis. This means, that although several requests might be involved in the processing of a single -message (publishing a message, fetching a message, probable multiple times by +message (publishing a message, fetching a message, potentially multiple times by multiple consumers), it is assumed that all components have access to the same per-message context information that allows correlating all the stages of processing a message. From b6758d61408d6aa89f92fee165e3a1511f0f81f1 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:31:36 -0700 Subject: [PATCH 11/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 788a1969c..e55cccc17 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -159,7 +159,7 @@ While preserving the freedom for instrumentation authors to choose how to propagate context, in the future these conventions should list recommended ways of how to propagate context using well-established messaging protocols. -Currently, several attempts exist to standardize context propagation for different +There are several work-in-progress efforts to standardize context propagation for different messaging protocols and scenarios: * [AMQP](https://w3c.github.io/trace-context-amqp/) From 239cf53a00b231c08d1778b9b056e494cf771c43 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:32:13 -0700 Subject: [PATCH 12/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index e55cccc17..e618009f4 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -43,7 +43,7 @@ processing a message. To achieve this desired outcome, a context needs to be attached to a message, and intermediaries must not alter the context attached to the message. _This requirement should be documented, as it is an important factor in deciding how -to propagate context for message scenarios, and how to standardize context +to propagate context for message scenarios and how to standardize context propagation for existing message protocols._ The additions proposed in this document are not breaking or invalidating any of From 32a90ae537f23f2b0cd0257446692ce5ee25164d Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:32:33 -0700 Subject: [PATCH 13/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index e618009f4..955457b15 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -46,7 +46,7 @@ requirement should be documented, as it is an important factor in deciding how to propagate context for message scenarios and how to standardize context propagation for existing message protocols._ -The additions proposed in this document are not breaking or invalidating any of +The additions proposed in this document neither break nor invalidate any of the existing semantic conventions for messaging, but rather make an implicit requirement explicit. From a9f0fd7359a3054024e7db83cdc3c39ec14fad90 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:35:06 -0700 Subject: [PATCH 14/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 955457b15..4086ed5ba 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -103,7 +103,7 @@ cannot be assumed, and in many cases, it is not even desired, that all those components and layers are instrumented and propagate context according to OpenTelemetry requirements. -A _creation context_ allows correlating the producer with the consumers of a +A _creation context_ allows correlating the producer with the consumer(s) of a message, regardless of intermediary instrumentation. The creation context is created by the producer and must be propagated to the consumers. It must not be altered by intermediaries. This context helps to model dependencies between From 667c3589ea530ca0b85350fdff1798f6a1e227e3 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:35:25 -0700 Subject: [PATCH 15/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 4086ed5ba..f7e694e81 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -54,7 +54,7 @@ requirement explicit. Many intermediaries (message brokers) offer REST APIs for publishing and fetching messages. A producer can publish a message by sending an HTTP request -to the intermediary, a consumer can pull a message by sending an HTTP request +to the intermediary and a consumer can pull a message by sending an HTTP request to the intermediary: ``` From 4f969182f8e71e2ca3970ad13e9574ad9369f0be Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:35:44 -0700 Subject: [PATCH 16/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Joao Grassi --- .../0205-messaging-semantic-conventions-context-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index f7e694e81..e7eb31134 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -105,7 +105,7 @@ OpenTelemetry requirements. A _creation context_ allows correlating the producer with the consumer(s) of a message, regardless of intermediary instrumentation. The creation context is -created by the producer and must be propagated to the consumers. It must not be +created by the producer and must be propagated to the consumer(s). It must not be altered by intermediaries. This context helps to model dependencies between producers and consumers, regardless of the underlying messaging transport mechanism and its instrumentation. From c78431ff14a73d2be12c3d8ee496d30c8368caea Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:40:31 -0700 Subject: [PATCH 17/24] PR comments --- ...saging-semantic-conventions-context-propagation.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index e7eb31134..7e5469b6f 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -123,12 +123,11 @@ SHOULD be attached in a way so that it is not possible to be changed by intermed ### Transport context propagation -As illustrated in the [example](#example) above, a creation context can be -attached to a message, while different contexts are propagated via HTTP requests -publishing and fetching the message. When coming up with conventions and -guidance for intermediary instrumentation, it will be beneficial to clearly -outline those two layers of context propagation and build conventions for -intermediary instrumentation on top of this outline: +A creation context can be attached to a message, while different contexts are +propagated with requests that publish and fetch a message. When coming up with +conventions and guidance for intermediary instrumentation, it will be +beneficial to clearly outline those two layers of context propagation and build +conventions for intermediary instrumentation on top of this outline: 1. The _creation context layer_ allows correlating the producer with the consumers of a message, regardless of intermediary instrumentation. The From d62e4f56ec528036279d897cdadc95da9b3b3c80 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:41:59 -0700 Subject: [PATCH 18/24] PR comments --- ...5-messaging-semantic-conventions-context-propagation.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 7e5469b6f..7a5284e93 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -25,10 +25,11 @@ For terms used in this document, refer to [OTEP 173](#0173-messaging-semantic-co ## Motivation The current [messaging semantic conventions for tracing](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.11.0/specification/trace/semantic_conventions/messaging.md) -provide a list of [examples](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.11.0/specification/trace/semantic_conventions/messaging.md#examples), -which illustrate how producer and consumer spans can be correlated by +provide a list of [examples](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.11.0/specification/trace/semantic_conventions/messaging.md#examples). +Those examples illustrate how producer and consumer spans can be correlated by parent/child relationships or links. All the examples assume that context -information for a given message is propagated from the producer to the consumer. +information for a given message is propagated from the producer to the +consumer. However, this is not a trivial assumption, and it is not easily accommodated by existing established context propagation mechanisms. Those mechanisms propagate From d523157829478dba0ce94f3c53428b434a22e915 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Fri, 3 Jun 2022 14:51:55 -0700 Subject: [PATCH 19/24] PR comments --- ...5-messaging-semantic-conventions-context-propagation.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 7a5284e93..a6b54db08 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -166,8 +166,5 @@ messaging protocols and scenarios: * [MQTT](https://w3c.github.io/trace-context-mqtt/) * [CloudEvents via HTTP](https://github.com/cloudevents/spec/blob/v1.0.1/extensions/distributed-tracing.md) -Those standards are in draft states and/or are not widely adopted yet. It is -planned to drive those standards to a stable state and to make sure they cover -requirements put forth by these semantic conventions. Finally, these semantic -conventions should give a clear and stable recommendation for each protocol and -scenario. +Once the standards reach a stable state, these semantic conventions will give a +clear and stable recommendation for each protocol and scenario. From fe827fc38843cceb510ab520f0ac706d3abb1273 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Wed, 8 Jun 2022 14:31:43 -0700 Subject: [PATCH 20/24] Fix wording to make example case clearer --- ...-semantic-conventions-context-propagation.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index a6b54db08..e23c25bd0 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -76,12 +76,17 @@ to the intermediary: +----------+ ``` -In this scenario, it is not possible to correlate producer and consumer traces -for a given message by using HTTP context propagation mechanisms and without -special broker instrumentation. Both HTTP requests, the one that publishes and -the one that pulls the message, originate from different traces. There is no -way the context attached to the HTTP POST request publishing the message can be -forwarded by the HTTP GET request fetching a message. +Existing semantic conventions suppose that the consumer can use context +information from the producer trace to create links or parent/child +relationships between consumer and producer traces. For this to be possible, +context information from the producer needs to be propagated to the consumer. +In the example outlined above, the consumers sends an HTTP GET request to the +intermediary to fetch a message, the message is returned as part of the +response. Via this HTTP request, context information can be propagated from the +consumer to the intermediary, but not from the intermediary to the consumer. +The consumer can obtain the necessary producer context information only if it +is propagated as part of the message itself, independent of HTTP context +propagation. For correlating producer and consumer traces without special intermediary instrumentation it is thus necessary to attach a producer context to the From f0d392c8fcb27c67c104bb7ab0c1299dc7c1fbb2 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Thu, 9 Jun 2022 09:19:08 -0700 Subject: [PATCH 21/24] Rename "creation context" to "message creation context" --- ...emantic-conventions-context-propagation.md | 26 +++++++++---------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index e23c25bd0..63279583a 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -109,12 +109,12 @@ cannot be assumed, and in many cases, it is not even desired, that all those components and layers are instrumented and propagate context according to OpenTelemetry requirements. -A _creation context_ allows correlating the producer with the consumer(s) of a -message, regardless of intermediary instrumentation. The creation context is -created by the producer and must be propagated to the consumer(s). It must not be -altered by intermediaries. This context helps to model dependencies between -producers and consumers, regardless of the underlying messaging transport -mechanism and its instrumentation. +A _message creation context_ allows correlating the producer with the +consumer(s) of a message, regardless of intermediary instrumentation. The +message creation context is created by the producer and must be propagated to +the consumer(s). It must not be altered by intermediaries. This context helps +to model dependencies between producers and consumers, regardless of the +underlying messaging transport mechanism and its instrumentation. Instrumentors are required to instrument producer and consumer applications so that context is attached to messages and extracted from messages in a @@ -122,20 +122,20 @@ coordinated way. Future versions of these conventions might recommend [context p ### Requirements -A producer SHOULD attach a creation context to each message. The creation context +A producer SHOULD attach a message creation context to each message. The message creation context SHOULD be attached in a way so that it is not possible to be changed by intermediaries. ## Future possibilities ### Transport context propagation -A creation context can be attached to a message, while different contexts are -propagated with requests that publish and fetch a message. When coming up with -conventions and guidance for intermediary instrumentation, it will be -beneficial to clearly outline those two layers of context propagation and build -conventions for intermediary instrumentation on top of this outline: +A message creation context can be attached to a message, while different +contexts are propagated with requests that publish and fetch a message. When +coming up with conventions and guidance for intermediary instrumentation, it +will be beneficial to clearly outline those two layers of context propagation +and build conventions for intermediary instrumentation on top of this outline: -1. The _creation context layer_ allows correlating the producer with the +1. The _message context layer_ allows correlating the producer with the consumers of a message, regardless of intermediary instrumentation. The creation context is created by the producer and must be propagated to the consumers. It must not be altered by intermediaries. From a65c09399a5693f374d67e999e9ed4c5f86d7345 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Mon, 13 Jun 2022 16:10:45 -0700 Subject: [PATCH 22/24] PR comment --- ...-messaging-semantic-conventions-context-propagation.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 63279583a..57eb33807 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -111,14 +111,16 @@ OpenTelemetry requirements. A _message creation context_ allows correlating the producer with the consumer(s) of a message, regardless of intermediary instrumentation. The -message creation context is created by the producer and must be propagated to -the consumer(s). It must not be altered by intermediaries. This context helps -to model dependencies between producers and consumers, regardless of the +message creation context is created by the producer and should be propagated to +the consumer(s). It should not be altered by intermediaries. This context +helps to model dependencies between producers and consumers, regardless of the underlying messaging transport mechanism and its instrumentation. Instrumentors are required to instrument producer and consumer applications so that context is attached to messages and extracted from messages in a coordinated way. Future versions of these conventions might recommend [context propagation according to certain industry standards](#standards-for-context-propagation). +If the message creation context cannot be attached to the message and +propagated, consumer traces cannot be directly correlated to producer traces. ### Requirements From 18460bb6b1cb85472f4a296d49f7bb7251554bd1 Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Thu, 16 Jun 2022 11:35:14 -0700 Subject: [PATCH 23/24] Update text/trace/0205-messaging-semantic-conventions-context-propagation.md Co-authored-by: Duane Pauls --- ...205-messaging-semantic-conventions-context-propagation.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 57eb33807..34c6bce8d 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -173,5 +173,6 @@ messaging protocols and scenarios: * [MQTT](https://w3c.github.io/trace-context-mqtt/) * [CloudEvents via HTTP](https://github.com/cloudevents/spec/blob/v1.0.1/extensions/distributed-tracing.md) -Once the standards reach a stable state, these semantic conventions will give a -clear and stable recommendation for each protocol and scenario. +Once the standards reach a stable state and define how the message creation +context and transport context are represented, these semantic conventions will +give a clear and stable recommendation for each protocol and scenario. From 58d748652764aaacfac6b6f54e96e55732b3af8e Mon Sep 17 00:00:00 2001 From: Johannes Tax Date: Thu, 16 Jun 2022 11:41:26 -0700 Subject: [PATCH 24/24] PR comments --- .../0205-messaging-semantic-conventions-context-propagation.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/text/trace/0205-messaging-semantic-conventions-context-propagation.md b/text/trace/0205-messaging-semantic-conventions-context-propagation.md index 34c6bce8d..626d5dcd9 100644 --- a/text/trace/0205-messaging-semantic-conventions-context-propagation.md +++ b/text/trace/0205-messaging-semantic-conventions-context-propagation.md @@ -148,8 +148,7 @@ and build conventions for intermediary instrumentation on top of this outline: 2. An additional _transport context layer_ allows correlating the producer and the consumer with an intermediary. It also allows correlating multiple intermediaries among each other. The transport context can be changed by - intermediaries, according to intermediary instrumentations. Intermediaries that - are not instrumented might simply drop the transport context. + intermediaries, according to intermediary instrumentations. This layer helps to gain insights into details of the message transport.