From d8aa6db992c16ccd83445f31d865ea429a460f4f Mon Sep 17 00:00:00 2001 From: David Ashpole Date: Tue, 16 Aug 2022 17:33:43 +0000 Subject: [PATCH 1/7] reorganize OpenCensus compatibility specification --- specification/compatibility/opencensus.md | 99 +++++++++++------------ 1 file changed, 46 insertions(+), 53 deletions(-) diff --git a/specification/compatibility/opencensus.md b/specification/compatibility/opencensus.md index d19fc2a74d4..d29f3e98ca4 100644 --- a/specification/compatibility/opencensus.md +++ b/specification/compatibility/opencensus.md @@ -2,16 +2,18 @@ **Status**: [Experimental](../document-status.md), Unless otherwise specified. -## Abstract - The OpenTelemetry project aims to provide backwards compatibility with the [OpenCensus](https://opencensus.io) project in order to ease migration of instrumented codebases. -This functionality will be provided as a bridge layer implementing the -[OpenCensus API](https://github.com/census-instrumentation/opencensus-specs) -using the OpenTelemetry API. This layer MUST NOT rely on implementation specific -details of the OpenTelemetry SDK. +## Trace Bridge + +**Status**: [Experimental, Feature Freeze](../document-status.md) + +The trace bridge is provided as a shim layer implementing the +[OpenCensus Trace API](https://github.com/census-instrumentation/opencensus-specs) +using the OpenTelemetry Trace API. This layer MUST NOT rely on implementation +specific details of the OpenTelemetry SDK. More specifically, the intention is to allow OpenCensus instrumentation to be recorded using OpenTelemetry. This Shim Layer MUST NOT publicly expose any @@ -63,33 +65,19 @@ Finally, the Application would update all usages of OpenCensus to OpenTelemetry. |-- Application -> Using OpenTelemetry to generate a sub Trace B-- | ``` -OpenCensus supports two primary types of telemetry: Traces and Stats (Metrics). -Compatibility for these is defined separately. - -> The overriding philosophy for compatibility is that OpenCensus instrumented -> libraries and applications need make *no change* to their API usage in order -> to use OpenTelemetry. All changes should be solely configuration / setup. - -## Goals +### Requirements -OpenTelemetry<->OpenCensus compatibility has the following goals: +OpenTelemetry<->OpenCensus compatibility has the following requirements: 1. OpenCensus has no hard dependency on OpenTelemetry 2. Minimal changes to OpenCensus for implementation 3. Easy for users to use, ideally no change to their code -Additionally, for tracing there are the following goals: +Additionally, for tracing there are the following requirements: 1. Maintain parent-child span relationship between applications and libraries 2. Maintain span link relationships between applications and libraries -## Trace - -**Status**: [Experimental, Feature Freeze](../document-status.md) - -OpenTelemetry will provide an OpenCensus-Trace-Shim component that can be -added as a dependency to ensure compatibility with OpenCensus. - This component MUST be an optional dependency. ### Creating Spans in OpenCensus @@ -105,6 +93,15 @@ and auto injection of dependencies. All specified methods in OpenCensus will delegate to the underlying `Span` of OpenTelemetry. +### Span Attributes + +Span attributes SHOULD be mapped following +[semantic convention mappings](#semantic-convention-mappings) described below. + +### Resources + +Note: resources appear not to be usable in the "API" section of OpenCensus. + #### Known Incompatibilities Below are listed known incompatibilities between OpenTelemetry and OpenCensus @@ -131,12 +128,12 @@ using the OpenCensus <-> OpenTelemetry bridge. "sampled". In this case, the OpenCensus bridge will do its best to support and translate unspecified flags into the closest OpenTelemetry equivalent. -### Context Propagation +## OpenCensus Context Propagation The shim will provide an OpenCensus `PropagationComponent` implementation which maps OpenCenus binary and text propagation to OpenTelemetry context. -#### Text Context +### Text Context This adapter MUST use an OpenTelemetry `TextMapPropagator` to implement the OpenCensus `TextFormat`. @@ -148,43 +145,17 @@ This adapter MUST provide a default `W3CTraceContextPropagator`. If OpenTelemetry defines a global TextMapPropogator, OpenCensus SHOULD use this for OpenCensus `traceContextFormat` propagation. -#### B3 Context +### B3 Context This adapter SHOULD use a contributed OpenTelemetry `B3Propagator` for the B3 text format. -#### OpenCensus Binary Context +### OpenCensus Binary Context This adapter MUST provide an implementation of OpenCensus `BinaryPropogator` to write OpenCensus binary format using OpenTelemetry's context. This implementation may be drawn from OpenCensus if applicable. -### Resources - -Note: resources appear not to be usable in the "API" section of OpenCensus. - -### Semantic Convention Mappings - -Where possible, the tracing shim should provide mappings of labels to attributes -defined within the OpenTelemetry semantic convetions. - -> The principle is to ensure OpenTelemetry exporters, which use these semantic -> conventions, are likely to export the correct data. - -#### HTTP Attributes - -OpenCensus specifies the following [HTTP Attributes](https://github.com/census-instrumentation/opencensus-specs/blob/master/trace/HTTP.md#attributes): - -| OpenCensus Name | OpenTelemetry Name | Comments | -| ------------------ | ------------------ |----------------------| -| `http.host` | `http.host` | | -| `http.method` | `http.method` | | -| `http.user_agent` | `http.user_agent` | | -| `http.status_code` | `http.status_code` | | -| `http.url` | `http.url` | | -| `http.path` | `http.target` | key-name change only | -| `http.route` | N/A | Pass through ok | - ## Metrics / Stats OpenTelemetry will provide an OpenCensus-Metrics-Shim component which @@ -234,3 +205,25 @@ metric exporters. * OpenTelemetry does not currently support context-based attributes (tags). * OpenTelemetry does not support OpenCensus' SumOfSquaredDeviation field; this is dropped when using the bridge. + +## Semantic Convention Mappings + +Where possible, the tracing and metrics shims SHOULD provide mappings of labels +to attributes defined within the OpenTelemetry semantic convetions. + +> The principle is to ensure OpenTelemetry exporters, which use these semantic +> conventions, are likely to export the correct data. + +### HTTP Attributes + +OpenCensus specifies the following [HTTP Attributes](https://github.com/census-instrumentation/opencensus-specs/blob/master/trace/HTTP.md#attributes): + +| OpenCensus Name | OpenTelemetry Name | Comments | +| ------------------ | ------------------ |----------------------| +| `http.host` | `http.host` | | +| `http.method` | `http.method` | | +| `http.user_agent` | `http.user_agent` | | +| `http.status_code` | `http.status_code` | | +| `http.url` | `http.url` | | +| `http.path` | `http.target` | key-name change only | +| `http.route` | N/A | Pass through ok | \ No newline at end of file From 60c9cd41c3051ac558c43a40ff00a96900276f45 Mon Sep 17 00:00:00 2001 From: David Ashpole Date: Tue, 16 Aug 2022 17:34:07 +0000 Subject: [PATCH 2/7] add OpenCensus migration path --- specification/compatibility/opencensus.md | 95 +++++++++++++++++++---- 1 file changed, 81 insertions(+), 14 deletions(-) diff --git a/specification/compatibility/opencensus.md b/specification/compatibility/opencensus.md index d29f3e98ca4..dd1d2c00ad2 100644 --- a/specification/compatibility/opencensus.md +++ b/specification/compatibility/opencensus.md @@ -6,6 +6,77 @@ The OpenTelemetry project aims to provide backwards compatibility with the [OpenCensus](https://opencensus.io) project in order to ease migration of instrumented codebases. +## Migration Path + +### Breaking Changes when migrating to OpenTelemetry + +Migrating from OpenCensus to OpenTelemetry may require breaking changes to the telemetry produced +because of: + +* Different or new semantic conventions for names and attributes (e.g. [`grpc.test.EchoService/Echo`](https://github.com/census-instrumentation/opencensus-specs/blob/master/trace/gRPC.md#grpc-trace) vs [`rpc.server.duration`](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.12.0/semantic_conventions/trace/rpc.yaml#L64)) +* Data model differences (e.g. OpenCensus supports [SumOfSquaredDeviations](https://github.com/census-instrumentation/opencensus-proto/blob/v0.3.0/src/opencensus/proto/metrics/v1/metrics.proto#L195), OTLP does not) +* Instrumentation API feature differences (e.g. OpenCensus supports [context-based attributes](https://github.com/census-instrumentation/opencensus-specs/blob/master/stats/Record.md#recording-stats)), OTel does not) +* Differences between equivalent OC and OTel exporters (e.g. the OpenTelemetry Prometheus exporter [adds type and unit suffixes](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/data-model.md#metric-metadata-1); OpenCensus [does not](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus/blob/v0.4.1/prometheus.go#L227)) + +This migration path groups most breaking changes into the installation of bridges. This gives +users control over introducing the initial breaking change, and makes subsequent migrations of +instrumentation (including in third-party libraries) non-breaking. + +### Migration Plans + +#### Migrating from the OpenCensus Agent and Protocol + +Starting with a deployment of the OC agent, using the OC protocol, migrate by: + +1. Deploy the OpenTelemetry collector with OpenCensus and OTLP receivers and equivalent processors and exporters. +2. **breaking**: For each workload sending the OC protocol, change to sending to the OpenTelemetry collector's OpenCensus receiver. +3. Remove the deployment of the OC Agent +4. For each workload, migrate the application from OpenCensus to OpenTelemetry, following the guidance below, and use the OTLP exporter + +#### Migrating an Application using Bridges + +Starting with an application using entirely OpenCensus instrumention for traces and metrics, it can be migrated by: + +1. Migrate the exporters (SDK) + 1. Install the OpenTelemetry SDK, with an equivalent exporter + 1. If using an OpenCensus exporter, switch to using an OTLP exporter + 2. Install equivalent OpenTelemetry resource detectors + 3. Install OpenTelemetry propagators for OpenCensus' `TextFormat` and `BinaryPropagator` formats. + 4. **breaking**: Install the metrics and trace bridges + 5. Remove initialization of OpenCensus exporters +2. Migrate the instrumentation (API) + 1. **breaking**: For OpenCensus instrumentation packages, migrate to the OpenTelemetry equivalent. + 2. For external dependencies, wait for it to migrate to OpenTelemetry, and update the dependency. + 3. For instrumentation which is part of the application, migrate it following the "library" guidance below. +3. Clean up: Remove the metrics and trace bridges + +#### Migrating Libraries using OC Instrumentation + +##### In-place Migration + +Libraries which want a simple migration can choose to replace instrumentation in-place. + +Starting with a library using OpenCensus Instrumentation: + +1. Annouce to users the library's transition from OpenCensus to OpenTelemetry, and recommend users adopt OC bridges. +2. Change unit tests to use the OC bridges, and use OpenTelemetry unit testing frameworks. +3. After a notification period, migrate instrumentation line-by-line to OpenTelemetry. The notification period should be long for popular libraries. +4. Remove the OC bridge from unit tests. + +##### Migration via Config + +Libraries which are eager to add native OpenTelemetry instrumentation sooner, and/or want to +provide extended support for OpenCensus may choose to provide users the option to use OpenCensus +instrumentation _or_ OpenTelemetry instrumentation. + +Starting with a library using OpenCensus Instrumentation: + +1. Change unit tests to use the OC bridges, and use OpenTelemetry unit testing frameworks. +2. Add configuration allowing users to enable OpenTelemetry instrumentation and disable OpenCensus instrumentation. +3. Add OpenTelemetry instrumentation gated by the configuration, and tested using the same sets of unit tests. +4. After a notification period, switch to using OpenTelemetry instrumentation by default. +5. After a deprecation period, remove the option to use OpenCensus instrumentation. + ## Trace Bridge **Status**: [Experimental, Feature Freeze](../document-status.md) @@ -67,18 +138,14 @@ Finally, the Application would update all usages of OpenCensus to OpenTelemetry. ### Requirements -OpenTelemetry<->OpenCensus compatibility has the following requirements: - -1. OpenCensus has no hard dependency on OpenTelemetry -2. Minimal changes to OpenCensus for implementation -3. Easy for users to use, ideally no change to their code - -Additionally, for tracing there are the following requirements: - -1. Maintain parent-child span relationship between applications and libraries -2. Maintain span link relationships between applications and libraries +The OpenTelemetry<->OpenCensus trace bridge has the following requirements: -This component MUST be an optional dependency. +* OpenCensus has no hard dependency on OpenTelemetry +* Minimal changes to OpenCensus for implementation +* Easy for users to use, ideally no change to their code +* Maintain parent-child span relationship between applications and libraries +* Maintain span link relationships between applications and libraries +* This component MUST be an optional dependency. ### Creating Spans in OpenCensus @@ -105,8 +172,8 @@ Note: resources appear not to be usable in the "API" section of OpenCensus. #### Known Incompatibilities Below are listed known incompatibilities between OpenTelemetry and OpenCensus -specifications. Applications leveraging unspecified behavior from OpenCensus -that *is* specified incompatibly within OpenTelemetry are not eligible for +specifications. Applications leveraging unspecified behavior from OpenCensus +that _is_ specified incompatibly within OpenTelemetry are not eligible for using the OpenCensus <-> OpenTelemetry bridge. 1. In OpenCensus, there is [no specification](https://github.com/census-instrumentation/opencensus-specs/blob/master/trace/Span.md#span-creation) @@ -226,4 +293,4 @@ OpenCensus specifies the following [HTTP Attributes](https://github.com/census-i | `http.status_code` | `http.status_code` | | | `http.url` | `http.url` | | | `http.path` | `http.target` | key-name change only | -| `http.route` | N/A | Pass through ok | \ No newline at end of file +| `http.route` | N/A | Pass through ok | From d3e5dedac2f30855933684a894c2917707ebc933 Mon Sep 17 00:00:00 2001 From: David Ashpole Date: Wed, 14 Dec 2022 16:24:03 +0000 Subject: [PATCH 3/7] update example of different conventions --- specification/compatibility/opencensus.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/compatibility/opencensus.md b/specification/compatibility/opencensus.md index dd1d2c00ad2..1acfa32684a 100644 --- a/specification/compatibility/opencensus.md +++ b/specification/compatibility/opencensus.md @@ -13,7 +13,7 @@ instrumented codebases. Migrating from OpenCensus to OpenTelemetry may require breaking changes to the telemetry produced because of: -* Different or new semantic conventions for names and attributes (e.g. [`grpc.test.EchoService/Echo`](https://github.com/census-instrumentation/opencensus-specs/blob/master/trace/gRPC.md#grpc-trace) vs [`rpc.server.duration`](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.12.0/semantic_conventions/trace/rpc.yaml#L64)) +* Different or new semantic conventions for names and attributes (e.g. [`grpc.io/server/server_latency`](https://github.com/census-instrumentation/opencensus-specs/blob/master/stats/gRPC.md#server) vs [`rpc.server.duration`](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/semantic_conventions/rpc-metrics.md#rpc-server)) * Data model differences (e.g. OpenCensus supports [SumOfSquaredDeviations](https://github.com/census-instrumentation/opencensus-proto/blob/v0.3.0/src/opencensus/proto/metrics/v1/metrics.proto#L195), OTLP does not) * Instrumentation API feature differences (e.g. OpenCensus supports [context-based attributes](https://github.com/census-instrumentation/opencensus-specs/blob/master/stats/Record.md#recording-stats)), OTel does not) * Differences between equivalent OC and OTel exporters (e.g. the OpenTelemetry Prometheus exporter [adds type and unit suffixes](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/data-model.md#metric-metadata-1); OpenCensus [does not](https://github.com/census-ecosystem/opencensus-go-exporter-prometheus/blob/v0.4.1/prometheus.go#L227)) From bb22f11210c4e60cc2a222e706c26c1fce5c1db3 Mon Sep 17 00:00:00 2001 From: David Ashpole Date: Thu, 12 Jan 2023 15:35:36 +0000 Subject: [PATCH 4/7] don't translate semantic conventions in bridges --- specification/compatibility/opencensus.md | 22 ---------------------- 1 file changed, 22 deletions(-) diff --git a/specification/compatibility/opencensus.md b/specification/compatibility/opencensus.md index 1acfa32684a..2f3066c4272 100644 --- a/specification/compatibility/opencensus.md +++ b/specification/compatibility/opencensus.md @@ -272,25 +272,3 @@ metric exporters. * OpenTelemetry does not currently support context-based attributes (tags). * OpenTelemetry does not support OpenCensus' SumOfSquaredDeviation field; this is dropped when using the bridge. - -## Semantic Convention Mappings - -Where possible, the tracing and metrics shims SHOULD provide mappings of labels -to attributes defined within the OpenTelemetry semantic convetions. - -> The principle is to ensure OpenTelemetry exporters, which use these semantic -> conventions, are likely to export the correct data. - -### HTTP Attributes - -OpenCensus specifies the following [HTTP Attributes](https://github.com/census-instrumentation/opencensus-specs/blob/master/trace/HTTP.md#attributes): - -| OpenCensus Name | OpenTelemetry Name | Comments | -| ------------------ | ------------------ |----------------------| -| `http.host` | `http.host` | | -| `http.method` | `http.method` | | -| `http.user_agent` | `http.user_agent` | | -| `http.status_code` | `http.status_code` | | -| `http.url` | `http.url` | | -| `http.path` | `http.target` | key-name change only | -| `http.route` | N/A | Pass through ok | From 0c503456f7c18a66c6fc0ecc29d67f529719422d Mon Sep 17 00:00:00 2001 From: David Ashpole Date: Thu, 12 Jan 2023 16:24:35 +0000 Subject: [PATCH 5/7] remove irrelevant propatators, and gRPC uses binarypropagation directly --- specification/compatibility/opencensus.md | 45 ++++++----------------- 1 file changed, 12 insertions(+), 33 deletions(-) diff --git a/specification/compatibility/opencensus.md b/specification/compatibility/opencensus.md index 2f3066c4272..4f5dbeea729 100644 --- a/specification/compatibility/opencensus.md +++ b/specification/compatibility/opencensus.md @@ -41,11 +41,12 @@ Starting with an application using entirely OpenCensus instrumention for traces 1. Install the OpenTelemetry SDK, with an equivalent exporter 1. If using an OpenCensus exporter, switch to using an OTLP exporter 2. Install equivalent OpenTelemetry resource detectors - 3. Install OpenTelemetry propagators for OpenCensus' `TextFormat` and `BinaryPropagator` formats. + 3. Install OpenTelemetry [`W3c TraceContext`](../context/api-propagators.md#propagators-distribution) propagator, which is the equivalent of the OpenCensus' `TextFormat` propagator. 4. **breaking**: Install the metrics and trace bridges 5. Remove initialization of OpenCensus exporters 2. Migrate the instrumentation (API) 1. **breaking**: For OpenCensus instrumentation packages, migrate to the OpenTelemetry equivalent. + 1. If migrating gRPC, enable the [`BinaryPropagation`](../context/api-propagators.md#propagators-distribution) propagator if the language supports it. Otherwise, enable OpenCensus `BinaryPropagation` on the OpenTelemetry gRPC instrumentation. 2. For external dependencies, wait for it to migrate to OpenTelemetry, and update the dependency. 3. For instrumentation which is part of the application, migrate it following the "library" guidance below. 3. Clean up: Remove the metrics and trace bridges @@ -160,11 +161,6 @@ and auto injection of dependencies. All specified methods in OpenCensus will delegate to the underlying `Span` of OpenTelemetry. -### Span Attributes - -Span attributes SHOULD be mapped following -[semantic convention mappings](#semantic-convention-mappings) described below. - ### Resources Note: resources appear not to be usable in the "API" section of OpenCensus. @@ -195,34 +191,22 @@ using the OpenCensus <-> OpenTelemetry bridge. "sampled". In this case, the OpenCensus bridge will do its best to support and translate unspecified flags into the closest OpenTelemetry equivalent. -## OpenCensus Context Propagation - -The shim will provide an OpenCensus `PropagationComponent` implementation which -maps OpenCenus binary and text propagation to OpenTelemetry context. - -### Text Context - -This adapter MUST use an OpenTelemetry `TextMapPropagator` to implement the -OpenCensus `TextFormat`. - -This adapter SHOULD use configured OpenTelemetry `TextMapPropagator` on the -OpenTelemetry `TraceProvider` for text format propagation. +## OpenCensus Binary Context Propagation -This adapter MUST provide a default `W3CTraceContextPropagator`. If -OpenTelemetry defines a global TextMapPropogator, OpenCensus SHOULD use this -for OpenCensus `traceContextFormat` propagation. - -### B3 Context - -This adapter SHOULD use a contributed OpenTelemetry `B3Propagator` for the -B3 text format. - -### OpenCensus Binary Context +The shim will provide an OpenCensus `BinaryPropogator` implementation which +maps [OpenCenus binary trace context format](https://github.com/census-instrumentation/opencensus-specs/blob/master/encodings/BinaryEncoding.md#trace-context) to an OpenTelemetry +[SpanContext](../overview.md#spancontext). This adapter MUST provide an implementation of OpenCensus `BinaryPropogator` to write OpenCensus binary format using OpenTelemetry's context. This implementation may be drawn from OpenCensus if applicable. +The `BinaryPropagator` MUST be a [TextMapPropagator](../context/api-propagators.md#textmap-propagator) +if possible in the language. Otherwise, the `BinaryPropagator` MUST be a library, +which is expected to be used by OpenTelemetry gRPC instrumentation. gRPC +instrumentation in those languages SHOULD NOT enable the `BinaryPropagator` by +default, but SHOULD provide configuration to allow users to enable it. + ## Metrics / Stats OpenTelemetry will provide an OpenCensus-Metrics-Shim component which @@ -248,11 +232,6 @@ The shim MUST discard the resource attached to OpenCensus metrics, and insert the resource provided during initialization, or fall back to the the default OpenTelemetry resource. -### Metric Attributes - -Metric attributes SHOULD be mapped following -[semantic convention mappings](#semantic-convention-mappings). - ### Instrumentation Scope The shim MUST add an instrumentation scope name and version which identifies From b1278523ce46a7910b0dd457ecaa13ff8d19c067 Mon Sep 17 00:00:00 2001 From: David Ashpole Date: Tue, 17 Jan 2023 15:24:14 +0000 Subject: [PATCH 6/7] changelog --- CHANGELOG.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index a84359118f2..903609f58be 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -44,6 +44,9 @@ release. ### Compatibility - Add Tracer.Close() to the OpenTracing Shim layer. +- Add OpenCensus migration guide and add BinaryPropagation as an option to gRPC + instrumentation for OpenCensus compatibility + ([#3015](https://github.com/open-telemetry/opentelemetry-specification/pull/3015)). ### OpenTelemetry Protocol From 65acf0166281bb6c4023eee88b72874bb44905ce Mon Sep 17 00:00:00 2001 From: David Ashpole Date: Tue, 17 Jan 2023 12:00:07 -0500 Subject: [PATCH 7/7] Update specification/compatibility/opencensus.md Co-authored-by: Reiley Yang --- specification/compatibility/opencensus.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/compatibility/opencensus.md b/specification/compatibility/opencensus.md index 4f5dbeea729..9b1582bd799 100644 --- a/specification/compatibility/opencensus.md +++ b/specification/compatibility/opencensus.md @@ -30,8 +30,8 @@ Starting with a deployment of the OC agent, using the OC protocol, migrate by: 1. Deploy the OpenTelemetry collector with OpenCensus and OTLP receivers and equivalent processors and exporters. 2. **breaking**: For each workload sending the OC protocol, change to sending to the OpenTelemetry collector's OpenCensus receiver. -3. Remove the deployment of the OC Agent -4. For each workload, migrate the application from OpenCensus to OpenTelemetry, following the guidance below, and use the OTLP exporter +3. Remove the deployment of the OC Agent. +4. For each workload, migrate the application from OpenCensus to OpenTelemetry, following the guidance below, and use the OTLP exporter. #### Migrating an Application using Bridges