From 971383f4a59878fe28a4c4c5213ed0eabd67a9d0 Mon Sep 17 00:00:00 2001 From: Alexander Wert Date: Thu, 16 Nov 2023 13:41:39 +0100 Subject: [PATCH 001/124] Changed `messaging.system` attribute type to an open enum (#517) Signed-off-by: Alexander Wert --- CHANGELOG.md | 2 ++ docs/attributes-registry/messaging.md | 17 +++++++++++- docs/messaging/messaging-spans.md | 17 +++++++++++- model/registry/messaging.yaml | 38 ++++++++++++++++++++++++--- 4 files changed, 69 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 44baae1131..21cbd4c1bb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,8 @@ release. ([#317](https://github.com/open-telemetry/semantic-conventions/pull/317)) - BREAKING: Change type of `host.cpu.model.id` and `host.cpu.model.family` to string. ([#495](https://github.com/open-telemetry/semantic-conventions/issues/495)) +- Changed `messaging.system` attribute type to an open enum + ([#517](https://github.com/open-telemetry/semantic-conventions/pull/517)) ### Features diff --git a/docs/attributes-registry/messaging.md b/docs/attributes-registry/messaging.md index 27bbae84c4..fe5523c85d 100644 --- a/docs/attributes-registry/messaging.md +++ b/docs/attributes-registry/messaging.md @@ -36,7 +36,7 @@ | `messaging.rocketmq.message.tag` | string | The secondary classifier of message besides topic. | `tagA` | | `messaging.rocketmq.message.type` | string | Type of message. | `normal` | | `messaging.rocketmq.namespace` | string | Namespace of RocketMQ resources, resources in different namespaces are individual. | `myNamespace` | -| `messaging.system` | string | A string identifying the messaging system. | `kafka`; `rabbitmq`; `rocketmq`; `activemq` | +| `messaging.system` | string | An identifier for the messaging system being used. See below for a list of well-known identifiers. | `activemq` | **[1]:** Instrumentations SHOULD NOT set `messaging.batch.message_count` on spans that operate with a single message. When a messaging client library supports both batch and single-message API for the same operation, instrumentations SHOULD use `messaging.batch.message_count` for batching APIs and SHOULD NOT use it for single-message APIs. @@ -82,4 +82,19 @@ size should be used. | `fifo` | FIFO message | | `delay` | Delay message | | `transaction` | Transaction message | + +`messaging.system` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `activemq` | Apache ActiveMQ | +| `aws_sqs` | Amazon Simple Queue Service (SQS) | +| `azure_eventgrid` | Azure Event Grid | +| `azure_eventhubs` | Azure Event Hubs | +| `azure_servicebus` | Azure Service Bus | +| `gcp_pubsub` | Google Cloud Pub/Sub | +| `jms` | Java Message Service | +| `kafka` | Apache Kafka | +| `rabbitmq` | RabbitMQ | +| `rocketmq` | Apache RocketMQ | diff --git a/docs/messaging/messaging-spans.md b/docs/messaging/messaging-spans.md index fa95e8c3b2..19cfbc43e1 100644 --- a/docs/messaging/messaging-spans.md +++ b/docs/messaging/messaging-spans.md @@ -287,7 +287,7 @@ messages were received). For each message it accounts for, the "Deliver" or | [`messaging.message.envelope.size`](../attributes-registry/messaging.md) | int | The size of the message body and metadata in bytes. [12] | `2738` | Recommended: [13] | | [`messaging.message.id`](../attributes-registry/messaging.md) | string | A value used by the messaging system as an identifier for the message, represented as a string. | `452a7c7c7c7048c2f887f61572b18fc2` | Recommended: [14] | | [`messaging.operation`](../attributes-registry/messaging.md) | string | A string identifying the kind of messaging operation. [15] | `publish` | Required | -| [`messaging.system`](../attributes-registry/messaging.md) | string | A string identifying the messaging system. | `kafka`; `rabbitmq`; `rocketmq`; `activemq` | Required | +| [`messaging.system`](../attributes-registry/messaging.md) | string | An identifier for the messaging system being used. See below for a list of well-known identifiers. | `activemq` | Required | | [`network.peer.address`](../attributes-registry/network.md) | string | Peer address of the network connection - IP address or Unix domain socket name. | `10.1.2.80`; `/tmp/my.sock` | Recommended | | [`network.peer.port`](../attributes-registry/network.md) | int | Peer port number of the network connection. | `65123` | Recommended: If `network.peer.address` is set. | | [`network.protocol.name`](../attributes-registry/network.md) | string | [OSI application layer](https://osi-model.com/application-layer/) or non-OSI equivalent. [16] | `amqp`; `mqtt` | Recommended | @@ -352,6 +352,21 @@ different processes could be listening on TCP port 12345 and UDP port 12345. | `receive` | One or more messages are requested by a consumer. This operation refers to pull-based scenarios, where consumers explicitly call methods of messaging SDKs to receive messages. | | `deliver` | One or more messages are passed to a consumer. This operation refers to push-based scenarios, where consumer register callbacks which get called by messaging SDKs. | +`messaging.system` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `activemq` | Apache ActiveMQ | +| `aws_sqs` | Amazon Simple Queue Service (SQS) | +| `azure_eventgrid` | Azure Event Grid | +| `azure_eventhubs` | Azure Event Hubs | +| `azure_servicebus` | Azure Service Bus | +| `gcp_pubsub` | Google Cloud Pub/Sub | +| `jms` | Java Message Service | +| `kafka` | Apache Kafka | +| `rabbitmq` | RabbitMQ | +| `rocketmq` | Apache RocketMQ | + `network.transport` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. | Value | Description | diff --git a/model/registry/messaging.yaml b/model/registry/messaging.yaml index f7dd6257ab..64dc6fe5a0 100644 --- a/model/registry/messaging.yaml +++ b/model/registry/messaging.yaml @@ -203,6 +203,38 @@ groups: Namespace of RocketMQ resources, resources in different namespaces are individual. examples: 'myNamespace' - id: system - type: string - brief: 'A string identifying the messaging system.' - examples: ['kafka', 'rabbitmq', 'rocketmq', 'activemq'] + brief: > + An identifier for the messaging system being used. See below for a list of well-known identifiers. + type: + allow_custom_values: true + members: + - id: activemq + value: 'activemq' + brief: 'Apache ActiveMQ' + - id: aws_sqs + value: 'aws_sqs' + brief: 'Amazon Simple Queue Service (SQS)' + - id: azure_eventgrid + value: 'azure_eventgrid' + brief: 'Azure Event Grid' + - id: azure_eventhubs + value: 'azure_eventhubs' + brief: 'Azure Event Hubs' + - id: azure_servicebus + value: 'azure_servicebus' + brief: 'Azure Service Bus' + - id: gcp_pubsub + value: 'gcp_pubsub' + brief: 'Google Cloud Pub/Sub' + - id: jms + value: 'jms' + brief: 'Java Message Service' + - id: kafka + value: 'kafka' + brief: 'Apache Kafka' + - id: rabbitmq + value: 'rabbitmq' + brief: 'RabbitMQ' + - id: rocketmq + value: 'rocketmq' + brief: 'Apache RocketMQ' From 2817a7fa5b5d230f70314fdf425e1c468163f1e0 Mon Sep 17 00:00:00 2001 From: Alexander Wert Date: Thu, 16 Nov 2023 17:53:14 +0100 Subject: [PATCH 002/124] Temp fix for separation of resource and semantic attributes (#524) Signed-off-by: Alexander Wert Co-authored-by: Josh Suereth --- model/registry/cloud.yaml | 2 +- model/registry/container.yaml | 2 +- model/registry/device.yaml | 2 +- model/registry/host.yaml | 2 +- model/registry/oci.yaml | 2 +- model/registry/os.yaml | 2 +- model/registry/process.yaml | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/model/registry/cloud.yaml b/model/registry/cloud.yaml index 695bf64719..2c669052e8 100644 --- a/model/registry/cloud.yaml +++ b/model/registry/cloud.yaml @@ -1,7 +1,7 @@ groups: - id: registry.cloud prefix: cloud - type: attribute_group + type: resource brief: > A cloud environment (e.g. GCP, Azure, AWS). attributes: diff --git a/model/registry/container.yaml b/model/registry/container.yaml index 6909fe4ecb..0755d078af 100644 --- a/model/registry/container.yaml +++ b/model/registry/container.yaml @@ -1,7 +1,7 @@ groups: - id: registry.container prefix: container - type: attribute_group + type: resource brief: > A container instance. attributes: diff --git a/model/registry/device.yaml b/model/registry/device.yaml index c109981f11..6eaf8e3b5e 100644 --- a/model/registry/device.yaml +++ b/model/registry/device.yaml @@ -1,7 +1,7 @@ groups: - id: registry.device prefix: device - type: attribute_group + type: resource brief: > Describes device attributes. attributes: diff --git a/model/registry/host.yaml b/model/registry/host.yaml index fed665548e..3deb5f5707 100644 --- a/model/registry/host.yaml +++ b/model/registry/host.yaml @@ -1,7 +1,7 @@ groups: - id: registry.host prefix: host - type: attribute_group + type: resource brief: > A host is defined as a computing instance. For example, physical servers, virtual machines, switches or disk array. attributes: diff --git a/model/registry/oci.yaml b/model/registry/oci.yaml index 24e0cb93f2..45e2838796 100644 --- a/model/registry/oci.yaml +++ b/model/registry/oci.yaml @@ -1,7 +1,7 @@ groups: - id: registry.oci.manifest prefix: oci.manifest - type: attribute_group + type: resource brief: > An OCI image manifest. attributes: diff --git a/model/registry/os.yaml b/model/registry/os.yaml index af8dbfcd2a..5b699b1faf 100644 --- a/model/registry/os.yaml +++ b/model/registry/os.yaml @@ -1,7 +1,7 @@ groups: - id: registry.os prefix: os - type: attribute_group + type: resource brief: > The operating system (OS) on which the process represented by this resource is running. note: > diff --git a/model/registry/process.yaml b/model/registry/process.yaml index f85a94f56b..dd26c09b77 100644 --- a/model/registry/process.yaml +++ b/model/registry/process.yaml @@ -1,7 +1,7 @@ groups: - id: registry.process prefix: process - type: attribute_group + type: resource brief: > An operating system process. attributes: From 56071a86427836bf725470663611a7200ff23341 Mon Sep 17 00:00:00 2001 From: Bogdan Drutu Date: Wed, 22 Jul 2020 11:30:11 -0700 Subject: [PATCH 003/124] Centralize attributes definition (#722) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Centralize attributes definition Signed-off-by: Bogdan Drutu * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/overview.md Co-authored-by: Christian Neumüller Co-authored-by: Christian Neumüller --- specification/common/common.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 specification/common/common.md diff --git a/specification/common/common.md b/specification/common/common.md new file mode 100644 index 0000000000..b8d99a5eef --- /dev/null +++ b/specification/common/common.md @@ -0,0 +1,29 @@ +# Common specification concepts + +
+ +Table of Contents + + +- [Attributes](#attribute) + +
+ +## Attributes + +Attributes are a list of zero or more key-value pairs. An `Attribute` MUST have the following properties: + +- The attribute key, which MUST be a non-`null` and non-empty string. +- The attribute value, which is either: + - A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer. + - An array of primitive type values. The array MUST be homogeneous, + i.e. it MUST NOT contain values of different types. For protocols that do + not natively support array values such values SHOULD be represented as JSON strings. + +`null` values within arrays MUST be preserved as-is (i.e., passed on to span +processors / exporters as `null`). If exporters do not support exporting `null` +values, they MAY replace those values by 0, `false`, or empty strings. +This is required for map/dictionary structures represented as two arrays with +indices that are kept in sync (e.g., two attributes `header_keys` and `header_values`, +both containing an array of strings to represent a mapping +`header_keys[i] -> header_values[i]`). From c82cc039d7d4ab2908034fa49be2c9e76814185b Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Mon, 27 Jul 2020 10:51:44 -0700 Subject: [PATCH 004/124] DocFX sanity check (#742) --- specification/common/common.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/common.md b/specification/common/common.md index b8d99a5eef..b55f47c1bb 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -5,7 +5,7 @@ Table of Contents -- [Attributes](#attribute) +- [Attributes](#attributes) From 19fced8d4f5051ca50110ba87aae32a7eb92391f Mon Sep 17 00:00:00 2001 From: Giovanni Liva Date: Thu, 13 Aug 2020 19:10:46 +0200 Subject: [PATCH 005/124] Consistency between Span and Resource attributes (#777) * Consistency between Span and Resource attributes * Address feedback * Wording --- specification/common/common.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index b55f47c1bb..681721fb11 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -20,6 +20,15 @@ Attributes are a list of zero or more key-value pairs. An `Attribute` MUST have i.e. it MUST NOT contain values of different types. For protocols that do not natively support array values such values SHOULD be represented as JSON strings. +Attributes SHOULD preserve the order in which they're set. + +Attribute values expressing a numerical value of zero, an empty string, or an +empty array are considered meaningful and MUST be stored and passed on to +processors / exporters. Attribute values of `null` are considered to be not set +and get discarded as if that `Attribute` has never been created. +As an exception to this, if overwriting of values is supported, this results in +removing the attribute. + `null` values within arrays MUST be preserved as-is (i.e., passed on to span processors / exporters as `null`). If exporters do not support exporting `null` values, they MAY replace those values by 0, `false`, or empty strings. From 795086a9a596b54ffc4e280d6dae91abc382b4c4 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Mon, 17 Aug 2020 12:40:17 -0400 Subject: [PATCH 006/124] Add conventions for attribute names (#807) * Add conventions for attribute names Resolves: https://github.com/open-telemetry/opentelemetry-specification/issues/726 * Address PR comments * Re-word company/application specific attribute recommendations --- specification/common/common.md | 88 +++++++++++++++++++++++++++++++++- 1 file changed, 87 insertions(+), 1 deletion(-) diff --git a/specification/common/common.md b/specification/common/common.md index 681721fb11..62637d5ec9 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -6,7 +6,7 @@ Table of Contents - [Attributes](#attributes) - + - [Attribute Naming](#attribute-naming) ## Attributes @@ -36,3 +36,89 @@ This is required for map/dictionary structures represented as two arrays with indices that are kept in sync (e.g., two attributes `header_keys` and `header_values`, both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). + +### Attribute Naming + +Attribute name (also known as the "attribute key") MUST be a valid Unicode +sequence. + +Attribute names SHOULD follow these rules: + +- Use namespacing to avoid name clashes. Delimit the namespaces using a dot + character. For example `service.version` denotes the service version where + `service` is the namespace and `version` is an attribute in that namespace. + +- Namespaces can be nested. For example `telemetry.sdk` is a namespace inside + top-level `telemetry` namespace and `telemetry.sdk.name` is an attribute + inside `telemetry.sdk` namespace. + +- For each multi-word dot-delimited component of the attribute name separate the + words by underscores (i.e. use snake_case). For example `http.status_code` + denotes the status code in the http namespace. + +- Attribute names SHOULD NOT coincide with namespaces. For example if + `service.instance.id` is an attribute name then it is no longer valid to have + an attribute named `service.instance` because `service.instance` is already a + namespace. Because of this rule be careful when choosing attribute names: + every existing attribute name prohibits existence of an equally named + namespace in the future, and vice versa: any existing namespace prohibits + existence of an equally named attribute in the future. + +#### Recommendations for OpenTelemetry Authors + +- All attribute names that are part of OpenTelemetry semantic conventions + SHOULD be part of a namespace. + +- When coming up with a new convention make sure to check existing namespaces + for + [Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), + [Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), + and + [Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions) + to see if the new attributes fits. + +- When a new namespace is necessary consider whether it should be a top-level + namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). + +- Semantic conventions MUST limit attribute names to printable Basic Latin + characters (more precisely to + [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) + subset of Unicode code points). It is recommended to further limit attribute + names to the following Unicode code points: Latin alphabet, Numeric, + Underscore, Dot (as namespace delimiter). + +#### Recommendations for Application Developers + +As an application developer when you need to record an attribute first consult +existing semantic conventions for +[Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), +[Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), +and +[Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions). +If appropriate attribute name does not exists you will need to come up with a +new name. To do that consider a few options: + +- The attribute is specific to your company and may be possibly used outside the + company as well. To avoid name clashes with attributes introduced by other + companies (in a distributed system that uses applications from multiple + vendors) it is recommended to prefix the attribute name by your company's + reverse domain name, e.g. `com.acme.shopname`. + +- The attribute is specific to your application that will be used internally + only. If you already have an internal company process that helps you to ensure + no name clashes happen then feel free to follow it. Otherwise it is + recommended to prefix the attribute name by your application name, provided + that the application name is reasonably unique within your organization (e.g. + `myuniquemapapp.longitude` is likely fine). Make sure the application name + does not clash with an existing semantic convention namespace. + +- The attribute may be generally applicable to applications in the industry. In + that case consider submitting a proposal to this specification to add a new + attribute to the semantic conventions, if necessary also to add a new + namespace for the attribute. + +It is recommended to limit attribute names to printable Basic Latin characters +(more precisely to +[U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) +subset of Unicode code points). + From 2096cb3ec5061046df8f98148af1c4ddab645a2b Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Wed, 19 Aug 2020 13:49:28 -0400 Subject: [PATCH 007/124] Extend attribute naming rules to metric labels (#821) * Extend attribute naming rules to metric labels We earlier defined naming rules for attributes, however we do not have similar rules for metric labels. This commit extends the exact same set of rules to metric labels. This was brought up in this comment https://github.com/open-telemetry/opentelemetry-specification/pull/807#discussion_r471550053 * Address PR comments --- specification/common/common.md | 95 +++++++++++++++++++--------------- 1 file changed, 52 insertions(+), 43 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 62637d5ec9..7cedad5f3f 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -6,7 +6,7 @@ Table of Contents - [Attributes](#attributes) - - [Attribute Naming](#attribute-naming) + - [Attribute and Label Naming](#attribute-and-label-naming) ## Attributes @@ -37,12 +37,22 @@ indices that are kept in sync (e.g., two attributes `header_keys` and `header_va both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). -### Attribute Naming +## Attribute and Label Naming -Attribute name (also known as the "attribute key") MUST be a valid Unicode -sequence. +_This section applies to Resource, Span and Log attribute names (also known as +the "attribute keys") and to keys of Metric labels. For brevity within this +section when we use the term "name" without an adjective it is implied to mean +"attribute name or metric label key"._ -Attribute names SHOULD follow these rules: +Every name MUST be a valid Unicode sequence. + +_Note: we merely require that the names are represented as Unicode sequences. +This specification does not define how exactly the Unicode sequences are +encoded. The encoding can vary from one programming language to another and from +one wire format to another. Use the idiomatic way to represent Unicode in the +particular programming language or wire format._ + +Names SHOULD follow these rules: - Use namespacing to avoid name clashes. Delimit the namespaces using a dot character. For example `service.version` denotes the service version where @@ -56,68 +66,67 @@ Attribute names SHOULD follow these rules: words by underscores (i.e. use snake_case). For example `http.status_code` denotes the status code in the http namespace. -- Attribute names SHOULD NOT coincide with namespaces. For example if +- Names SHOULD NOT coincide with namespaces. For example if `service.instance.id` is an attribute name then it is no longer valid to have an attribute named `service.instance` because `service.instance` is already a - namespace. Because of this rule be careful when choosing attribute names: - every existing attribute name prohibits existence of an equally named - namespace in the future, and vice versa: any existing namespace prohibits - existence of an equally named attribute in the future. + namespace. Because of this rule be careful when choosing names: every existing + name prohibits existence of an equally named namespace in the future, and vice + versa: any existing namespace prohibits existence of an equally named + attribute or label key in the future. -#### Recommendations for OpenTelemetry Authors +### Recommendations for OpenTelemetry Authors -- All attribute names that are part of OpenTelemetry semantic conventions - SHOULD be part of a namespace. +- All names that are part of OpenTelemetry semantic conventions SHOULD be part + of a namespace. -- When coming up with a new convention make sure to check existing namespaces - for +- When coming up with a new semantic convention make sure to check existing + namespaces for [Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), [Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), and [Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions) - to see if the new attributes fits. + to see if the new name fits. - When a new namespace is necessary consider whether it should be a top-level namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). -- Semantic conventions MUST limit attribute names to printable Basic Latin - characters (more precisely to +- Semantic conventions MUST limit names to printable Basic Latin characters + (more precisely to [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) - subset of Unicode code points). It is recommended to further limit attribute - names to the following Unicode code points: Latin alphabet, Numeric, - Underscore, Dot (as namespace delimiter). + subset of Unicode code points). It is recommended to further limit names to + the following Unicode code points: Latin alphabet, Numeric, Underscore, Dot + (as namespace delimiter). -#### Recommendations for Application Developers +### Recommendations for Application Developers -As an application developer when you need to record an attribute first consult -existing semantic conventions for +As an application developer when you need to record an attribute or a label +first consult existing semantic conventions for [Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), [Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), and [Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions). -If appropriate attribute name does not exists you will need to come up with a -new name. To do that consider a few options: - -- The attribute is specific to your company and may be possibly used outside the - company as well. To avoid name clashes with attributes introduced by other - companies (in a distributed system that uses applications from multiple - vendors) it is recommended to prefix the attribute name by your company's - reverse domain name, e.g. `com.acme.shopname`. - -- The attribute is specific to your application that will be used internally - only. If you already have an internal company process that helps you to ensure - no name clashes happen then feel free to follow it. Otherwise it is - recommended to prefix the attribute name by your application name, provided - that the application name is reasonably unique within your organization (e.g. +If an appropriate name does not exists you will need to come up with a new name. +To do that consider a few options: + +- The name is specific to your company and may be possibly used outside the + company as well. To avoid clashes with names introduced by other companies (in + a distributed system that uses applications from multiple vendors) it is + recommended to prefix the new name by your company's reverse domain name, e.g. + `com.acme.shopname`. + +- The name is specific to your application that will be used internally only. If + you already have an internal company process that helps you to ensure no name + clashes happen then feel free to follow it. Otherwise it is recommended to + prefix the attribute name or label key by your application name, provided that + the application name is reasonably unique within your organization (e.g. `myuniquemapapp.longitude` is likely fine). Make sure the application name does not clash with an existing semantic convention namespace. -- The attribute may be generally applicable to applications in the industry. In - that case consider submitting a proposal to this specification to add a new - attribute to the semantic conventions, if necessary also to add a new - namespace for the attribute. +- The name may be generally applicable to applications in the industry. In that + case consider submitting a proposal to this specification to add a new name to + the semantic conventions, and if necessary also to add a new namespace. -It is recommended to limit attribute names to printable Basic Latin characters +It is recommended to limit names to printable Basic Latin characters (more precisely to [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) subset of Unicode code points). From 52061e2f521387e447bb7b1cfaf5d6c10a60a41d Mon Sep 17 00:00:00 2001 From: Bogdan Drutu Date: Wed, 19 Aug 2020 14:43:05 -0700 Subject: [PATCH 008/124] Fix lint check in makefile, fix errors (#837) Signed-off-by: Bogdan Drutu --- specification/common/common.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/common.md b/specification/common/common.md index 7cedad5f3f..f6a917b4cf 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -7,6 +7,7 @@ Table of Contents - [Attributes](#attributes) - [Attribute and Label Naming](#attribute-and-label-naming) + ## Attributes @@ -130,4 +131,3 @@ It is recommended to limit names to printable Basic Latin characters (more precisely to [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) subset of Unicode code points). - From a59a1b218c6d26ab3832a01bcf1cd4fc1246dcf3 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Mon, 24 Aug 2020 13:14:37 -0400 Subject: [PATCH 009/124] Require that names and namespaces are one global space across all semantic convention areas (#832) * Require that names and namespaces are one global space across all semantic convention areas We have semantic conventions for Resources, Spans and Metrics (in the future also Logs are expected). It was not clear if the attribute names across all convention areas should be globally unique. This commit asserts that conventions are one space, they are not independent spaces with their own namespaces each. We prohibit using the same Span or Resource attribute name or metric label name but give them slightly different meanings or value sets. Resolves: https://github.com/open-telemetry/opentelemetry-specification/issues/815 * Address PR comments Co-authored-by: Bogdan Drutu --- specification/common/common.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index f6a917b4cf..44f5ccf2f5 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -91,6 +91,15 @@ Names SHOULD follow these rules: - When a new namespace is necessary consider whether it should be a top-level namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). +- Semantic conventions exist for four areas: for Resource, Span and Log + attribute names as well as Metric label keys. In addition, for spans we have + two more areas: Event and Link attribute names. Identical namespaces or names + in all these areas MUST have identical meanings. For example the `http.method` + span attribute name denotes exactly the same concept as the `http.method` + metric label, has the same data type and the same set of possible values (in + both cases it records the value of the HTTP protocol's request method as a + string). + - Semantic conventions MUST limit names to printable Basic Latin characters (more precisely to [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) From aa5effe43c77f58c2e0226f6d3456f8f6e06c7c2 Mon Sep 17 00:00:00 2001 From: Armin Ruech Date: Fri, 25 Sep 2020 11:49:15 +0200 Subject: [PATCH 010/124] Define `null` as an invalid value for attributes and declare attempts to set `null` as undefined behavior (#992) --- specification/common/common.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 44f5ccf2f5..974cf51821 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -25,10 +25,10 @@ Attributes SHOULD preserve the order in which they're set. Attribute values expressing a numerical value of zero, an empty string, or an empty array are considered meaningful and MUST be stored and passed on to -processors / exporters. Attribute values of `null` are considered to be not set -and get discarded as if that `Attribute` has never been created. -As an exception to this, if overwriting of values is supported, this results in -removing the attribute. +processors / exporters. + +Attribute values of `null` are not valid and attempting to set a `null` value is +undefined behavior. `null` values within arrays MUST be preserved as-is (i.e., passed on to span processors / exporters as `null`). If exporters do not support exporting `null` From 2d55f5436f93d88908ebbad5144dc2a74228d291 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20Neum=C3=BCller?= Date: Wed, 4 Nov 2020 21:29:48 +0100 Subject: [PATCH 011/124] Fix absolute links to spec (#1192) * Remove absolute links where possible. * Work around semantic conventions. * Docfx. * Fix YAML. * More docfx. --- specification/common/common.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 974cf51821..0352f819ab 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -82,10 +82,10 @@ Names SHOULD follow these rules: - When coming up with a new semantic convention make sure to check existing namespaces for - [Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), - [Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), + [Resources](../resource/semantic_conventions/README.md), + [Spans](../trace/semantic_conventions/README.md), and - [Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions) + [Metrics](../metrics/semantic_conventions/README.md) to see if the new name fits. - When a new namespace is necessary consider whether it should be a top-level @@ -111,10 +111,10 @@ Names SHOULD follow these rules: As an application developer when you need to record an attribute or a label first consult existing semantic conventions for -[Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), -[Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), +[Resources](../resource/semantic_conventions/README.md), +[Spans](../trace/semantic_conventions/README.md), and -[Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions). +[Metrics](../metrics/semantic_conventions/README.md). If an appropriate name does not exists you will need to come up with a new name. To do that consider a few options: From 550a2b904b4580180e37bd4ef711f48ba66c74ad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20Neum=C3=BCller?= Date: Tue, 10 Nov 2020 17:05:58 +0100 Subject: [PATCH 012/124] Remove ordering reqirement for attributes. (#1212) * Remove ordering for attributes. * Fill in CHANGELOG link --- specification/common/common.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 0352f819ab..fca4b007a1 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -21,8 +21,6 @@ Attributes are a list of zero or more key-value pairs. An `Attribute` MUST have i.e. it MUST NOT contain values of different types. For protocols that do not natively support array values such values SHOULD be represented as JSON strings. -Attributes SHOULD preserve the order in which they're set. - Attribute values expressing a numerical value of zero, an empty string, or an empty array are considered meaningful and MUST be stored and passed on to processors / exporters. From 01e5a6d827d2ba18b26410a1de6386b4d56e2a50 Mon Sep 17 00:00:00 2001 From: Przemek Maciolek <58699843+pmm-sumo@users.noreply.github.com> Date: Tue, 10 Nov 2020 21:19:52 +0100 Subject: [PATCH 013/124] Include attribute name pluralization guidelines (#1115) (#1140) --- specification/common/common.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index fca4b007a1..3a759e09e0 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -7,6 +7,9 @@ Table of Contents - [Attributes](#attributes) - [Attribute and Label Naming](#attribute-and-label-naming) + - [Name Pluralization Guidelines](#name-pluralization-guidelines) + - [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) + - [Recommendations for Application Developers](#recommendations-for-application-developers) @@ -72,6 +75,19 @@ Names SHOULD follow these rules: name prohibits existence of an equally named namespace in the future, and vice versa: any existing namespace prohibits existence of an equally named attribute or label key in the future. + +### Name Pluralization guidelines + +- When an attribute represents a single entity, the attribute name SHOULD be singular. + Examples: `host.name`, `db.user`, `container.id`. + +- When attribute can represent multiple entities, the attribute name SHOULD be pluralized + and the value type SHOULD be an array. E.g. `process.command_args` might include multiple + values: the executable name and command arguments. + +- When an attribute represents a measurement, + [Metric Name Pluralization Guidelines](../metrics/semantic_conventions/README.md#pluralization) + SHOULD be followed for the attribute name. ### Recommendations for OpenTelemetry Authors From fb0804ed305920e65ddc56807865c9ae95eb7a44 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Tue, 10 Nov 2020 15:51:16 -0500 Subject: [PATCH 014/124] Add guidance on when to use and not use nested namespaces (#1197) This topic has come up at least 3 times now. I believe a clarification is warranted. The clarification is aligned with previous recommendations: https://github.com/open-telemetry/opentelemetry-specification/pull/789#issuecomment-675506855 https://github.com/open-telemetry/opentelemetry-specification/pull/882#issuecomment-690945900 https://github.com/open-telemetry/opentelemetry-specification/pull/1194#issuecomment-722431982 --- specification/common/common.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index 3a759e09e0..15d25304c4 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -63,6 +63,10 @@ Names SHOULD follow these rules: - Namespaces can be nested. For example `telemetry.sdk` is a namespace inside top-level `telemetry` namespace and `telemetry.sdk.name` is an attribute inside `telemetry.sdk` namespace. + Note: the fact that an entity is located within another entity is typically + not a sufficient reason for forming nested namespaces. The purpose of a + namespace is to avoid name clashes, not to indicate entity hierarchies. This + purpose should primarily drive the decision about forming nested namespaces. - For each multi-word dot-delimited component of the attribute name separate the words by underscores (i.e. use snake_case). For example `http.status_code` From 1cbb4af73d55710bba5905e2e725ed17b18ac3a0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20Neum=C3=BCller?= Date: Wed, 11 Nov 2020 02:15:41 +0100 Subject: [PATCH 015/124] Nulls SHOULD NOT be allowed in arrays. (#1214) * Nulls SHOULD NOT be allowed in arrays. * Fill in CHANGELOG link Co-authored-by: Armin Ruech --- specification/common/common.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index 15d25304c4..f1526b9518 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -31,6 +31,9 @@ processors / exporters. Attribute values of `null` are not valid and attempting to set a `null` value is undefined behavior. +`null` values SHOULD NOT be allowed in arrays. However, if it is impossible to +make sure that no `null` values are accepted +(e.g. in languages that do not have appropriate compile-time type checking), `null` values within arrays MUST be preserved as-is (i.e., passed on to span processors / exporters as `null`). If exporters do not support exporting `null` values, they MAY replace those values by 0, `false`, or empty strings. From e8587bacc85f1ab7dca1de7defe2a8eca3e13777 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Tue, 17 Nov 2020 16:41:56 -0500 Subject: [PATCH 016/124] Declare freeze of Trace API Specification 1.0 (#1121) * Declare freeze of Trace Specification 1.0 We want to freeze Trace specification 1.0 so that we no longer accept substantial changes (unless a fundamental problem is found in the spec). Resolves https://github.com/open-telemetry/opentelemetry-specification/issues/1120 --- .../common/attribute-and-label-naming.md | 130 ++++++++++++++++++ specification/common/common.md | 126 +---------------- 2 files changed, 133 insertions(+), 123 deletions(-) create mode 100644 specification/common/attribute-and-label-naming.md diff --git a/specification/common/attribute-and-label-naming.md b/specification/common/attribute-and-label-naming.md new file mode 100644 index 0000000000..d84eab4c62 --- /dev/null +++ b/specification/common/attribute-and-label-naming.md @@ -0,0 +1,130 @@ +# Attribute and Label Naming + +
+ +Table of Contents + + +- [Name Pluralization Guidelines](#name-pluralization-guidelines) +- [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) +- [Recommendations for Application Developers](#recommendations-for-application-developers) + +
+ +_This section applies to Resource, Span and Log attribute names (also known as +the "attribute keys") and to keys of Metric labels. For brevity within this +section when we use the term "name" without an adjective it is implied to mean +"attribute name or metric label key"._ + +Every name MUST be a valid Unicode sequence. + +_Note: we merely require that the names are represented as Unicode sequences. +This specification does not define how exactly the Unicode sequences are +encoded. The encoding can vary from one programming language to another and from +one wire format to another. Use the idiomatic way to represent Unicode in the +particular programming language or wire format._ + +Names SHOULD follow these rules: + +- Use namespacing to avoid name clashes. Delimit the namespaces using a dot + character. For example `service.version` denotes the service version where + `service` is the namespace and `version` is an attribute in that namespace. + +- Namespaces can be nested. For example `telemetry.sdk` is a namespace inside + top-level `telemetry` namespace and `telemetry.sdk.name` is an attribute + inside `telemetry.sdk` namespace. + Note: the fact that an entity is located within another entity is typically + not a sufficient reason for forming nested namespaces. The purpose of a + namespace is to avoid name clashes, not to indicate entity hierarchies. This + purpose should primarily drive the decision about forming nested namespaces. + +- For each multi-word dot-delimited component of the attribute name separate the + words by underscores (i.e. use snake_case). For example `http.status_code` + denotes the status code in the http namespace. + +- Names SHOULD NOT coincide with namespaces. For example if + `service.instance.id` is an attribute name then it is no longer valid to have + an attribute named `service.instance` because `service.instance` is already a + namespace. Because of this rule be careful when choosing names: every existing + name prohibits existence of an equally named namespace in the future, and vice + versa: any existing namespace prohibits existence of an equally named + attribute or label key in the future. + +## Name Pluralization guidelines + +- When an attribute represents a single entity, the attribute name SHOULD be singular. + Examples: `host.name`, `db.user`, `container.id`. + +- When attribute can represent multiple entities, the attribute name SHOULD be pluralized + and the value type SHOULD be an array. E.g. `process.command_args` might include multiple + values: the executable name and command arguments. + +- When an attribute represents a measurement, + [Metric Name Pluralization Guidelines](../metrics/semantic_conventions/README.md#pluralization) + SHOULD be followed for the attribute name. + +## Recommendations for OpenTelemetry Authors + +- All names that are part of OpenTelemetry semantic conventions SHOULD be part + of a namespace. + +- When coming up with a new semantic convention make sure to check existing + namespaces for + [Resources](../resource/semantic_conventions/README.md), + [Spans](../trace/semantic_conventions/README.md), + and + [Metrics](../metrics/semantic_conventions/README.md) + to see if the new name fits. + +- When a new namespace is necessary consider whether it should be a top-level + namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). + +- Semantic conventions exist for four areas: for Resource, Span and Log + attribute names as well as Metric label keys. In addition, for spans we have + two more areas: Event and Link attribute names. Identical namespaces or names + in all these areas MUST have identical meanings. For example the `http.method` + span attribute name denotes exactly the same concept as the `http.method` + metric label, has the same data type and the same set of possible values (in + both cases it records the value of the HTTP protocol's request method as a + string). + +- Semantic conventions MUST limit names to printable Basic Latin characters + (more precisely to + [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) + subset of Unicode code points). It is recommended to further limit names to + the following Unicode code points: Latin alphabet, Numeric, Underscore, Dot + (as namespace delimiter). + +## Recommendations for Application Developers + +As an application developer when you need to record an attribute or a label +first consult existing semantic conventions for +[Resources](../resource/semantic_conventions/README.md), +[Spans](../trace/semantic_conventions/README.md), +and +[Metrics](../metrics/semantic_conventions/README.md). +If an appropriate name does not exists you will need to come up with a new name. +To do that consider a few options: + +- The name is specific to your company and may be possibly used outside the + company as well. To avoid clashes with names introduced by other companies (in + a distributed system that uses applications from multiple vendors) it is + recommended to prefix the new name by your company's reverse domain name, e.g. + `com.acme.shopname`. + +- The name is specific to your application that will be used internally only. If + you already have an internal company process that helps you to ensure no name + clashes happen then feel free to follow it. Otherwise it is recommended to + prefix the attribute name or label key by your application name, provided that + the application name is reasonably unique within your organization (e.g. + `myuniquemapapp.longitude` is likely fine). Make sure the application name + does not clash with an existing semantic convention namespace. + +- The name may be generally applicable to applications in the industry. In that + case consider submitting a proposal to this specification to add a new name to + the semantic conventions, and if necessary also to add a new namespace. + +It is recommended to limit names to printable Basic Latin characters +(more precisely to +[U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) +subset of Unicode code points). diff --git a/specification/common/common.md b/specification/common/common.md index f1526b9518..51ce946c48 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -1,15 +1,13 @@ # Common specification concepts +**Status**: [Feature-freeze](../document-status.md). +
Table of Contents - [Attributes](#attributes) - - [Attribute and Label Naming](#attribute-and-label-naming) - - [Name Pluralization Guidelines](#name-pluralization-guidelines) - - [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) - - [Recommendations for Application Developers](#recommendations-for-application-developers)
@@ -42,122 +40,4 @@ indices that are kept in sync (e.g., two attributes `header_keys` and `header_va both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). -## Attribute and Label Naming - -_This section applies to Resource, Span and Log attribute names (also known as -the "attribute keys") and to keys of Metric labels. For brevity within this -section when we use the term "name" without an adjective it is implied to mean -"attribute name or metric label key"._ - -Every name MUST be a valid Unicode sequence. - -_Note: we merely require that the names are represented as Unicode sequences. -This specification does not define how exactly the Unicode sequences are -encoded. The encoding can vary from one programming language to another and from -one wire format to another. Use the idiomatic way to represent Unicode in the -particular programming language or wire format._ - -Names SHOULD follow these rules: - -- Use namespacing to avoid name clashes. Delimit the namespaces using a dot - character. For example `service.version` denotes the service version where - `service` is the namespace and `version` is an attribute in that namespace. - -- Namespaces can be nested. For example `telemetry.sdk` is a namespace inside - top-level `telemetry` namespace and `telemetry.sdk.name` is an attribute - inside `telemetry.sdk` namespace. - Note: the fact that an entity is located within another entity is typically - not a sufficient reason for forming nested namespaces. The purpose of a - namespace is to avoid name clashes, not to indicate entity hierarchies. This - purpose should primarily drive the decision about forming nested namespaces. - -- For each multi-word dot-delimited component of the attribute name separate the - words by underscores (i.e. use snake_case). For example `http.status_code` - denotes the status code in the http namespace. - -- Names SHOULD NOT coincide with namespaces. For example if - `service.instance.id` is an attribute name then it is no longer valid to have - an attribute named `service.instance` because `service.instance` is already a - namespace. Because of this rule be careful when choosing names: every existing - name prohibits existence of an equally named namespace in the future, and vice - versa: any existing namespace prohibits existence of an equally named - attribute or label key in the future. - -### Name Pluralization guidelines - -- When an attribute represents a single entity, the attribute name SHOULD be singular. - Examples: `host.name`, `db.user`, `container.id`. - -- When attribute can represent multiple entities, the attribute name SHOULD be pluralized - and the value type SHOULD be an array. E.g. `process.command_args` might include multiple - values: the executable name and command arguments. - -- When an attribute represents a measurement, - [Metric Name Pluralization Guidelines](../metrics/semantic_conventions/README.md#pluralization) - SHOULD be followed for the attribute name. - -### Recommendations for OpenTelemetry Authors - -- All names that are part of OpenTelemetry semantic conventions SHOULD be part - of a namespace. - -- When coming up with a new semantic convention make sure to check existing - namespaces for - [Resources](../resource/semantic_conventions/README.md), - [Spans](../trace/semantic_conventions/README.md), - and - [Metrics](../metrics/semantic_conventions/README.md) - to see if the new name fits. - -- When a new namespace is necessary consider whether it should be a top-level - namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). - -- Semantic conventions exist for four areas: for Resource, Span and Log - attribute names as well as Metric label keys. In addition, for spans we have - two more areas: Event and Link attribute names. Identical namespaces or names - in all these areas MUST have identical meanings. For example the `http.method` - span attribute name denotes exactly the same concept as the `http.method` - metric label, has the same data type and the same set of possible values (in - both cases it records the value of the HTTP protocol's request method as a - string). - -- Semantic conventions MUST limit names to printable Basic Latin characters - (more precisely to - [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) - subset of Unicode code points). It is recommended to further limit names to - the following Unicode code points: Latin alphabet, Numeric, Underscore, Dot - (as namespace delimiter). - -### Recommendations for Application Developers - -As an application developer when you need to record an attribute or a label -first consult existing semantic conventions for -[Resources](../resource/semantic_conventions/README.md), -[Spans](../trace/semantic_conventions/README.md), -and -[Metrics](../metrics/semantic_conventions/README.md). -If an appropriate name does not exists you will need to come up with a new name. -To do that consider a few options: - -- The name is specific to your company and may be possibly used outside the - company as well. To avoid clashes with names introduced by other companies (in - a distributed system that uses applications from multiple vendors) it is - recommended to prefix the new name by your company's reverse domain name, e.g. - `com.acme.shopname`. - -- The name is specific to your application that will be used internally only. If - you already have an internal company process that helps you to ensure no name - clashes happen then feel free to follow it. Otherwise it is recommended to - prefix the attribute name or label key by your application name, provided that - the application name is reasonably unique within your organization (e.g. - `myuniquemapapp.longitude` is likely fine). Make sure the application name - does not clash with an existing semantic convention namespace. - -- The name may be generally applicable to applications in the industry. In that - case consider submitting a proposal to this specification to add a new name to - the semantic conventions, and if necessary also to add a new namespace. - -It is recommended to limit names to printable Basic Latin characters -(more precisely to -[U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) -subset of Unicode code points). +See [Attribute and Label Naming](attribute-and-label-naming.md) for naming guidelines. From 7a3f37d933250f54fed68a7c61563db7832bdc68 Mon Sep 17 00:00:00 2001 From: Ted Young Date: Thu, 4 Feb 2021 06:06:46 -0800 Subject: [PATCH 017/124] Add lifecycle statuses to all documents (#1385) --- specification/common/attribute-and-label-naming.md | 2 ++ specification/common/common.md | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/specification/common/attribute-and-label-naming.md b/specification/common/attribute-and-label-naming.md index d84eab4c62..0a9e3356e8 100644 --- a/specification/common/attribute-and-label-naming.md +++ b/specification/common/attribute-and-label-naming.md @@ -1,5 +1,7 @@ # Attribute and Label Naming +**Status**: [Experimental](../document-status.md) +
Table of Contents diff --git a/specification/common/common.md b/specification/common/common.md index 51ce946c48..203a25b912 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -1,6 +1,6 @@ # Common specification concepts -**Status**: [Feature-freeze](../document-status.md). +**Status**: [Stable, Feature-freeze](../document-status.md)
From 71df386e08283f0b4e76d3ceae2e4f3bfdcc2cdd Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Wed, 28 Apr 2021 10:38:46 -0400 Subject: [PATCH 018/124] Clarify usage of "otel." attribute namespace (#1640) I noticed developers adding their own attributes to this namespace without going through the specification. We need to regulate this namespace through the specification, just like we do it for other semantic conventions. --- specification/common/attribute-and-label-naming.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/specification/common/attribute-and-label-naming.md b/specification/common/attribute-and-label-naming.md index 0a9e3356e8..6ebe80ed2f 100644 --- a/specification/common/attribute-and-label-naming.md +++ b/specification/common/attribute-and-label-naming.md @@ -130,3 +130,17 @@ It is recommended to limit names to printable Basic Latin characters (more precisely to [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) subset of Unicode code points). + +## otel.* Namespace + +Attribute and label names that start with `otel.` are reserved to be defined by +OpenTelemetry specification. These are typically used to express OpenTelemetry +concepts in formats that don't have a corresponding concept. + +For example, the `otel.library.name` attribute is used to record the +instrumentation library name, which is an OpenTelemetry concept that is natively +represented in OTLP, but does not have an equivalent in other telemetry formats +and protocols. + +Any additions to the `otel.*` namespace MUST be approved as part of +OpenTelemetry specification. From dcf74c311addc4aedaa46d08df5f56e0b2df006b Mon Sep 17 00:00:00 2001 From: Jakub Malinowski Date: Thu, 1 Jul 2021 00:11:54 +0200 Subject: [PATCH 019/124] Rename Metrics labels to attributes (#1775) --- ...nd-label-naming.md => attribute-naming.md} | 36 +++++++++---------- specification/common/common.md | 2 +- 2 files changed, 18 insertions(+), 20 deletions(-) rename specification/common/{attribute-and-label-naming.md => attribute-naming.md} (84%) diff --git a/specification/common/attribute-and-label-naming.md b/specification/common/attribute-naming.md similarity index 84% rename from specification/common/attribute-and-label-naming.md rename to specification/common/attribute-naming.md index 6ebe80ed2f..6599f6b239 100644 --- a/specification/common/attribute-and-label-naming.md +++ b/specification/common/attribute-naming.md @@ -1,4 +1,4 @@ -# Attribute and Label Naming +# Attribute Naming **Status**: [Experimental](../document-status.md) @@ -13,10 +13,9 @@ Table of Contents
-_This section applies to Resource, Span and Log attribute names (also known as -the "attribute keys") and to keys of Metric labels. For brevity within this -section when we use the term "name" without an adjective it is implied to mean -"attribute name or metric label key"._ +_This section applies to Resource, Span, Log, and Metric attribute names (also +known as the "attribute keys"). For brevity within this section when we use the +term "name" without an adjective it is implied to mean "attribute name"._ Every name MUST be a valid Unicode sequence. @@ -50,8 +49,8 @@ Names SHOULD follow these rules: namespace. Because of this rule be careful when choosing names: every existing name prohibits existence of an equally named namespace in the future, and vice versa: any existing namespace prohibits existence of an equally named - attribute or label key in the future. - + attribute key in the future. + ## Name Pluralization guidelines - When an attribute represents a single entity, the attribute name SHOULD be singular. @@ -81,14 +80,13 @@ Names SHOULD follow these rules: - When a new namespace is necessary consider whether it should be a top-level namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). -- Semantic conventions exist for four areas: for Resource, Span and Log - attribute names as well as Metric label keys. In addition, for spans we have - two more areas: Event and Link attribute names. Identical namespaces or names - in all these areas MUST have identical meanings. For example the `http.method` - span attribute name denotes exactly the same concept as the `http.method` - metric label, has the same data type and the same set of possible values (in - both cases it records the value of the HTTP protocol's request method as a - string). +- Semantic conventions exist for four areas: for Resource, Span, Log, and Metric + attribute names. In addition, for spans we have two more areas: Event and Link + attribute names. Identical namespaces or names in all these areas MUST have + identical meanings. For example the `http.method` span attribute name denotes + exactly the same concept as the `http.method` metric attribute, has the same + data type and the same set of possible values (in both cases it records the + value of the HTTP protocol's request method as a string). - Semantic conventions MUST limit names to printable Basic Latin characters (more precisely to @@ -99,8 +97,8 @@ Names SHOULD follow these rules: ## Recommendations for Application Developers -As an application developer when you need to record an attribute or a label -first consult existing semantic conventions for +As an application developer when you need to record an attribute first consult +existing semantic conventions for [Resources](../resource/semantic_conventions/README.md), [Spans](../trace/semantic_conventions/README.md), and @@ -117,7 +115,7 @@ To do that consider a few options: - The name is specific to your application that will be used internally only. If you already have an internal company process that helps you to ensure no name clashes happen then feel free to follow it. Otherwise it is recommended to - prefix the attribute name or label key by your application name, provided that + prefix the attribute name by your application name, provided that the application name is reasonably unique within your organization (e.g. `myuniquemapapp.longitude` is likely fine). Make sure the application name does not clash with an existing semantic convention namespace. @@ -133,7 +131,7 @@ subset of Unicode code points). ## otel.* Namespace -Attribute and label names that start with `otel.` are reserved to be defined by +Attribute names that start with `otel.` are reserved to be defined by OpenTelemetry specification. These are typically used to express OpenTelemetry concepts in formats that don't have a corresponding concept. diff --git a/specification/common/common.md b/specification/common/common.md index 203a25b912..1b34c4f9f8 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -40,4 +40,4 @@ indices that are kept in sync (e.g., two attributes `header_keys` and `header_va both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). -See [Attribute and Label Naming](attribute-and-label-naming.md) for naming guidelines. +See [Attribute Naming](attribute-naming.md) for naming guidelines. From 2ecdb1f1a2a195442368d56f74cbb05c143e28e9 Mon Sep 17 00:00:00 2001 From: Jakub Malinowski Date: Wed, 4 Aug 2021 00:26:10 +0200 Subject: [PATCH 020/124] Add an option to limit length of values of attributes and metric values (#1130) --- specification/common/common.md | 51 ++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index 1b34c4f9f8..6cb26c12f9 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -8,6 +8,8 @@ Table of Contents - [Attributes](#attributes) + - [Attribute Limits](#attribute-limits) + - [Exempt Entities](#exempt-entities)
@@ -41,3 +43,52 @@ both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). See [Attribute Naming](attribute-naming.md) for naming guidelines. + +### Attribute Limits + +Execution of erroneous code can result in unintended attributes. If there are no +limits placed on attributes, they can quickly exhaust available memory, resulting +in crashes that are difficult to recover from safely. + +By default an SDK SHOULD apply truncation as per the list of +[configurable parameters](#attribute-limits-configuration) below. + +If an SDK provides a way to: + +- set an attribute value length limit such that for each + attribute value: + - if it is a string, if it exceeds that limit (counting any character in it as + 1), SDKs MUST truncate that value, so that its length is at most equal + to the limit, + - if it is an array of strings, then apply the above rule to each of the + values separately, + - otherwise a value MUST NOT be truncated; +- set a limit of unique attribute keys such that: + - for each unique attributes key, addition of which would result in exceeding + the limit, SDK MUST discard that key/value pair. + +There MAY be a log emitted to indicate to the user that an attribute was +truncated or discarded. To prevent excessive logging, the log MUST NOT be +emitted more than once per record on which an attribute is set. + +If the SDK implements the limits above, it MUST provide a way to change these +limits programmatically. Names of the configuration options SHOULD be the same as +in the list below. + +An SDK MAY implement model-specific limits, for example +`SpanAttributeCountLimit`. If both a general and a model-specific limit are +implemented, then the SDK MUST first attempt to use the model-specific limit, if +it isn't set and doesn't have a default, then the SDK MUST attempt to use the +general limit. + + +**Configurable parameters:** + +* `AttributeCountLimit` (Default=128) - Maximum allowed attribute count per record; +* `AttributeValueLengthLimit` (Default=Infinity) - Maximum allowed attribute value length; + +#### Exempt Entities + +Attributes, which belong to Metrics, are exempt from the limits described above +at this time, as discussed in +[Metrics Attribute Limits](../metrics/sdk.md#attribute-limits). From 8c6e0dd69c8c6f55b4f47d18c27923274e2fff74 Mon Sep 17 00:00:00 2001 From: Owais Lone Date: Tue, 14 Sep 2021 16:40:45 +0530 Subject: [PATCH 021/124] Prefer global limit over model-specific default (#1893) --- specification/common/common.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 6cb26c12f9..c3f434aa89 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -78,8 +78,9 @@ in the list below. An SDK MAY implement model-specific limits, for example `SpanAttributeCountLimit`. If both a general and a model-specific limit are implemented, then the SDK MUST first attempt to use the model-specific limit, if -it isn't set and doesn't have a default, then the SDK MUST attempt to use the -general limit. +it isn't set, then the SDK MUST attempt to use the general limit. If neither are +defined, then the SDK MUST try to use the model-specific limit default value, +followed by the global limit default value. **Configurable parameters:** From 8d35d0c0fd8959db009b51beeccff763a6ae40f1 Mon Sep 17 00:00:00 2001 From: Owais Lone Date: Thu, 23 Sep 2021 05:19:00 +0530 Subject: [PATCH 022/124] Exempt resources from attribute limits (#1892) Resources are not susceptible to scenarios where excessive attributes can be recorded unlike Spans. Resources are also immutable and it can be hard for some SDKs to apply the limits at source at the time the attributes are added to a resource. Furthermore, limits and Resources both are generally defined and passed on to a TracerProvider which forces a TracerProvider to either mutate the resource or generate a new one with duplicate attributes in order to apply the limits to it. Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> --- specification/common/common.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index c3f434aa89..b0779ad652 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -90,6 +90,14 @@ followed by the global limit default value. #### Exempt Entities +Resource attributes SHOULD be exempt from the limits described above as resources +are not susceptible to the scenarios (auto-instrumentation) that result in +excessive attributes count or size. Resources are also sent only once per batch +instead of per span so it is relatively cheaper to have more/larger attributes +on them. Resources are also immutable by design and they are generally passed +down to TracerProvider along with limits. This makes it awkward to implement +attribute limits for Resources. + Attributes, which belong to Metrics, are exempt from the limits described above at this time, as discussed in [Metrics Attribute Limits](../metrics/sdk.md#attribute-limits). From 5c85918533f6cdd84cc4d6a67ee1f3fc8e498254 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Tue, 16 Nov 2021 10:05:01 -0500 Subject: [PATCH 023/124] Provide a normative definition of Attribute (singular) rather than Attributes (plural) (#2123) * Provide a normative definition of Attribute (singular) - Providing a normative definition of **attribute** (singular) - Other copyedits /cc @austinlparker @tedsuo * Move anchor to make markdown checker happy --- specification/common/common.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index b0779ad652..19d3d87eb7 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -7,21 +7,23 @@ Table of Contents -- [Attributes](#attributes) +- [Attribute](#attribute) - [Attribute Limits](#attribute-limits) - [Exempt Entities](#exempt-entities) -## Attributes +## Attribute -Attributes are a list of zero or more key-value pairs. An `Attribute` MUST have the following properties: + -- The attribute key, which MUST be a non-`null` and non-empty string. -- The attribute value, which is either: +An `Attribute` is a key-value pair, which MUST have the following properties: + +- The attribute key MUST be a non-`null` and non-empty string. +- The attribute value is either: - A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer. - An array of primitive type values. The array MUST be homogeneous, - i.e. it MUST NOT contain values of different types. For protocols that do + i.e., it MUST NOT contain values of different types. For protocols that do not natively support array values such values SHOULD be represented as JSON strings. Attribute values expressing a numerical value of zero, an empty string, or an @@ -64,7 +66,7 @@ If an SDK provides a way to: values separately, - otherwise a value MUST NOT be truncated; - set a limit of unique attribute keys such that: - - for each unique attributes key, addition of which would result in exceeding + - for each unique attribute key, addition of which would result in exceeding the limit, SDK MUST discard that key/value pair. There MAY be a log emitted to indicate to the user that an attribute was From 537b07fcd61d8b15f55e2c1dcf3b713aae653126 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Wed, 24 Nov 2021 09:57:38 -0500 Subject: [PATCH 024/124] Ensure all ToCs are generated using markdown-toc (#2146) --- specification/common/attribute-naming.md | 11 +++++++---- specification/common/common.md | 12 +++++++----- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index 6599f6b239..1cf593b9f7 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -3,13 +3,16 @@ **Status**: [Experimental](../document-status.md)
- -Table of Contents - +Table of Contents -- [Name Pluralization Guidelines](#name-pluralization-guidelines) + + +- [Name Pluralization guidelines](#name-pluralization-guidelines) - [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) - [Recommendations for Application Developers](#recommendations-for-application-developers) +- [otel.* Namespace](#otel-namespace) + +
diff --git a/specification/common/common.md b/specification/common/common.md index 19d3d87eb7..3e5f170456 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -3,13 +3,15 @@ **Status**: [Stable, Feature-freeze](../document-status.md)
- -Table of Contents - +Table of Contents + + - [Attribute](#attribute) - - [Attribute Limits](#attribute-limits) - - [Exempt Entities](#exempt-entities) + * [Attribute Limits](#attribute-limits) + + [Exempt Entities](#exempt-entities) + +
From 14280a41ae0c750bcb710f5004cd7fbf4671a7d3 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Mon, 13 Dec 2021 13:04:41 -0500 Subject: [PATCH 025/124] Prohibit usage of retired names in semantic conventions (#2191) * Prohibit usage of retired names in semantic conventions This change adds a prohibition clause that requires that no old metric or attribute name is used for a new attribute. This is important to ensure reversibility of schema transformation (converting from a new version to an old version of schema). Without this restriction the following is possible: Schema version 1. Attribute A exists. Schema version 2. Attribute A is renamed to B. Appropriate schema file is created. Schema version 3. Attribute A is introduced (a completely different new attribute). Now attempting to go from Version 3 to version 1 is impossible since it requires renaming B to A (for the change in version 2), but a different attribute A already exists. * Fix based on comments * Add changelog entry Co-authored-by: Carlos Alberto Cortez --- specification/common/attribute-naming.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index 1cf593b9f7..9cd2f187b5 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -8,6 +8,7 @@ - [Name Pluralization guidelines](#name-pluralization-guidelines) +- [Name Reuse Prohibition](#name-reuse-prohibition) - [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) - [Recommendations for Application Developers](#recommendations-for-application-developers) - [otel.* Namespace](#otel-namespace) @@ -67,6 +68,15 @@ Names SHOULD follow these rules: [Metric Name Pluralization Guidelines](../metrics/semantic_conventions/README.md#pluralization) SHOULD be followed for the attribute name. +## Name Reuse Prohibition + +A new attribute MUST NOT be added with the same name as an attribute that +existed in the past but was renamed (with a corresponding schema file). + +When introducing a new attribute name check all existing schema files to make +sure the name does not appear as a key of any "rename_attributes" section (keys +denote old attribute names in rename operations). + ## Recommendations for OpenTelemetry Authors - All names that are part of OpenTelemetry semantic conventions SHOULD be part From a2667a2dec60265f1199e000b90b6a6614ed0fec Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Wed, 26 Jan 2022 16:47:43 -0500 Subject: [PATCH 026/124] Clarify that attribute keys are unique in collections (#2248) Attributes keys must be unique. The key/value pair collections in the specification was always intended to model a map. There was a recent confusion about this. This change clarifies the spec. Resolves https://github.com/open-telemetry/opentelemetry-specification/issues/2245 --- specification/common/common.md | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index 3e5f170456..f098255a22 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -10,6 +10,7 @@ - [Attribute](#attribute) * [Attribute Limits](#attribute-limits) + [Exempt Entities](#exempt-entities) +- [Attribute Collections](#attribute-collections) @@ -105,3 +106,35 @@ attribute limits for Resources. Attributes, which belong to Metrics, are exempt from the limits described above at this time, as discussed in [Metrics Attribute Limits](../metrics/sdk.md#attribute-limits). + +## Attribute Collections + +[Resources](../resource/sdk.md), Metrics +[data points](../metrics/datamodel.md#metric-points), +[Spans](../trace/api.md#set-attributes), Span +[Events](../trace/api.md#add-events), Span +[Links](../trace/api.md#specifying-links) and +[Log Records](../logs/data-model.md) may contain a collection of attributes. The +keys in each such collection are unique, i.e. there MUST NOT exist more than one +key-value pair with the same key. The enforcement of uniqueness may be performed +in a variety of ways as it best fits the limitations of the particular +implementation. + +Normally for the telemetry generated using OpenTelemetry SDKs the attribute +key-value pairs are set via an API that either accepts a single key-value pair +or a collection of key-value pairs. Setting an attribute with the same key as an +existing attribute SHOULD overwrite the existing attribute's value. See for +example Span's [SetAttribute](../trace/api.md#set-attributes) API. + +A typical implementation of [SetAttribute](../trace/api.md#set-attributes) API +will enforce the uniqueness by overwriting any existing attribute values pending +to be exported, so that when the Span is eventually exported the exporters see +only unique attributes. The OTLP format in particular requires that exported +Resources, Spans, Metric data points and Log Records contain only unique +attributes. + +Some other implementations may use a streaming approach where every +[SetAttribute](../trace/api.md#set-attributes) API call immediately results in +that individual attribute value being exported using a streaming wire protocol. +In such cases the enforcement of uniqueness will likely be the responsibility of +the recipient of this data. From 16376773274bdcfd735825f1420eac1f059800eb Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Tue, 15 Feb 2022 11:34:11 -0500 Subject: [PATCH 027/124] Clarify that Trace/Meter are associated with Instrumentation Scope (#2276) * Clarify that Trace/Meter are associated with Instrumentation Scope This is a slightly different take on https://github.com/open-telemetry/opentelemetry-specification/issues/2203 Instead of renaming instrumentation library to instrumentation scope everywhere this PR suggests targetted editing of wording of the Trace/Meter obtaining API to allow not just instrumentation library but other instrumentation scopes to be used as a parameter. This change does not force renaming of API parameters and is not a breaking change. We consider it a clarification of usage to match the intent that we had for the "name" field. If this PR is accepted there will be a follow up PR that will suggest to rename InstrumentationLibrary* messages in OTLP proto to InstrumentationScope* message. Such a change will not be a breaking change for the OTLP wire format and is acceptable. If this PR is accepted we will also close https://github.com/open-telemetry/opentelemetry-specification/pull/1236 since it will be no longer necessary. The logger name will be recorded as the instrumentation scope. This clarification will be a follow up PR that clarifies the behavior in https://github.com/open-telemetry/oteps/blob/main/text/logs/0150-logging-library-sdk.md --- specification/common/attribute-naming.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index 9cd2f187b5..fecbf3b21b 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -148,8 +148,8 @@ Attribute names that start with `otel.` are reserved to be defined by OpenTelemetry specification. These are typically used to express OpenTelemetry concepts in formats that don't have a corresponding concept. -For example, the `otel.library.name` attribute is used to record the -instrumentation library name, which is an OpenTelemetry concept that is natively +For example, the `otel.scope.name` attribute is used to record the +instrumentation scope name, which is an OpenTelemetry concept that is natively represented in OTLP, but does not have an equivalent in other telemetry formats and protocols. From 5bf1bbed5b066606c17075ff13e82b56c63a496a Mon Sep 17 00:00:00 2001 From: David Ashpole Date: Thu, 3 Mar 2022 00:30:53 -0500 Subject: [PATCH 028/124] Describe how to convert non-string primitives for protocols which only support strings (#2343) * Describe how to handle converting non-string primitives for protocols that only support strings * update wording to make clear that only non-string values are converted to strings * unify language * Update specification/common/common.md Co-authored-by: Joshua MacDonald Co-authored-by: Joshua MacDonald Co-authored-by: Bogdan Drutu --- specification/common/common.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index f098255a22..80a13012f4 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -26,8 +26,9 @@ An `Attribute` is a key-value pair, which MUST have the following properties: - The attribute value is either: - A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer. - An array of primitive type values. The array MUST be homogeneous, - i.e., it MUST NOT contain values of different types. For protocols that do - not natively support array values such values SHOULD be represented as JSON strings. + i.e., it MUST NOT contain values of different types. + +For protocols that do not natively support non-string values, non-string values SHOULD be represented as JSON-encoded strings. For example, the expression `int64(100)` will be encoded as `100`, `float64(1.5)` will be encoded as `1.5`, and an empty array of any type will be encoded as `[]`. Attribute values expressing a numerical value of zero, an empty string, or an empty array are considered meaningful and MUST be stored and passed on to From d1296366cf9a72b8b2971d75ac48f4af4b9d80d2 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Mon, 21 Mar 2022 23:05:45 -0700 Subject: [PATCH 029/124] Fix links (#2426) --- specification/common/common.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 80a13012f4..774c236130 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -9,6 +9,7 @@ - [Attribute](#attribute) * [Attribute Limits](#attribute-limits) + + [Configurable Parameters](#configurable-parameters) + [Exempt Entities](#exempt-entities) - [Attribute Collections](#attribute-collections) @@ -57,7 +58,7 @@ limits placed on attributes, they can quickly exhaust available memory, resultin in crashes that are difficult to recover from safely. By default an SDK SHOULD apply truncation as per the list of -[configurable parameters](#attribute-limits-configuration) below. +[configurable parameters](#configurable-parameters) below. If an SDK provides a way to: @@ -88,8 +89,7 @@ it isn't set, then the SDK MUST attempt to use the general limit. If neither are defined, then the SDK MUST try to use the model-specific limit default value, followed by the global limit default value. - -**Configurable parameters:** +#### Configurable Parameters * `AttributeCountLimit` (Default=128) - Maximum allowed attribute count per record; * `AttributeValueLengthLimit` (Default=Infinity) - Maximum allowed attribute value length; From b1abb50dd6e3bbcb6481aae093d4058a22436e9c Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Tue, 22 Mar 2022 08:59:37 -0400 Subject: [PATCH 030/124] Implement OTEP 0178: Mapping external data to AnyValue (#2385) OTEP 0178 [0] proposed this mapping. This PR merges the proposal into specification. The content of this PR is mostly copy/paste of the text of the OTEP minus the irrelevant sections such as "Alternates", etc. 0 - https://github.com/open-telemetry/oteps/blob/main/text/0178-mapping-to-otlp-anyvalue.md --- .../common/attribute-type-mapping.md | 255 ++++++++++++++++++ specification/common/common.md | 3 + 2 files changed, 258 insertions(+) create mode 100644 specification/common/attribute-type-mapping.md diff --git a/specification/common/attribute-type-mapping.md b/specification/common/attribute-type-mapping.md new file mode 100644 index 0000000000..46a25c1504 --- /dev/null +++ b/specification/common/attribute-type-mapping.md @@ -0,0 +1,255 @@ +# Mapping Arbitrary Data to OTLP AnyValue + +**Status**: [Experimental](../document-status.md) + +
+Table of Contents + + + +- [Converting to AnyValue](#converting-to-anyvalue) + * [Primitive Values](#primitive-values) + + [Integer Values](#integer-values) + + [Enumerations](#enumerations) + + [Floating Point Values](#floating-point-values) + + [String Values](#string-values) + + [Byte Sequences](#byte-sequences) + * [Composite Values](#composite-values) + + [Array Values](#array-values) + + [Associative Arrays With Unique Keys](#associative-arrays-with-unique-keys) + + [Associative Arrays With Non-Unique Keys](#associative-arrays-with-non-unique-keys) + + [Sets](#sets) + * [Other Values](#other-values) + * [Empty Values](#empty-values) + + + +
+ +This document defines how to map (convert) arbitrary data (e.g. in-memory +objects) to OTLP's [AnyValue](https://github.com/open-telemetry/opentelemetry-proto/blob/cc4ed55c082cb75e084d40b4ddf3805eda099f97/opentelemetry/proto/common/v1/common.proto#L27). + +The mapping is needed when OpenTelemetry needs to convert a value produced outside +OpenTelemetry into a value that can be exported using an OTLP exporter, or otherwise be +converted to be used inside OpenTelemetry boundaries. Example use cases are the following: + +- In [Logging Library SDK](../logs/logging-library-sdk.md)s, to convert values received + from logging libraries into OpenTelemetry representation. +- In the Collector, to convert values received from various data sources into + [pdata](https://github.com/open-telemetry/opentelemetry-collector/blob/4998703dadd19fa91a88eabf7ccc68d728bee3fd/model/pdata/common.go#L84) + internal representation. + +## Converting to AnyValue + +[AnyValue](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L27) +is capable of representing primitive and structured data of certain types. + +Implementations that have source data in any form, such as in-memory objects +or data coming from other formats that needs to be converted to AnyValue SHOULD +follow the rules described below. + +### Primitive Values + +#### Integer Values + +Integer values which are within the range of 64 bit signed numbers +[-2^63..2^63-1] SHOULD be converted to AnyValue's +[int_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L33) +field. + +Integer values which are outside the range of 64 bit signed numbers SHOULD be +converted to AnyValue's +[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) +field using decimal representation. + +#### Enumerations + +Values, which belong to a limited enumerated set (e.g. a Java +[enum](https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html)), SHOULD be +converted to AnyValue's +[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) +field with the value of the string set to the symbolic name of the enumeration. + +If it is not possible to obtain the symbolic name of the enumeration, the +implementation SHOULD map enumeration values to AnyValue's +[int_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L33) +field set to the enum's ordinal value, when such an ordinal number is naturally +obtainable. + +If it is also not possible to obtain the ordinal value, the enumeration SHOULD be +converted to AnyValue's +[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) +field in any manner that the implementation deems reasonable. + +#### Floating Point Values + +Floating point values which are within the range and precision of IEEE 754 +64-bit floating point numbers (including IEEE 32-bit floating point values) +SHOULD be converted to AnyValue's +[double_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L34) +field. + +Floating point values which are outside the range or precision of IEEE 754 +64-bit floating point numbers (e.g. IEEE 128-bit floating bit values) SHOULD be +converted to AnyValue's +[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) +field using decimal floating point representation. + +#### String Values + +String values which are valid UTF-8 sequences SHOULD be converted to +AnyValue's +[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) +field. + +String values which are not valid Unicode sequences SHOULD be converted to +AnyValue's +[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) +with the bytes representing the string in the original order and format of the +source string. + +#### Byte Sequences + +Byte sequences (e.g. Go's `[]byte` slice or raw byte content of a file) SHOULD +be converted to AnyValue's +[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) +field. + +### Composite Values + +#### Array Values + +Values that represent ordered sequences of other values (such as +[arrays](https://docs.oracle.com/javase/specs/jls/se7/html/jls-10.html), +[vectors](https://en.cppreference.com/w/cpp/container/vector), ordered +[lists](https://docs.python.org/3/tutorial/datastructures.html#more-on-lists), +[slices](https://golang.org/ref/spec#Slice_types)) SHOULD be converted to +AnyValue's +[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) +field. String values and byte sequences are an exception from this rule (see +above). + +The rules described in this document should be applied recursively to each element +of the array. + +#### Associative Arrays With Unique Keys + +Values that represent associative arrays with unique keys (also often known +as maps, dictionaries or key-value stores) SHOULD be converted to AnyValue's +[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) +field. + +If the keys of the source array are not strings, they MUST be converted to +strings by any means available, often via a toString() or stringify functions +available in programming languages. The conversion function MUST be chosen in a +way that ensures that the resulting string keys are unique in the target array. + +The value part of each element of the source array SHOULD be converted to +AnyValue recursively. + +For example a JSON object `{"a": 123, "b": "def"}` SHOULD be converted to + +``` +AnyValue{ + kvlist_value:KeyValueList{ + values:[ + KeyValue{key:"a",value:AnyValue{int_value:123}}, + KeyValue{key:"b",value:AnyValue{string_value:"def"}}, + ] + } +} +``` + +The rules described in this document should be applied recursively to each value +of the associative array. + +#### Associative Arrays With Non-Unique Keys + +Values that represent an associative arrays with non-unique keys where multiple values may be associated with the same key (also sometimes known +as multimaps, multidicts) SHOULD be converted to AnyValue's +[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) +field. + +The resulting +[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) +field MUST list each key only once and the value of each element of +[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) +field MUST be an array represented using AnyValue's +[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) +field, each element of the +[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) +representing one value of source array associated with the given key. + +For example an associative array shown in the following table: + +|Key|Value| +|---|---| +|"abc"|123| +|"def"|"foo"| +|"def"|"bar"| + +SHOULD be converted to: + +``` +AnyValue{ + kvlist_value:KeyValueList{ + values:[ + KeyValue{ + key:"abc", + value:AnyValue{array_value:ArrayValue{values[ + AnyValue{int_value:123} + ]}} + }, + KeyValue{ + key:"def", + value:AnyValue{array_value:ArrayValue{values[ + AnyValue{string_value:"foo"}, + AnyValue{string_value:"bar"} + ]}} + }, + ] + } +} +``` + +The rules described in this document should be applied recursively to each value +of the associative array. + +#### Sets + +Unordered collections of unique values (such as +[Java Sets](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Set.html), +[C++ sets](https://en.cppreference.com/w/cpp/container/set), +[Python Sets](https://docs.python.org/3/tutorial/datastructures.html#sets)) SHOULD be +converted to AnyValue's +[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) +field, where each element of the set becomes an element of the array. + +The rules described in this document should be applied recursively to each value +of the set. + +### Other Values + +Any other values not listed above SHOULD be converted to AnyValue's +[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) +field if the source data can be serialized to a string (can be stringified) +using toString() or stringify functions available in programming languages. + +If the source data cannot be serialized to a string then the value SHOULD be +converted AnyValue's +[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) +field by serializing it into a byte sequence by any means available. + +If the source data cannot be serialized neither to a string nor to a byte +sequence then it SHOULD by converted to an empty AnyValue. + +### Empty Values + +If the source data has no type associated with it and is empty, null, nil or +otherwise indicates absence of data it SHOULD be converted to an +[empty](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L29) +AnyValue, where all the fields are unset. + +Empty values which has a type associated with them (e.g. empty associative +array) SHOULD be converted using the corresponding rules defined for the types +above. diff --git a/specification/common/common.md b/specification/common/common.md index 774c236130..6c3e80a527 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -51,6 +51,9 @@ both containing an array of strings to represent a mapping See [Attribute Naming](attribute-naming.md) for naming guidelines. +See [this document](attribute-type-mapping.md) to find out how to map values obtained +outside OpenTelemetry into OpenTelemetry attribute values. + ### Attribute Limits Execution of erroneous code can result in unintended attributes. If there are no From 7a55bf2c53c03c9832d866e3270373bd00c8b9dd Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Fri, 8 Apr 2022 13:07:21 -0400 Subject: [PATCH 031/124] Ensure common section has a README (#2479) --- specification/common/{common.md => README.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename specification/common/{common.md => README.md} (100%) diff --git a/specification/common/common.md b/specification/common/README.md similarity index 100% rename from specification/common/common.md rename to specification/common/README.md From b8926d2db99cbc1c9345fa35166db447912138fb Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 16 May 2022 10:16:33 -0700 Subject: [PATCH 032/124] Define attribute requirement levels (#2522) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * nits * review comments * No exceptions for Required attributes, clarifications on performance * Conditional clarifications * nits * Conditional -> required conditionally and minor fixes * Align requirement levels with RFC levels better * Clarify Note on required attributes * Update specification/common/attribute-requirement-level.md Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> * Clarify Note on required attributes * Remove performance from conditionally required attributes * Clarify Conditionally Required case when condition is false * Apply suggestions from code review Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Christian Neumüller * headings for levels and moving things around * nits: formatting Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Co-authored-by: Christian Neumüller Co-authored-by: Reiley Yang --- specification/common/README.md | 2 + .../common/attribute-requirement-level.md | 69 +++++++++++++++++++ 2 files changed, 71 insertions(+) create mode 100644 specification/common/attribute-requirement-level.md diff --git a/specification/common/README.md b/specification/common/README.md index 6c3e80a527..18d46fe00c 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -51,6 +51,8 @@ both containing an array of strings to represent a mapping See [Attribute Naming](attribute-naming.md) for naming guidelines. +See [Requirement Level](attribute-requirement-level.md) for requirement levels guidelines. + See [this document](attribute-type-mapping.md) to find out how to map values obtained outside OpenTelemetry into OpenTelemetry attribute values. diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md new file mode 100644 index 0000000000..5a7d6e323d --- /dev/null +++ b/specification/common/attribute-requirement-level.md @@ -0,0 +1,69 @@ +# Attribute Requirement Levels for Semantic Conventions + +**Status**: [Experimental](../document-status.md) + +
+Table of Contents + + + +- [Required](#required) +- [Conditionally Required](#conditionally-required) +- [Recommended](#recommended) +- [Optional](#optional) +- [Performance suggestions](#performance-suggestions) + + + +
+ +_This section applies to Log, Metric, Resource, and Span, and describes requirement levels for attributes defined in semantic conventions._ + +The following attribute requirement levels are specified: + +- [Required](#required) +- [Conditionally Required](#conditionally-required) +- [Recommended](#recommended) +- [Optional](#optional) + +The requirement level for attribute is defined by semantic conventions depending on attribute availability across instrumented entities, performance, security, and other factors. When defining requirement levels, semantic conventions MUST take into account signal-specific requirements. + +For example, Metric attributes that may have high cardinality can only be defined with `Optional` level. + +Semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. + +For example, [Database semantic convention](../trace/semantic_conventions/database.md) references `net.transport` attribute defined in [General attributes](../trace/semantic_conventions/span-general.md) with `Conditionally Required` level on it. + +## Required + +All instrumentations MUST populate the attribute. Semantic convention defining a Required attribute expects that an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, can ensure cardinality, security, and other requirements specific to signal defined by the convention. `http.method` is an example of a Required attribute. + +_Note: Consumers of telemetry can detect if telemetry item follows a specific semantic convention by checking the presence of a `Required` attribute defined by such convention. For example, the presence of `db.system` attribute on a span can be used as an indication that the span follows database semantics._ + +## Conditionally Required + +All instrumentations MUST add the attribute when given condition is satisfied. Semantic convention of a `Conditionally Required` level of an attribute MUST clarify the condition under which the attribute is expected to be populated. + +`http.route` is an example of a conditionally required attribute to be populated when instrumented HTTP framework provides route information for the instrumented request. Some low-level HTTP server implementations do not support routing and corresponding instrumentations can't populate the attribute. + +When the condition on `Conditionally Required` attribute is not satisfied and there is no requirement to populate attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Optional` requirement level on the attribute. + +For example, `net.peer.name` is `Conditionally Required` by [Database convention](../trace/semantic_conventions/database.md) when available. When only `net.peer.ip` is available, instrumentation can do a DNS lookup, cache and populate `net.peer.name` but only if user explicitly enables instrumentation to do so, considering performance issues the DNS lookup introduces. + +## Recommended + +Instrumentations SHOULD add the attribute by default if it's readily available and can be [efficiently populated](#performance-suggestions). Instrumentations MAY offer a configuration option to disable Recommended attributes. + +Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD use the `Optional` requirement level on them if the attributes are logically applicable. + +## Optional + +Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Optional` attributes. + +## Performance suggestions + +Here are several examples of expensive operations to be avoided by default: + +- DNS lookup to populate `net.peer.name` if only IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. +- forcing `http.route` calculation before HTTP framework calculates it +- reading response stream to find `http.response_content_length` when `Content-Length` header is not available From a1f9af0259fe6e43ea806fb318a7cfffa47343b6 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Fri, 20 May 2022 02:12:03 -0400 Subject: [PATCH 033/124] Add missing READMEs to section (#2559) --- specification/common/README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/specification/common/README.md b/specification/common/README.md index 18d46fe00c..3d1379fde4 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -1,3 +1,6 @@ + # Common specification concepts **Status**: [Stable, Feature-freeze](../document-status.md) From fb860bc6675cf3b118f7fb331c25ce6ea596ddb3 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Thu, 2 Jun 2022 03:25:38 -0400 Subject: [PATCH 034/124] Move non-otlp.md to common directory (#2587) --- specification/common/mapping-to-non-otlp.md | 76 +++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 specification/common/mapping-to-non-otlp.md diff --git a/specification/common/mapping-to-non-otlp.md b/specification/common/mapping-to-non-otlp.md new file mode 100644 index 0000000000..cb80f685ca --- /dev/null +++ b/specification/common/mapping-to-non-otlp.md @@ -0,0 +1,76 @@ +# OpenTelemetry Transformation to non-OTLP Formats + +**Status**: [Stable](../document-status.md) + +All OpenTelemetry concepts and data recorded using OpenTelemetry API can be +directly and precisely represented using corresponding messages and fields of +OTLP format. However, for other formats this is not always the case. Sometimes a +format will not have a native way to represent a particular OpenTelemetry +concept or a field of a concept. + +This document defines the transformation between OpenTelemetry and formats other +than OTLP, for OpenTelemetry fields and concepts that have no direct semantic +equivalent in those other formats. + +Note: when a format has a direct semantic equivalent for a particular field or +concept then the recommendation in this document MUST be ignored. + +See also additional specific transformation rules for +[Jaeger](../trace/sdk_exporters/jaeger.md) and [Zipkin](../trace/sdk_exporters/zipkin.md). +The specific rules for Jaeger and Zipkin take precedence over the generic rules defined +in this document. + +## Mappings + +### InstrumentationScope + +OpenTelemetry `InstrumentationScope`'s fields MUST be reported as key-value +pairs associated with the Span, Metric Data Point or LogRecord using the following mapping: + +| OpenTelemetry InstrumentationScope Field | non-OTLP Key | Notes | +| ------------------- | --- | --- | +| `InstrumentationScope.name`|`otel.scope.name`|since 1.10.0| +| `InstrumentationScope.version`|`otel.scope.version`|since 1.10.0| + +The following deprecated aliases MUST also be reported with exact same values for +backward compatibility reasons: + +| non-OTLP Key | Alias for | Notes | +| --- | --- | --- | +|`otel.library.name`|`otel.scope.name`|deprecated since 1.10.0| +|`otel.library.version`|`otel.scope.version`|deprecated since 1.10.0| + +### Span Status + +Span `Status` MUST be reported as key-value pairs associated with the Span, +unless the `Status` is `UNSET`. In the latter case it MUST NOT be reported. + +The following table defines the OpenTelemetry `Status`'s mapping to Span's +key-value pairs: + +|OpenTelemetry Status Field|non-OTLP Key|non-OTLP Value| +|--|--|--| +|Code | `otel.status_code` | Name of the code, either `OK` or `ERROR`. MUST NOT be set if the code is `UNSET`. | +|Description | `otel.status_description` | Description of the `Status` if it has a value otherwise not set. | + +### Dropped Attributes Count + +OpenTelemetry dropped attributes count MUST be reported as a key-value +pair associated with the corresponding data entity (e.g. Span, Span Link, Span Event, +Metric data point, LogRecord, etc). The key name MUST be `otel.dropped_attributes_count`. + +This key-value pair should only be recorded when it contains a non-zero value. + +### Dropped Events Count + +OpenTelemetry Span's dropped events count MUST be reported as a key-value pair +associated with the Span. The key name MUST be `otel.dropped_events_count`. + +This key-value pair should only be recorded when it contains a non-zero value. + +### Dropped Links Count + +OpenTelemetry Span's dropped links count MUST be reported as a key-value pair +associated with the Span. The key name MUST be `otel.dropped_links_count`. + +This key-value pair should only be recorded when it contains a non-zero value. From b0047edbc3eff495649ebfe698b429e8a49150f4 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Fri, 10 Jun 2022 22:19:33 -0400 Subject: [PATCH 035/124] Introduce Instrumentation Scope Attributes (#2579) --- specification/common/mapping-to-non-otlp.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/specification/common/mapping-to-non-otlp.md b/specification/common/mapping-to-non-otlp.md index cb80f685ca..656b37fa35 100644 --- a/specification/common/mapping-to-non-otlp.md +++ b/specification/common/mapping-to-non-otlp.md @@ -74,3 +74,9 @@ OpenTelemetry Span's dropped links count MUST be reported as a key-value pair associated with the Span. The key name MUST be `otel.dropped_links_count`. This key-value pair should only be recorded when it contains a non-zero value. + +### Instrumentation Scope Attributes + +Exporters to formats that don't have a concept that is equivalent to the Scope +SHOULD record the attributes at the most suitable place in their corresponding format, +typically at the Span, Metric or LogRecord equivalent. From aafd53c1294122b91307d5f7b1ed5b0a5dc29493 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Mon, 13 Jun 2022 12:00:00 -0400 Subject: [PATCH 036/124] Use consistent file name for data-model.md (#2586) --- specification/common/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/README.md b/specification/common/README.md index 3d1379fde4..35d6264b2a 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -119,7 +119,7 @@ at this time, as discussed in ## Attribute Collections [Resources](../resource/sdk.md), Metrics -[data points](../metrics/datamodel.md#metric-points), +[data points](../metrics/data-model.md#metric-points), [Spans](../trace/api.md#set-attributes), Span [Events](../trace/api.md#add-events), Span [Links](../trace/api.md#specifying-links) and From fe3cf1830c14860a5964ae33dafd88844cd6c435 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Fri, 8 Jul 2022 03:12:12 -0400 Subject: [PATCH 037/124] Add note to Hugo front matter (#2651) --- specification/common/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/README.md b/specification/common/README.md index 35d6264b2a..2b377a731c 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -1,4 +1,4 @@ - # Common specification concepts From a181cc323e3c2096f16d6e43c5eb4902604af3ba Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Wed, 13 Jul 2022 10:01:44 -0700 Subject: [PATCH 038/124] Define net.sock attributes and clarify logical net.peer|host.name attributes (#2614) --- specification/common/attribute-requirement-level.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index 5a7d6e323d..e015ef89eb 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -48,7 +48,7 @@ All instrumentations MUST add the attribute when given condition is satisfied. S When the condition on `Conditionally Required` attribute is not satisfied and there is no requirement to populate attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Optional` requirement level on the attribute. -For example, `net.peer.name` is `Conditionally Required` by [Database convention](../trace/semantic_conventions/database.md) when available. When only `net.peer.ip` is available, instrumentation can do a DNS lookup, cache and populate `net.peer.name` but only if user explicitly enables instrumentation to do so, considering performance issues the DNS lookup introduces. +For example, `net.peer.name` is `Conditionally Required` by [Database convention](../trace/semantic_conventions/database.md) when available. When only `net.sock.peer.addr` is available, instrumentation can do a DNS lookup, cache and populate `net.sock.peer.name` but only if user explicitly enables instrumentation to do so, considering performance issues the DNS lookup introduces. ## Recommended From 8267eacc84e712632fb75f798c5f8e38c234b581 Mon Sep 17 00:00:00 2001 From: jack-berg <34418638+jack-berg@users.noreply.github.com> Date: Tue, 13 Sep 2022 09:45:58 -0500 Subject: [PATCH 039/124] Align log sdk naming with api (#2768) Resolves #2752. This aligns log SDK and API concepts which have diverged after the merged of #2676. This PR brings alignment to the log API and SDK, and in brings the log signal into alignment with tracing and metrics where there is conceptual overlap. There shouldn't be any new concepts introduced here. - Rename `../logs/logging-library-sdk.md` to `../logs/sdk.md` - Remove wording from SDK that implies that an API doesn't exist, like [this](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/logging-library-sdk.md#L60-L62). - Move [How to Create Log4j Style Appender](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/logging-library-sdk.md#L219) to `api.md` since it describes an API use case. - Move [Implicit / Explicit Context Injection](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/logging-library-sdk.md#L270-L288) sections to `api.md` since they describe API level considerations. - Rename Logger [create](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/api.md#L133) method to be emit, to align with SDK concept of `LogRecordProcessor#onEmit(..)`. - Rename `LogProcessor`, `LogExporter` to `LogRecordProcessor`, `LogRecordExporter`. - Fill in various SDK level TODOs related to shutdown and flushing. The language from these was taken directly from the metrics / tracing SDK - no new concepts were introduced. --- specification/common/attribute-type-mapping.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-type-mapping.md b/specification/common/attribute-type-mapping.md index 46a25c1504..532d6156bc 100644 --- a/specification/common/attribute-type-mapping.md +++ b/specification/common/attribute-type-mapping.md @@ -33,7 +33,7 @@ The mapping is needed when OpenTelemetry needs to convert a value produced outsi OpenTelemetry into a value that can be exported using an OTLP exporter, or otherwise be converted to be used inside OpenTelemetry boundaries. Example use cases are the following: -- In [Logging Library SDK](../logs/logging-library-sdk.md)s, to convert values received +- In the [Logging SDK](../logs/sdk.md)s, to convert values received from logging libraries into OpenTelemetry representation. - In the Collector, to convert values received from various data sources into [pdata](https://github.com/open-telemetry/opentelemetry-collector/blob/4998703dadd19fa91a88eabf7ccc68d728bee3fd/model/pdata/common.go#L84) From fac9ea093cfedefa37ab919f6a4750d6f4c7a5ca Mon Sep 17 00:00:00 2001 From: Bogdan Drutu Date: Tue, 25 Oct 2022 08:46:06 -0700 Subject: [PATCH 040/124] Define semantic conventions yaml for non-otlp conventions (#2850) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Define semantic conventions yaml for non-otlp conventions Signed-off-by: Bogdan Drutu * Update semantic_conventions/trace/exporter/exporter.yaml Co-authored-by: Christian Neumüller * Update semantic_conventions/scope/exporter/exporter.yaml Co-authored-by: Joao Grassi * Rename otel to otel_span Signed-off-by: Bogdan Drutu Signed-off-by: Bogdan Drutu Co-authored-by: Christian Neumüller Co-authored-by: Joao Grassi --- specification/common/mapping-to-non-otlp.md | 37 ++++++++++++++------- 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/specification/common/mapping-to-non-otlp.md b/specification/common/mapping-to-non-otlp.md index 656b37fa35..a82dac3e63 100644 --- a/specification/common/mapping-to-non-otlp.md +++ b/specification/common/mapping-to-non-otlp.md @@ -27,18 +27,22 @@ in this document. OpenTelemetry `InstrumentationScope`'s fields MUST be reported as key-value pairs associated with the Span, Metric Data Point or LogRecord using the following mapping: -| OpenTelemetry InstrumentationScope Field | non-OTLP Key | Notes | -| ------------------- | --- | --- | -| `InstrumentationScope.name`|`otel.scope.name`|since 1.10.0| -| `InstrumentationScope.version`|`otel.scope.version`|since 1.10.0| + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `otel.scope.name` | string | The name of the instrumentation scope - (`InstrumentationScope.Name` in OTLP). | `io.opentelemetry.contrib.mongodb` | Recommended | +| `otel.scope.version` | string | The version of the instrumentation scope - (`InstrumentationScope.Version` in OTLP). | `1.0.0` | Recommended | + The following deprecated aliases MUST also be reported with exact same values for backward compatibility reasons: -| non-OTLP Key | Alias for | Notes | -| --- | --- | --- | -|`otel.library.name`|`otel.scope.name`|deprecated since 1.10.0| -|`otel.library.version`|`otel.scope.version`|deprecated since 1.10.0| + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `otel.library.name` | string | Deprecated, use the `otel.scope.name` attribute. | `io.opentelemetry.contrib.mongodb` | Recommended | +| `otel.library.version` | string | Deprecated, use the `otel.scope.version` attribute. | `1.0.0` | Recommended | + ### Span Status @@ -48,10 +52,19 @@ unless the `Status` is `UNSET`. In the latter case it MUST NOT be reported. The following table defines the OpenTelemetry `Status`'s mapping to Span's key-value pairs: -|OpenTelemetry Status Field|non-OTLP Key|non-OTLP Value| -|--|--|--| -|Code | `otel.status_code` | Name of the code, either `OK` or `ERROR`. MUST NOT be set if the code is `UNSET`. | -|Description | `otel.status_description` | Description of the `Status` if it has a value otherwise not set. | + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `otel.status_code` | string | Name of the code, either "OK" or "ERROR". MUST NOT be set if the status code is UNSET. | `OK` | Recommended | +| `otel.status_description` | string | Description of the Status if it has a value, otherwise not set. | `resource not found` | Recommended | + +`otel.status_code` MUST be one of the following: + +| Value | Description | +|---|---| +| `OK` | The operation has been validated by an Application developer or Operator to have completed successfully. | +| `ERROR` | The operation contains an error. | + ### Dropped Attributes Count From 6fb1c444abb11def67c20d80d5d779f3468d6e35 Mon Sep 17 00:00:00 2001 From: Alan West <3676547+alanwest@users.noreply.github.com> Date: Thu, 26 Jan 2023 16:22:06 -0800 Subject: [PATCH 041/124] Add log attribute limit configuration (#2861) Fixes #2860 Adds log attribute limit configuration. These new environment variables bring more consistency between spans and logs. --- specification/common/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/specification/common/README.md b/specification/common/README.md index 2b377a731c..f36cb92b85 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -91,11 +91,11 @@ limits programmatically. Names of the configuration options SHOULD be the same a in the list below. An SDK MAY implement model-specific limits, for example -`SpanAttributeCountLimit`. If both a general and a model-specific limit are -implemented, then the SDK MUST first attempt to use the model-specific limit, if -it isn't set, then the SDK MUST attempt to use the general limit. If neither are -defined, then the SDK MUST try to use the model-specific limit default value, -followed by the global limit default value. +`SpanAttributeCountLimit` or `LogRecordAttributeCountLimit`. If both a general +and a model-specific limit are implemented, then the SDK MUST first attempt to +use the model-specific limit, if it isn't set, then the SDK MUST attempt to use +the general limit. If neither are defined, then the SDK MUST try to use the +model-specific limit default value, followed by the global limit default value. #### Configurable Parameters From 6b6cba749ac1adeae31476cc769f7d8684c6e7d2 Mon Sep 17 00:00:00 2001 From: Josh Suereth Date: Thu, 23 Feb 2023 11:38:55 -0500 Subject: [PATCH 042/124] Mark Attribute naming conventions as stable. (#3220) --- specification/common/attribute-naming.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index fecbf3b21b..d761b5f9fd 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -1,6 +1,6 @@ # Attribute Naming -**Status**: [Experimental](../document-status.md) +**Status**: [Stable](../document-status.md)
Table of Contents From b8e3c86a2b965590ac7c40c499cd469900ae241c Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Wed, 1 Mar 2023 15:13:24 -0800 Subject: [PATCH 043/124] Rename Optional attribute requirement level to Opt-In (#3228) --- .../common/attribute-requirement-level.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index e015ef89eb..c030159409 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -10,7 +10,7 @@ - [Required](#required) - [Conditionally Required](#conditionally-required) - [Recommended](#recommended) -- [Optional](#optional) +- [Opt-In](#opt-in) - [Performance suggestions](#performance-suggestions) @@ -24,11 +24,11 @@ The following attribute requirement levels are specified: - [Required](#required) - [Conditionally Required](#conditionally-required) - [Recommended](#recommended) -- [Optional](#optional) +- [Opt-In](#opt-in) The requirement level for attribute is defined by semantic conventions depending on attribute availability across instrumented entities, performance, security, and other factors. When defining requirement levels, semantic conventions MUST take into account signal-specific requirements. -For example, Metric attributes that may have high cardinality can only be defined with `Optional` level. +For example, Metric attributes that may have high cardinality can only be defined with `Opt-In` level. Semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. @@ -46,7 +46,7 @@ All instrumentations MUST add the attribute when given condition is satisfied. S `http.route` is an example of a conditionally required attribute to be populated when instrumented HTTP framework provides route information for the instrumented request. Some low-level HTTP server implementations do not support routing and corresponding instrumentations can't populate the attribute. -When the condition on `Conditionally Required` attribute is not satisfied and there is no requirement to populate attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Optional` requirement level on the attribute. +When the condition on `Conditionally Required` attribute is not satisfied and there is no requirement to populate attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. For example, `net.peer.name` is `Conditionally Required` by [Database convention](../trace/semantic_conventions/database.md) when available. When only `net.sock.peer.addr` is available, instrumentation can do a DNS lookup, cache and populate `net.sock.peer.name` but only if user explicitly enables instrumentation to do so, considering performance issues the DNS lookup introduces. @@ -54,11 +54,11 @@ For example, `net.peer.name` is `Conditionally Required` by [Database convention Instrumentations SHOULD add the attribute by default if it's readily available and can be [efficiently populated](#performance-suggestions). Instrumentations MAY offer a configuration option to disable Recommended attributes. -Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD use the `Optional` requirement level on them if the attributes are logically applicable. +Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD use the `Opt-In` requirement level on them if the attributes are logically applicable. -## Optional +## Opt-In -Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Optional` attributes. +Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Opt-In` attributes. ## Performance suggestions From 8b0bccd2dedd0c1f9d5f90963f1ad363d9a71216 Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Mon, 6 Mar 2023 10:20:11 -0800 Subject: [PATCH 044/124] Proofread of attribute requirement levels in preparation for stability (#3270) --- .../common/attribute-requirement-level.md | 25 +++++++++++-------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index c030159409..59432e6096 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -26,44 +26,47 @@ The following attribute requirement levels are specified: - [Recommended](#recommended) - [Opt-In](#opt-in) -The requirement level for attribute is defined by semantic conventions depending on attribute availability across instrumented entities, performance, security, and other factors. When defining requirement levels, semantic conventions MUST take into account signal-specific requirements. +The requirement level for an attribute is specified by semantic conventions depending on attribute availability across instrumented entities, performance, security, and other factors. When specifying requirement levels, a semantic convention MUST take into account signal-specific requirements. For example, Metric attributes that may have high cardinality can only be defined with `Opt-In` level. -Semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. +A semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. For example, [Database semantic convention](../trace/semantic_conventions/database.md) references `net.transport` attribute defined in [General attributes](../trace/semantic_conventions/span-general.md) with `Conditionally Required` level on it. ## Required -All instrumentations MUST populate the attribute. Semantic convention defining a Required attribute expects that an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, can ensure cardinality, security, and other requirements specific to signal defined by the convention. `http.method` is an example of a Required attribute. +All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.method` is an example of a Required attribute. -_Note: Consumers of telemetry can detect if telemetry item follows a specific semantic convention by checking the presence of a `Required` attribute defined by such convention. For example, the presence of `db.system` attribute on a span can be used as an indication that the span follows database semantics._ +_Note: Consumers of telemetry can detect if a telemetry item follows a specific semantic convention by checking for the presence of a `Required` attribute defined by such convention. For example, the presence of the `db.system` attribute on a span can be used as an indication that the span follows database semantics._ ## Conditionally Required -All instrumentations MUST add the attribute when given condition is satisfied. Semantic convention of a `Conditionally Required` level of an attribute MUST clarify the condition under which the attribute is expected to be populated. +All instrumentations MUST populate the attribute when the given condition is satisfied. The semantic convention of a `Conditionally Required` attribute MUST clarify the condition under which the attribute is to be populated. -`http.route` is an example of a conditionally required attribute to be populated when instrumented HTTP framework provides route information for the instrumented request. Some low-level HTTP server implementations do not support routing and corresponding instrumentations can't populate the attribute. +`http.route` is an example of a conditionally required attribute that is populated when the instrumented HTTP framework provides route information for the instrumented request. Some low-level HTTP server implementations do not support routing and corresponding instrumentations can't populate the attribute. -When the condition on `Conditionally Required` attribute is not satisfied and there is no requirement to populate attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. +When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. -For example, `net.peer.name` is `Conditionally Required` by [Database convention](../trace/semantic_conventions/database.md) when available. When only `net.sock.peer.addr` is available, instrumentation can do a DNS lookup, cache and populate `net.sock.peer.name` but only if user explicitly enables instrumentation to do so, considering performance issues the DNS lookup introduces. +For example, `net.peer.name` is `Conditionally Required` by the [Database convention](../trace/semantic_conventions/database.md) when available. When `net.sock.peer.addr` is available instead, instrumentation can do a DNS lookup, cache and populate `net.peer.name`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. ## Recommended Instrumentations SHOULD add the attribute by default if it's readily available and can be [efficiently populated](#performance-suggestions). Instrumentations MAY offer a configuration option to disable Recommended attributes. -Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD use the `Opt-In` requirement level on them if the attributes are logically applicable. +Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD allow for users to +opt-in to emit them as defined for the `Opt-In` requirement level (if the attributes are logically applicable). ## Opt-In Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Opt-In` attributes. +This attribute requirement level is recommended for attributes that are particularly expensive to retrieve or might pose a security or privacy risk. These should therefore only be enabled enabled explicitly by a user making an informed decision. + ## Performance suggestions Here are several examples of expensive operations to be avoided by default: -- DNS lookup to populate `net.peer.name` if only IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. -- forcing `http.route` calculation before HTTP framework calculates it +- DNS lookups to populate `net.peer.name` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. +- forcing an `http.route` calculation before the HTTP framework calculates it - reading response stream to find `http.response_content_length` when `Content-Length` header is not available From c717ab4f4cf26669bb448d5caf50c656f42f670c Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Tue, 7 Mar 2023 11:22:32 -0800 Subject: [PATCH 045/124] Attribute requirement level follow-up edits (#3293) --- specification/common/attribute-requirement-level.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index 59432e6096..c7fb99345c 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -36,7 +36,7 @@ For example, [Database semantic convention](../trace/semantic_conventions/databa ## Required -All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.method` is an example of a Required attribute. +All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, and can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.method` is an example of a Required attribute. _Note: Consumers of telemetry can detect if a telemetry item follows a specific semantic convention by checking for the presence of a `Required` attribute defined by such convention. For example, the presence of the `db.system` attribute on a span can be used as an indication that the span follows database semantics._ @@ -61,7 +61,7 @@ opt-in to emit them as defined for the `Opt-In` requirement level (if the attrib Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Opt-In` attributes. -This attribute requirement level is recommended for attributes that are particularly expensive to retrieve or might pose a security or privacy risk. These should therefore only be enabled enabled explicitly by a user making an informed decision. +This attribute requirement level is recommended for attributes that are particularly expensive to retrieve or might pose a security or privacy risk. These should therefore only be enabled explicitly by a user making an informed decision. ## Performance suggestions From 4adbc387e952c33be6710e9174c9659e2b5d7a55 Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Thu, 16 Mar 2023 06:31:51 -0700 Subject: [PATCH 046/124] Clarify that attribute requirement levels apply to instrumentation libraries (#3289) Based on discussion in semconv stability WG Closes #3283 ## Changes Clarifies that attribute requirement levels apply to instrumentation. And that, because users can transform their telemetry in a number of ways (e.g. metric views, span processors, and collector transformations), these requirement levels cannot be relied on by telemetry consumers. --------- Co-authored-by: Liudmila Molkova Co-authored-by: Bogdan Drutu --- specification/common/attribute-requirement-level.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index c7fb99345c..3543970a29 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -19,6 +19,8 @@ _This section applies to Log, Metric, Resource, and Span, and describes requirement levels for attributes defined in semantic conventions._ +Attribute requirement levels apply to the [instrumentation library](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/glossary.md#instrumentation-library). + The following attribute requirement levels are specified: - [Required](#required) From e182ff311df190a2715910a507d938b6750a64e6 Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Thu, 6 Apr 2023 21:56:17 -0700 Subject: [PATCH 047/124] Mark attribute requirement levels as stable (#3368) --- specification/common/attribute-requirement-level.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index 3543970a29..e9da7f0702 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -1,6 +1,6 @@ # Attribute Requirement Levels for Semantic Conventions -**Status**: [Experimental](../document-status.md) +**Status**: [Stable](../document-status.md)
Table of Contents From f4f8be2cce78c378a8c9f86cac2cfd9658a90a57 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Fri, 7 Apr 2023 15:55:06 -0400 Subject: [PATCH 048/124] Use path, not external URL, for link into glossary (#3375) - Contributes to https://github.com/open-telemetry/opentelemetry.io/issues/2429 - This is part of an effort to normalize links, for improved link checking on the OTel website --- specification/common/attribute-requirement-level.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index e9da7f0702..cfbc22b9da 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -19,7 +19,7 @@ _This section applies to Log, Metric, Resource, and Span, and describes requirement levels for attributes defined in semantic conventions._ -Attribute requirement levels apply to the [instrumentation library](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/glossary.md#instrumentation-library). +Attribute requirement levels apply to the [instrumentation library](../glossary.md#instrumentation-library). The following attribute requirement levels are specified: From 1b014422186a07611d45958a31838625883b3c99 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 8 May 2023 16:19:51 -0700 Subject: [PATCH 049/124] BREAKING: Replace `net.peer.*`/`net.host.*` with `client.*`/`server.*` (and `source.*`/`destination.*`) (#3402) --- specification/common/attribute-requirement-level.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index cfbc22b9da..aca07f487c 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -50,7 +50,7 @@ All instrumentations MUST populate the attribute when the given condition is sat When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. -For example, `net.peer.name` is `Conditionally Required` by the [Database convention](../trace/semantic_conventions/database.md) when available. When `net.sock.peer.addr` is available instead, instrumentation can do a DNS lookup, cache and populate `net.peer.name`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. +For example, `server.address` is `Conditionally Required` by the [Database convention](../trace/semantic_conventions/database.md) when available. When `server.socket.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. ## Recommended @@ -69,6 +69,6 @@ This attribute requirement level is recommended for attributes that are particul Here are several examples of expensive operations to be avoided by default: -- DNS lookups to populate `net.peer.name` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. +- DNS lookups to populate `server.address` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. - forcing an `http.route` calculation before the HTTP framework calculates it - reading response stream to find `http.response_content_length` when `Content-Length` header is not available From d6f3da6f11d145ded1feced9a830ecbd792a6f5e Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Mon, 8 May 2023 17:46:55 -0700 Subject: [PATCH 050/124] BREAKING: Rename remaining network attributes from `net.*` to `network.*` and align definitions with ECS (#3426) --- specification/common/attribute-requirement-level.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index aca07f487c..fc3f5b89d3 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -34,7 +34,7 @@ For example, Metric attributes that may have high cardinality can only be define A semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. -For example, [Database semantic convention](../trace/semantic_conventions/database.md) references `net.transport` attribute defined in [General attributes](../trace/semantic_conventions/span-general.md) with `Conditionally Required` level on it. +For example, [Database semantic convention](../trace/semantic_conventions/database.md) references `network.transport` attribute defined in [General attributes](../trace/semantic_conventions/span-general.md) with `Conditionally Required` level on it. ## Required From cc85702f85b13b0538279b502d26569646e369c7 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 8 May 2023 19:55:53 -0700 Subject: [PATCH 051/124] BREAKING: Introduce common `url.*` attributes, and improve use of namespacing under `http.*` (#3355) --- specification/common/attribute-naming.md | 6 +-- .../common/attribute-requirement-level.md | 4 +- specification/common/url.md | 45 +++++++++++++++++++ 3 files changed, 50 insertions(+), 5 deletions(-) create mode 100644 specification/common/url.md diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index d761b5f9fd..67aee0ad09 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -44,7 +44,7 @@ Names SHOULD follow these rules: purpose should primarily drive the decision about forming nested namespaces. - For each multi-word dot-delimited component of the attribute name separate the - words by underscores (i.e. use snake_case). For example `http.status_code` + words by underscores (i.e. use snake_case). For example `http.response.status_code` denotes the status code in the http namespace. - Names SHOULD NOT coincide with namespaces. For example if @@ -96,8 +96,8 @@ denote old attribute names in rename operations). - Semantic conventions exist for four areas: for Resource, Span, Log, and Metric attribute names. In addition, for spans we have two more areas: Event and Link attribute names. Identical namespaces or names in all these areas MUST have - identical meanings. For example the `http.method` span attribute name denotes - exactly the same concept as the `http.method` metric attribute, has the same + identical meanings. For example the `http.request.method` span attribute name denotes + exactly the same concept as the `http.request.method` metric attribute, has the same data type and the same set of possible values (in both cases it records the value of the HTTP protocol's request method as a string). diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index fc3f5b89d3..52f8e34f26 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -38,7 +38,7 @@ For example, [Database semantic convention](../trace/semantic_conventions/databa ## Required -All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, and can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.method` is an example of a Required attribute. +All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, and can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.request.method` is an example of a Required attribute. _Note: Consumers of telemetry can detect if a telemetry item follows a specific semantic convention by checking for the presence of a `Required` attribute defined by such convention. For example, the presence of the `db.system` attribute on a span can be used as an indication that the span follows database semantics._ @@ -71,4 +71,4 @@ Here are several examples of expensive operations to be avoided by default: - DNS lookups to populate `server.address` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. - forcing an `http.route` calculation before the HTTP framework calculates it -- reading response stream to find `http.response_content_length` when `Content-Length` header is not available +- reading response stream to find `http.response.body.size` when `Content-Length` header is not available diff --git a/specification/common/url.md b/specification/common/url.md new file mode 100644 index 0000000000..bbe041ba67 --- /dev/null +++ b/specification/common/url.md @@ -0,0 +1,45 @@ +# Semantic conventions for URL + +**Status**: [Experimental](../document-status.md) + +This document defines semantic conventions that describe URL and its components. + +
+Table of Contents + + + +- [Attributes](#attributes) +- [Sensitive information](#sensitive-information) + + + +
+ +## Attributes + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `url.scheme` | string | The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol. | `https`; `ftp`; `telnet` | Recommended | +| `url.full` | string | Absolute URL describing a network resource according to [RFC3986](https://www.rfc-editor.org/rfc/rfc3986) [1] | `https://www.foo.bar/search?q=OpenTelemetry#SemConv`; `//localhost` | Recommended | +| `url.path` | string | The [URI path](https://www.rfc-editor.org/rfc/rfc3986#section-3.3) component [2] | `/search` | Recommended | +| `url.query` | string | The [URI query](https://www.rfc-editor.org/rfc/rfc3986#section-3.4) component [3] | `q=OpenTelemetry` | Recommended | +| `url.fragment` | string | The [URI fragment](https://www.rfc-editor.org/rfc/rfc3986#section-3.5) component | `SemConv` | Recommended | + +**[1]:** For network calls, URL usually has `scheme://host[:port][path][?query][#fragment]` format, where the fragment is not transmitted over HTTP, but if it is known, it should be included nevertheless. +`url.full` MUST NOT contain credentials passed via URL in form of `https://username:password@www.example.com/`. In such case username and password should be redacted and attribute's value should be `https://REDACTED:REDACTED@www.example.com/`. +`url.full` SHOULD capture the absolute URL when it is available (or can be reconstructed) and SHOULD NOT be validated or modified except for sanitizing purposes. + +**[2]:** When missing, the value is assumed to be `/` + +**[3]:** Sensitive content provided in query string SHOULD be scrubbed when instrumentations can identify it. + + +## Sensitive information + +Capturing URL and its components MAY impose security risk. User and password information, when they are provided in [User Information](https://datatracker.ietf.org/doc/html/rfc3986#section-3.2.1) subcomponent, MUST NOT be recorded. + +Instrumentations that are aware of specific sensitive query string parameters MUST scrub their values before capturing `url.query` attribute. For example, native instrumentation of a client library that passes credentials or user location in URL, must scrub corresponding properties. + +_Note: Applications and telemetry consumers should scrub sensitive information from URL attributes on collected telemetry. In systems unable to identify sensitive information, certain attribute values may be redacted entirely._ From bd2b51a2495697a6b77620930f6cdb8b11bc052a Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Fri, 26 May 2023 11:23:42 -0400 Subject: [PATCH 052/124] Explain why custom attributes are not recommended to be placed in Otel namespaces (#3507) The @open-telemetry/technical-committee discussed and decided to keep the existing recommendations but clarify them and explain the purpose. --- specification/common/attribute-naming.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index 67aee0ad09..badb9acbd0 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -133,6 +133,13 @@ To do that consider a few options: `myuniquemapapp.longitude` is likely fine). Make sure the application name does not clash with an existing semantic convention namespace. +- It is not recommended to use existing OpenTelemetry semantic convention namespace + as a prefix for a new company- or application-specific attribute name. Doing so + may result in a name clash in the future, if OpenTelemetry decides to use that + same name for a different purpose or if some other third party instrumentation + decides to use that exact same attribute name and you combine that instrumentation + with your own. + - The name may be generally applicable to applications in the industry. In that case consider submitting a proposal to this specification to add a new name to the semantic conventions, and if necessary also to add a new namespace. From 9d7cdb92927eabef6df30b2b824ede02209c232c Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Wed, 12 Jul 2023 10:58:54 -0400 Subject: [PATCH 053/124] Hugo front-matter fixes for aliases and linkTitle (#3592) - Followup changes for https://github.com/open-telemetry/opentelemetry.io/issues/2793 - There are only changes to Hugo front matter - Adds `likeTitle`s for "Compatibility" pages - Adds aliases for pages that have moved or were renamed - Related: https://github.com/open-telemetry/opentelemetry.io/issues/3013 -- the `compatibility/openmetrics` spec page is in the list because it was renamed /cc @svrnm @cartermp --- specification/common/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specification/common/README.md b/specification/common/README.md index f36cb92b85..239bdf2014 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -1,6 +1,7 @@ + # Common specification concepts **Status**: [Stable, Feature-freeze](../document-status.md) From e3352d43bc1078e249d84688b3ab83e47d97b78f Mon Sep 17 00:00:00 2001 From: Josh Suereth Date: Tue, 3 Oct 2023 12:13:38 -0400 Subject: [PATCH 054/124] Remove local stubs of semantic conventions. (#3711) --- specification/common/attribute-naming.md | 14 +++----------- .../common/attribute-requirement-level.md | 6 ++++-- 2 files changed, 7 insertions(+), 13 deletions(-) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index badb9acbd0..548703db6f 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -65,7 +65,7 @@ Names SHOULD follow these rules: values: the executable name and command arguments. - When an attribute represents a measurement, - [Metric Name Pluralization Guidelines](../metrics/semantic_conventions/README.md#pluralization) + [Metric Name Pluralization Guidelines](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/metrics.md#pluralization) SHOULD be followed for the attribute name. ## Name Reuse Prohibition @@ -83,11 +83,7 @@ denote old attribute names in rename operations). of a namespace. - When coming up with a new semantic convention make sure to check existing - namespaces for - [Resources](../resource/semantic_conventions/README.md), - [Spans](../trace/semantic_conventions/README.md), - and - [Metrics](../metrics/semantic_conventions/README.md) + namespaces ([Semantic Conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md)) to see if the new name fits. - When a new namespace is necessary consider whether it should be a top-level @@ -111,11 +107,7 @@ denote old attribute names in rename operations). ## Recommendations for Application Developers As an application developer when you need to record an attribute first consult -existing semantic conventions for -[Resources](../resource/semantic_conventions/README.md), -[Spans](../trace/semantic_conventions/README.md), -and -[Metrics](../metrics/semantic_conventions/README.md). +existing [semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md). If an appropriate name does not exists you will need to come up with a new name. To do that consider a few options: diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index 52f8e34f26..1866c413ca 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -34,7 +34,8 @@ For example, Metric attributes that may have high cardinality can only be define A semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. -For example, [Database semantic convention](../trace/semantic_conventions/database.md) references `network.transport` attribute defined in [General attributes](../trace/semantic_conventions/span-general.md) with `Conditionally Required` level on it. + +For example, [Database semantic convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) references `network.transport` attribute defined in [General attributes](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/README.md) with `Conditionally Required` level on it. ## Required @@ -50,7 +51,8 @@ All instrumentations MUST populate the attribute when the given condition is sat When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. -For example, `server.address` is `Conditionally Required` by the [Database convention](../trace/semantic_conventions/database.md) when available. When `server.socket.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. + +For example, `server.address` is `Conditionally Required` by the [Database convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) when available. When `server.socket.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. ## Recommended From 7ee67a7192013216aaa723196b4dbbfd88bd1415 Mon Sep 17 00:00:00 2001 From: Carlos Alberto Cortez Date: Wed, 11 Oct 2023 01:15:24 +0200 Subject: [PATCH 055/124] Add a new AddLink() operation to Span. (#3678) Fixes #454 Related to #3337 As the Messaging SIG merged its last OTEP 222, we will be adding operations that require Links after Span creation, taking a direct route with `AddLink()`, albeit without any of the new members suggested in #3337, e.g. `timestamp` (to be discussed in a separate issue). ``` AddLink(spanContext, attributes /* optional */) ``` --- specification/common/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/README.md b/specification/common/README.md index 239bdf2014..1f17536bb6 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -123,7 +123,7 @@ at this time, as discussed in [data points](../metrics/data-model.md#metric-points), [Spans](../trace/api.md#set-attributes), Span [Events](../trace/api.md#add-events), Span -[Links](../trace/api.md#specifying-links) and +[Links](../trace/api.md#link) and [Log Records](../logs/data-model.md) may contain a collection of attributes. The keys in each such collection are unique, i.e. there MUST NOT exist more than one key-value pair with the same key. The enforcement of uniqueness may be performed From e56432cc1d9eff83e091a1776c8b3f897a0105a7 Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Mon, 23 Oct 2023 08:19:08 -0700 Subject: [PATCH 056/124] Rename/replace `(client|server).socket.(address|port)` attributes with `network.(peer|local).(address|port)`. (#3713) Co-authored-by: Armin Ruech --- specification/common/attribute-requirement-level.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index 1866c413ca..7f510fe731 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -52,7 +52,7 @@ All instrumentations MUST populate the attribute when the given condition is sat When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. -For example, `server.address` is `Conditionally Required` by the [Database convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) when available. When `server.socket.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. +For example, `server.address` is `Conditionally Required` by the [Database convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) when available. When `network.peer.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. ## Recommended From eece45b83c434fbbe7a0a3bcc0ddfec9f5e1bfee Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Tue, 7 Nov 2023 14:01:24 -0600 Subject: [PATCH 057/124] Remove files to keep --- specification/common/README.md | 150 ----------- .../common/attribute-type-mapping.md | 255 ------------------ specification/common/mapping-to-non-otlp.md | 95 ------- specification/common/url.md | 45 ---- 4 files changed, 545 deletions(-) delete mode 100644 specification/common/README.md delete mode 100644 specification/common/attribute-type-mapping.md delete mode 100644 specification/common/mapping-to-non-otlp.md delete mode 100644 specification/common/url.md diff --git a/specification/common/README.md b/specification/common/README.md deleted file mode 100644 index 1f17536bb6..0000000000 --- a/specification/common/README.md +++ /dev/null @@ -1,150 +0,0 @@ - - -# Common specification concepts - -**Status**: [Stable, Feature-freeze](../document-status.md) - -
-Table of Contents - - - -- [Attribute](#attribute) - * [Attribute Limits](#attribute-limits) - + [Configurable Parameters](#configurable-parameters) - + [Exempt Entities](#exempt-entities) -- [Attribute Collections](#attribute-collections) - - - -
- -## Attribute - - - -An `Attribute` is a key-value pair, which MUST have the following properties: - -- The attribute key MUST be a non-`null` and non-empty string. -- The attribute value is either: - - A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer. - - An array of primitive type values. The array MUST be homogeneous, - i.e., it MUST NOT contain values of different types. - -For protocols that do not natively support non-string values, non-string values SHOULD be represented as JSON-encoded strings. For example, the expression `int64(100)` will be encoded as `100`, `float64(1.5)` will be encoded as `1.5`, and an empty array of any type will be encoded as `[]`. - -Attribute values expressing a numerical value of zero, an empty string, or an -empty array are considered meaningful and MUST be stored and passed on to -processors / exporters. - -Attribute values of `null` are not valid and attempting to set a `null` value is -undefined behavior. - -`null` values SHOULD NOT be allowed in arrays. However, if it is impossible to -make sure that no `null` values are accepted -(e.g. in languages that do not have appropriate compile-time type checking), -`null` values within arrays MUST be preserved as-is (i.e., passed on to span -processors / exporters as `null`). If exporters do not support exporting `null` -values, they MAY replace those values by 0, `false`, or empty strings. -This is required for map/dictionary structures represented as two arrays with -indices that are kept in sync (e.g., two attributes `header_keys` and `header_values`, -both containing an array of strings to represent a mapping -`header_keys[i] -> header_values[i]`). - -See [Attribute Naming](attribute-naming.md) for naming guidelines. - -See [Requirement Level](attribute-requirement-level.md) for requirement levels guidelines. - -See [this document](attribute-type-mapping.md) to find out how to map values obtained -outside OpenTelemetry into OpenTelemetry attribute values. - -### Attribute Limits - -Execution of erroneous code can result in unintended attributes. If there are no -limits placed on attributes, they can quickly exhaust available memory, resulting -in crashes that are difficult to recover from safely. - -By default an SDK SHOULD apply truncation as per the list of -[configurable parameters](#configurable-parameters) below. - -If an SDK provides a way to: - -- set an attribute value length limit such that for each - attribute value: - - if it is a string, if it exceeds that limit (counting any character in it as - 1), SDKs MUST truncate that value, so that its length is at most equal - to the limit, - - if it is an array of strings, then apply the above rule to each of the - values separately, - - otherwise a value MUST NOT be truncated; -- set a limit of unique attribute keys such that: - - for each unique attribute key, addition of which would result in exceeding - the limit, SDK MUST discard that key/value pair. - -There MAY be a log emitted to indicate to the user that an attribute was -truncated or discarded. To prevent excessive logging, the log MUST NOT be -emitted more than once per record on which an attribute is set. - -If the SDK implements the limits above, it MUST provide a way to change these -limits programmatically. Names of the configuration options SHOULD be the same as -in the list below. - -An SDK MAY implement model-specific limits, for example -`SpanAttributeCountLimit` or `LogRecordAttributeCountLimit`. If both a general -and a model-specific limit are implemented, then the SDK MUST first attempt to -use the model-specific limit, if it isn't set, then the SDK MUST attempt to use -the general limit. If neither are defined, then the SDK MUST try to use the -model-specific limit default value, followed by the global limit default value. - -#### Configurable Parameters - -* `AttributeCountLimit` (Default=128) - Maximum allowed attribute count per record; -* `AttributeValueLengthLimit` (Default=Infinity) - Maximum allowed attribute value length; - -#### Exempt Entities - -Resource attributes SHOULD be exempt from the limits described above as resources -are not susceptible to the scenarios (auto-instrumentation) that result in -excessive attributes count or size. Resources are also sent only once per batch -instead of per span so it is relatively cheaper to have more/larger attributes -on them. Resources are also immutable by design and they are generally passed -down to TracerProvider along with limits. This makes it awkward to implement -attribute limits for Resources. - -Attributes, which belong to Metrics, are exempt from the limits described above -at this time, as discussed in -[Metrics Attribute Limits](../metrics/sdk.md#attribute-limits). - -## Attribute Collections - -[Resources](../resource/sdk.md), Metrics -[data points](../metrics/data-model.md#metric-points), -[Spans](../trace/api.md#set-attributes), Span -[Events](../trace/api.md#add-events), Span -[Links](../trace/api.md#link) and -[Log Records](../logs/data-model.md) may contain a collection of attributes. The -keys in each such collection are unique, i.e. there MUST NOT exist more than one -key-value pair with the same key. The enforcement of uniqueness may be performed -in a variety of ways as it best fits the limitations of the particular -implementation. - -Normally for the telemetry generated using OpenTelemetry SDKs the attribute -key-value pairs are set via an API that either accepts a single key-value pair -or a collection of key-value pairs. Setting an attribute with the same key as an -existing attribute SHOULD overwrite the existing attribute's value. See for -example Span's [SetAttribute](../trace/api.md#set-attributes) API. - -A typical implementation of [SetAttribute](../trace/api.md#set-attributes) API -will enforce the uniqueness by overwriting any existing attribute values pending -to be exported, so that when the Span is eventually exported the exporters see -only unique attributes. The OTLP format in particular requires that exported -Resources, Spans, Metric data points and Log Records contain only unique -attributes. - -Some other implementations may use a streaming approach where every -[SetAttribute](../trace/api.md#set-attributes) API call immediately results in -that individual attribute value being exported using a streaming wire protocol. -In such cases the enforcement of uniqueness will likely be the responsibility of -the recipient of this data. diff --git a/specification/common/attribute-type-mapping.md b/specification/common/attribute-type-mapping.md deleted file mode 100644 index 532d6156bc..0000000000 --- a/specification/common/attribute-type-mapping.md +++ /dev/null @@ -1,255 +0,0 @@ -# Mapping Arbitrary Data to OTLP AnyValue - -**Status**: [Experimental](../document-status.md) - -
-Table of Contents - - - -- [Converting to AnyValue](#converting-to-anyvalue) - * [Primitive Values](#primitive-values) - + [Integer Values](#integer-values) - + [Enumerations](#enumerations) - + [Floating Point Values](#floating-point-values) - + [String Values](#string-values) - + [Byte Sequences](#byte-sequences) - * [Composite Values](#composite-values) - + [Array Values](#array-values) - + [Associative Arrays With Unique Keys](#associative-arrays-with-unique-keys) - + [Associative Arrays With Non-Unique Keys](#associative-arrays-with-non-unique-keys) - + [Sets](#sets) - * [Other Values](#other-values) - * [Empty Values](#empty-values) - - - -
- -This document defines how to map (convert) arbitrary data (e.g. in-memory -objects) to OTLP's [AnyValue](https://github.com/open-telemetry/opentelemetry-proto/blob/cc4ed55c082cb75e084d40b4ddf3805eda099f97/opentelemetry/proto/common/v1/common.proto#L27). - -The mapping is needed when OpenTelemetry needs to convert a value produced outside -OpenTelemetry into a value that can be exported using an OTLP exporter, or otherwise be -converted to be used inside OpenTelemetry boundaries. Example use cases are the following: - -- In the [Logging SDK](../logs/sdk.md)s, to convert values received - from logging libraries into OpenTelemetry representation. -- In the Collector, to convert values received from various data sources into - [pdata](https://github.com/open-telemetry/opentelemetry-collector/blob/4998703dadd19fa91a88eabf7ccc68d728bee3fd/model/pdata/common.go#L84) - internal representation. - -## Converting to AnyValue - -[AnyValue](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L27) -is capable of representing primitive and structured data of certain types. - -Implementations that have source data in any form, such as in-memory objects -or data coming from other formats that needs to be converted to AnyValue SHOULD -follow the rules described below. - -### Primitive Values - -#### Integer Values - -Integer values which are within the range of 64 bit signed numbers -[-2^63..2^63-1] SHOULD be converted to AnyValue's -[int_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L33) -field. - -Integer values which are outside the range of 64 bit signed numbers SHOULD be -converted to AnyValue's -[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) -field using decimal representation. - -#### Enumerations - -Values, which belong to a limited enumerated set (e.g. a Java -[enum](https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html)), SHOULD be -converted to AnyValue's -[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) -field with the value of the string set to the symbolic name of the enumeration. - -If it is not possible to obtain the symbolic name of the enumeration, the -implementation SHOULD map enumeration values to AnyValue's -[int_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L33) -field set to the enum's ordinal value, when such an ordinal number is naturally -obtainable. - -If it is also not possible to obtain the ordinal value, the enumeration SHOULD be -converted to AnyValue's -[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) -field in any manner that the implementation deems reasonable. - -#### Floating Point Values - -Floating point values which are within the range and precision of IEEE 754 -64-bit floating point numbers (including IEEE 32-bit floating point values) -SHOULD be converted to AnyValue's -[double_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L34) -field. - -Floating point values which are outside the range or precision of IEEE 754 -64-bit floating point numbers (e.g. IEEE 128-bit floating bit values) SHOULD be -converted to AnyValue's -[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) -field using decimal floating point representation. - -#### String Values - -String values which are valid UTF-8 sequences SHOULD be converted to -AnyValue's -[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) -field. - -String values which are not valid Unicode sequences SHOULD be converted to -AnyValue's -[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) -with the bytes representing the string in the original order and format of the -source string. - -#### Byte Sequences - -Byte sequences (e.g. Go's `[]byte` slice or raw byte content of a file) SHOULD -be converted to AnyValue's -[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) -field. - -### Composite Values - -#### Array Values - -Values that represent ordered sequences of other values (such as -[arrays](https://docs.oracle.com/javase/specs/jls/se7/html/jls-10.html), -[vectors](https://en.cppreference.com/w/cpp/container/vector), ordered -[lists](https://docs.python.org/3/tutorial/datastructures.html#more-on-lists), -[slices](https://golang.org/ref/spec#Slice_types)) SHOULD be converted to -AnyValue's -[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) -field. String values and byte sequences are an exception from this rule (see -above). - -The rules described in this document should be applied recursively to each element -of the array. - -#### Associative Arrays With Unique Keys - -Values that represent associative arrays with unique keys (also often known -as maps, dictionaries or key-value stores) SHOULD be converted to AnyValue's -[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) -field. - -If the keys of the source array are not strings, they MUST be converted to -strings by any means available, often via a toString() or stringify functions -available in programming languages. The conversion function MUST be chosen in a -way that ensures that the resulting string keys are unique in the target array. - -The value part of each element of the source array SHOULD be converted to -AnyValue recursively. - -For example a JSON object `{"a": 123, "b": "def"}` SHOULD be converted to - -``` -AnyValue{ - kvlist_value:KeyValueList{ - values:[ - KeyValue{key:"a",value:AnyValue{int_value:123}}, - KeyValue{key:"b",value:AnyValue{string_value:"def"}}, - ] - } -} -``` - -The rules described in this document should be applied recursively to each value -of the associative array. - -#### Associative Arrays With Non-Unique Keys - -Values that represent an associative arrays with non-unique keys where multiple values may be associated with the same key (also sometimes known -as multimaps, multidicts) SHOULD be converted to AnyValue's -[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) -field. - -The resulting -[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) -field MUST list each key only once and the value of each element of -[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) -field MUST be an array represented using AnyValue's -[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) -field, each element of the -[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) -representing one value of source array associated with the given key. - -For example an associative array shown in the following table: - -|Key|Value| -|---|---| -|"abc"|123| -|"def"|"foo"| -|"def"|"bar"| - -SHOULD be converted to: - -``` -AnyValue{ - kvlist_value:KeyValueList{ - values:[ - KeyValue{ - key:"abc", - value:AnyValue{array_value:ArrayValue{values[ - AnyValue{int_value:123} - ]}} - }, - KeyValue{ - key:"def", - value:AnyValue{array_value:ArrayValue{values[ - AnyValue{string_value:"foo"}, - AnyValue{string_value:"bar"} - ]}} - }, - ] - } -} -``` - -The rules described in this document should be applied recursively to each value -of the associative array. - -#### Sets - -Unordered collections of unique values (such as -[Java Sets](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Set.html), -[C++ sets](https://en.cppreference.com/w/cpp/container/set), -[Python Sets](https://docs.python.org/3/tutorial/datastructures.html#sets)) SHOULD be -converted to AnyValue's -[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) -field, where each element of the set becomes an element of the array. - -The rules described in this document should be applied recursively to each value -of the set. - -### Other Values - -Any other values not listed above SHOULD be converted to AnyValue's -[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) -field if the source data can be serialized to a string (can be stringified) -using toString() or stringify functions available in programming languages. - -If the source data cannot be serialized to a string then the value SHOULD be -converted AnyValue's -[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) -field by serializing it into a byte sequence by any means available. - -If the source data cannot be serialized neither to a string nor to a byte -sequence then it SHOULD by converted to an empty AnyValue. - -### Empty Values - -If the source data has no type associated with it and is empty, null, nil or -otherwise indicates absence of data it SHOULD be converted to an -[empty](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L29) -AnyValue, where all the fields are unset. - -Empty values which has a type associated with them (e.g. empty associative -array) SHOULD be converted using the corresponding rules defined for the types -above. diff --git a/specification/common/mapping-to-non-otlp.md b/specification/common/mapping-to-non-otlp.md deleted file mode 100644 index a82dac3e63..0000000000 --- a/specification/common/mapping-to-non-otlp.md +++ /dev/null @@ -1,95 +0,0 @@ -# OpenTelemetry Transformation to non-OTLP Formats - -**Status**: [Stable](../document-status.md) - -All OpenTelemetry concepts and data recorded using OpenTelemetry API can be -directly and precisely represented using corresponding messages and fields of -OTLP format. However, for other formats this is not always the case. Sometimes a -format will not have a native way to represent a particular OpenTelemetry -concept or a field of a concept. - -This document defines the transformation between OpenTelemetry and formats other -than OTLP, for OpenTelemetry fields and concepts that have no direct semantic -equivalent in those other formats. - -Note: when a format has a direct semantic equivalent for a particular field or -concept then the recommendation in this document MUST be ignored. - -See also additional specific transformation rules for -[Jaeger](../trace/sdk_exporters/jaeger.md) and [Zipkin](../trace/sdk_exporters/zipkin.md). -The specific rules for Jaeger and Zipkin take precedence over the generic rules defined -in this document. - -## Mappings - -### InstrumentationScope - -OpenTelemetry `InstrumentationScope`'s fields MUST be reported as key-value -pairs associated with the Span, Metric Data Point or LogRecord using the following mapping: - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `otel.scope.name` | string | The name of the instrumentation scope - (`InstrumentationScope.Name` in OTLP). | `io.opentelemetry.contrib.mongodb` | Recommended | -| `otel.scope.version` | string | The version of the instrumentation scope - (`InstrumentationScope.Version` in OTLP). | `1.0.0` | Recommended | - - -The following deprecated aliases MUST also be reported with exact same values for -backward compatibility reasons: - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `otel.library.name` | string | Deprecated, use the `otel.scope.name` attribute. | `io.opentelemetry.contrib.mongodb` | Recommended | -| `otel.library.version` | string | Deprecated, use the `otel.scope.version` attribute. | `1.0.0` | Recommended | - - -### Span Status - -Span `Status` MUST be reported as key-value pairs associated with the Span, -unless the `Status` is `UNSET`. In the latter case it MUST NOT be reported. - -The following table defines the OpenTelemetry `Status`'s mapping to Span's -key-value pairs: - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `otel.status_code` | string | Name of the code, either "OK" or "ERROR". MUST NOT be set if the status code is UNSET. | `OK` | Recommended | -| `otel.status_description` | string | Description of the Status if it has a value, otherwise not set. | `resource not found` | Recommended | - -`otel.status_code` MUST be one of the following: - -| Value | Description | -|---|---| -| `OK` | The operation has been validated by an Application developer or Operator to have completed successfully. | -| `ERROR` | The operation contains an error. | - - -### Dropped Attributes Count - -OpenTelemetry dropped attributes count MUST be reported as a key-value -pair associated with the corresponding data entity (e.g. Span, Span Link, Span Event, -Metric data point, LogRecord, etc). The key name MUST be `otel.dropped_attributes_count`. - -This key-value pair should only be recorded when it contains a non-zero value. - -### Dropped Events Count - -OpenTelemetry Span's dropped events count MUST be reported as a key-value pair -associated with the Span. The key name MUST be `otel.dropped_events_count`. - -This key-value pair should only be recorded when it contains a non-zero value. - -### Dropped Links Count - -OpenTelemetry Span's dropped links count MUST be reported as a key-value pair -associated with the Span. The key name MUST be `otel.dropped_links_count`. - -This key-value pair should only be recorded when it contains a non-zero value. - -### Instrumentation Scope Attributes - -Exporters to formats that don't have a concept that is equivalent to the Scope -SHOULD record the attributes at the most suitable place in their corresponding format, -typically at the Span, Metric or LogRecord equivalent. diff --git a/specification/common/url.md b/specification/common/url.md deleted file mode 100644 index bbe041ba67..0000000000 --- a/specification/common/url.md +++ /dev/null @@ -1,45 +0,0 @@ -# Semantic conventions for URL - -**Status**: [Experimental](../document-status.md) - -This document defines semantic conventions that describe URL and its components. - -
-Table of Contents - - - -- [Attributes](#attributes) -- [Sensitive information](#sensitive-information) - - - -
- -## Attributes - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `url.scheme` | string | The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol. | `https`; `ftp`; `telnet` | Recommended | -| `url.full` | string | Absolute URL describing a network resource according to [RFC3986](https://www.rfc-editor.org/rfc/rfc3986) [1] | `https://www.foo.bar/search?q=OpenTelemetry#SemConv`; `//localhost` | Recommended | -| `url.path` | string | The [URI path](https://www.rfc-editor.org/rfc/rfc3986#section-3.3) component [2] | `/search` | Recommended | -| `url.query` | string | The [URI query](https://www.rfc-editor.org/rfc/rfc3986#section-3.4) component [3] | `q=OpenTelemetry` | Recommended | -| `url.fragment` | string | The [URI fragment](https://www.rfc-editor.org/rfc/rfc3986#section-3.5) component | `SemConv` | Recommended | - -**[1]:** For network calls, URL usually has `scheme://host[:port][path][?query][#fragment]` format, where the fragment is not transmitted over HTTP, but if it is known, it should be included nevertheless. -`url.full` MUST NOT contain credentials passed via URL in form of `https://username:password@www.example.com/`. In such case username and password should be redacted and attribute's value should be `https://REDACTED:REDACTED@www.example.com/`. -`url.full` SHOULD capture the absolute URL when it is available (or can be reconstructed) and SHOULD NOT be validated or modified except for sanitizing purposes. - -**[2]:** When missing, the value is assumed to be `/` - -**[3]:** Sensitive content provided in query string SHOULD be scrubbed when instrumentations can identify it. - - -## Sensitive information - -Capturing URL and its components MAY impose security risk. User and password information, when they are provided in [User Information](https://datatracker.ietf.org/doc/html/rfc3986#section-3.2.1) subcomponent, MUST NOT be recorded. - -Instrumentations that are aware of specific sensitive query string parameters MUST scrub their values before capturing `url.query` attribute. For example, native instrumentation of a client library that passes credentials or user location in URL, must scrub corresponding properties. - -_Note: Applications and telemetry consumers should scrub sensitive information from URL attributes on collected telemetry. In systems unable to identify sensitive information, certain attribute values may be redacted entirely._ From d445ea5eec77394085f39c610eb42f700e07e709 Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Tue, 7 Nov 2023 14:02:50 -0600 Subject: [PATCH 058/124] Move attribute documents --- {specification/common => docs/general}/attribute-naming.md | 0 .../common => docs/general}/attribute-requirement-level.md | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename {specification/common => docs/general}/attribute-naming.md (100%) rename {specification/common => docs/general}/attribute-requirement-level.md (100%) diff --git a/specification/common/attribute-naming.md b/docs/general/attribute-naming.md similarity index 100% rename from specification/common/attribute-naming.md rename to docs/general/attribute-naming.md diff --git a/specification/common/attribute-requirement-level.md b/docs/general/attribute-requirement-level.md similarity index 100% rename from specification/common/attribute-requirement-level.md rename to docs/general/attribute-requirement-level.md From 84cd08ef20d86c265ca79ec5e198dc5a592df55e Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Thu, 9 Nov 2023 09:30:19 -0600 Subject: [PATCH 059/124] Change to relative links --- docs/general/attribute-naming.md | 6 +++--- docs/general/attribute-requirement-level.md | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/general/attribute-naming.md b/docs/general/attribute-naming.md index 548703db6f..1849b09c02 100644 --- a/docs/general/attribute-naming.md +++ b/docs/general/attribute-naming.md @@ -65,7 +65,7 @@ Names SHOULD follow these rules: values: the executable name and command arguments. - When an attribute represents a measurement, - [Metric Name Pluralization Guidelines](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/metrics.md#pluralization) + [Metric Name Pluralization Guidelines](./metrics.md#pluralization) SHOULD be followed for the attribute name. ## Name Reuse Prohibition @@ -83,7 +83,7 @@ denote old attribute names in rename operations). of a namespace. - When coming up with a new semantic convention make sure to check existing - namespaces ([Semantic Conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md)) + namespaces ([Semantic Conventions](./README.md)) to see if the new name fits. - When a new namespace is necessary consider whether it should be a top-level @@ -107,7 +107,7 @@ denote old attribute names in rename operations). ## Recommendations for Application Developers As an application developer when you need to record an attribute first consult -existing [semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md). +existing [semantic conventions](./README.md). If an appropriate name does not exists you will need to come up with a new name. To do that consider a few options: diff --git a/docs/general/attribute-requirement-level.md b/docs/general/attribute-requirement-level.md index 7f510fe731..45ba93887b 100644 --- a/docs/general/attribute-requirement-level.md +++ b/docs/general/attribute-requirement-level.md @@ -1,4 +1,4 @@ -# Attribute Requirement Levels for Semantic Conventions +# Attribute Requirement Levels **Status**: [Stable](../document-status.md) @@ -35,7 +35,7 @@ For example, Metric attributes that may have high cardinality can only be define A semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. -For example, [Database semantic convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) references `network.transport` attribute defined in [General attributes](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/README.md) with `Conditionally Required` level on it. +For example, [Database semantic convention](../database/README.md) references `network.transport` attribute defined in [General attributes](./README.md) with `Conditionally Required` level on it. ## Required @@ -52,7 +52,7 @@ All instrumentations MUST populate the attribute when the given condition is sat When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. -For example, `server.address` is `Conditionally Required` by the [Database convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) when available. When `network.peer.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. +For example, `server.address` is `Conditionally Required` by the [Database convention](../database/README.md) when available. When `network.peer.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. ## Recommended From f183527bf7c764497b50cf6ce8694b2d3af8fb05 Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Thu, 9 Nov 2023 09:48:53 -0600 Subject: [PATCH 060/124] Fix more links --- docs/general/attribute-naming.md | 4 +++- docs/general/attribute-requirement-level.md | 6 ++++-- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/general/attribute-naming.md b/docs/general/attribute-naming.md index 1849b09c02..a800f7bd39 100644 --- a/docs/general/attribute-naming.md +++ b/docs/general/attribute-naming.md @@ -1,6 +1,6 @@ # Attribute Naming -**Status**: [Stable](../document-status.md) +**Status**: [Stable][DocumentStatus]
Table of Contents @@ -154,3 +154,5 @@ and protocols. Any additions to the `otel.*` namespace MUST be approved as part of OpenTelemetry specification. + +[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/document-status.md diff --git a/docs/general/attribute-requirement-level.md b/docs/general/attribute-requirement-level.md index 45ba93887b..ef8b46a0b3 100644 --- a/docs/general/attribute-requirement-level.md +++ b/docs/general/attribute-requirement-level.md @@ -1,6 +1,6 @@ # Attribute Requirement Levels -**Status**: [Stable](../document-status.md) +**Status**: [Stable][DocumentStatus]
Table of Contents @@ -19,7 +19,7 @@ _This section applies to Log, Metric, Resource, and Span, and describes requirement levels for attributes defined in semantic conventions._ -Attribute requirement levels apply to the [instrumentation library](../glossary.md#instrumentation-library). +Attribute requirement levels apply to the [instrumentation library](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/glossary.md#instrumentation-library). The following attribute requirement levels are specified: @@ -74,3 +74,5 @@ Here are several examples of expensive operations to be avoided by default: - DNS lookups to populate `server.address` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. - forcing an `http.route` calculation before the HTTP framework calculates it - reading response stream to find `http.response.body.size` when `Content-Length` header is not available + +[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/document-status.md From 1dc1e04ab61f47bd7e1c65ee5f478cfec25233da Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Fri, 10 Nov 2023 08:05:47 -0600 Subject: [PATCH 061/124] Point to 1.26.0 of specification --- docs/general/attribute-requirement-level.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/general/attribute-requirement-level.md b/docs/general/attribute-requirement-level.md index ef8b46a0b3..eb89336df3 100644 --- a/docs/general/attribute-requirement-level.md +++ b/docs/general/attribute-requirement-level.md @@ -19,7 +19,7 @@ _This section applies to Log, Metric, Resource, and Span, and describes requirement levels for attributes defined in semantic conventions._ -Attribute requirement levels apply to the [instrumentation library](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/glossary.md#instrumentation-library). +Attribute requirement levels apply to the [instrumentation library](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/glossary.md#instrumentation-library). The following attribute requirement levels are specified: From b85a0db524135dac1115c880dfb92a9e4cff1920 Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Mon, 13 Nov 2023 10:44:46 -0600 Subject: [PATCH 062/124] Update spec references to point to local files --- docs/attributes-registry/README.md | 2 +- docs/resource/README.md | 2 +- model/README.md | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/attributes-registry/README.md b/docs/attributes-registry/README.md index 40c63bf9f5..607e9cfc88 100644 --- a/docs/attributes-registry/README.md +++ b/docs/attributes-registry/README.md @@ -49,4 +49,4 @@ Currently, the following namespaces exist: * [URL](url.md) * [User agent](user-agent.md) -[developers recommendations]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/common/attribute-naming.md#recommendations-for-application-developers +[developers recommendations]: ../general/attribute-naming.md#recommendations-for-application-developers diff --git a/docs/resource/README.md b/docs/resource/README.md index 4d3e747dd6..5a12b18593 100644 --- a/docs/resource/README.md +++ b/docs/resource/README.md @@ -47,7 +47,7 @@ This document defines standard attributes for resources. These attributes are ty Attributes are grouped logically by the type of the concept that they described. Attributes in the same group have a common prefix that ends with a dot. For example all attributes that describe Kubernetes properties start with "k8s." -See [Attribute Requirement Levels](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/common/attribute-requirement-level.md) for details on when attributes +See [Attribute Requirement Levels](../general/attribute-requirement-level.md) for details on when attributes should be included. ## Attributes with Special Handling diff --git a/model/README.md b/model/README.md index 482c59ab14..2fc6f39b35 100644 --- a/model/README.md +++ b/model/README.md @@ -10,8 +10,8 @@ the generated markdown output in the [docs](../docs/README.md) folder. ## Writing semantic conventions Semantic conventions for the spec MUST adhere to the -[attribute naming](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/common/attribute-naming.md), -[attribute requirement level](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/common/attribute-requirement-level.md), +[attribute naming](../docs/general/attribute-naming.md), +[attribute requirement level](../docs/general/attribute-requirement-level.md), and [metric requirement level](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/metrics/metric-requirement-level.md) conventions. Refer to the [syntax](https://github.com/open-telemetry/build-tools/tree/v0.23.0/semantic-conventions/syntax.md) From 4f5831ca5fc6b9e1eb4717b1c31e43e5d30a5858 Mon Sep 17 00:00:00 2001 From: Bogdan Drutu Date: Wed, 22 Jul 2020 11:30:11 -0700 Subject: [PATCH 063/124] Centralize attributes definition (#722) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Centralize attributes definition Signed-off-by: Bogdan Drutu * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/common/common.md Co-authored-by: Christian Neumüller * Update specification/overview.md Co-authored-by: Christian Neumüller Co-authored-by: Christian Neumüller --- specification/common/common.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 specification/common/common.md diff --git a/specification/common/common.md b/specification/common/common.md new file mode 100644 index 0000000000..b8d99a5eef --- /dev/null +++ b/specification/common/common.md @@ -0,0 +1,29 @@ +# Common specification concepts + +
+ +Table of Contents + + +- [Attributes](#attribute) + +
+ +## Attributes + +Attributes are a list of zero or more key-value pairs. An `Attribute` MUST have the following properties: + +- The attribute key, which MUST be a non-`null` and non-empty string. +- The attribute value, which is either: + - A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer. + - An array of primitive type values. The array MUST be homogeneous, + i.e. it MUST NOT contain values of different types. For protocols that do + not natively support array values such values SHOULD be represented as JSON strings. + +`null` values within arrays MUST be preserved as-is (i.e., passed on to span +processors / exporters as `null`). If exporters do not support exporting `null` +values, they MAY replace those values by 0, `false`, or empty strings. +This is required for map/dictionary structures represented as two arrays with +indices that are kept in sync (e.g., two attributes `header_keys` and `header_values`, +both containing an array of strings to represent a mapping +`header_keys[i] -> header_values[i]`). From 4a2a4ef946cc316e177676c03f64ea1f1c822389 Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Mon, 27 Jul 2020 10:51:44 -0700 Subject: [PATCH 064/124] DocFX sanity check (#742) --- specification/common/common.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/common.md b/specification/common/common.md index b8d99a5eef..b55f47c1bb 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -5,7 +5,7 @@ Table of Contents -- [Attributes](#attribute) +- [Attributes](#attributes)
From 46b79ae1e78a970b35b03828abafbf008c65761e Mon Sep 17 00:00:00 2001 From: Giovanni Liva Date: Thu, 13 Aug 2020 19:10:46 +0200 Subject: [PATCH 065/124] Consistency between Span and Resource attributes (#777) * Consistency between Span and Resource attributes * Address feedback * Wording --- specification/common/common.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index b55f47c1bb..681721fb11 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -20,6 +20,15 @@ Attributes are a list of zero or more key-value pairs. An `Attribute` MUST have i.e. it MUST NOT contain values of different types. For protocols that do not natively support array values such values SHOULD be represented as JSON strings. +Attributes SHOULD preserve the order in which they're set. + +Attribute values expressing a numerical value of zero, an empty string, or an +empty array are considered meaningful and MUST be stored and passed on to +processors / exporters. Attribute values of `null` are considered to be not set +and get discarded as if that `Attribute` has never been created. +As an exception to this, if overwriting of values is supported, this results in +removing the attribute. + `null` values within arrays MUST be preserved as-is (i.e., passed on to span processors / exporters as `null`). If exporters do not support exporting `null` values, they MAY replace those values by 0, `false`, or empty strings. From 7a01501836e0af659eaea57f2942d963d6081a29 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Mon, 17 Aug 2020 12:40:17 -0400 Subject: [PATCH 066/124] Add conventions for attribute names (#807) * Add conventions for attribute names Resolves: https://github.com/open-telemetry/opentelemetry-specification/issues/726 * Address PR comments * Re-word company/application specific attribute recommendations --- specification/common/common.md | 88 +++++++++++++++++++++++++++++++++- 1 file changed, 87 insertions(+), 1 deletion(-) diff --git a/specification/common/common.md b/specification/common/common.md index 681721fb11..62637d5ec9 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -6,7 +6,7 @@ Table of Contents - [Attributes](#attributes) - + - [Attribute Naming](#attribute-naming)
## Attributes @@ -36,3 +36,89 @@ This is required for map/dictionary structures represented as two arrays with indices that are kept in sync (e.g., two attributes `header_keys` and `header_values`, both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). + +### Attribute Naming + +Attribute name (also known as the "attribute key") MUST be a valid Unicode +sequence. + +Attribute names SHOULD follow these rules: + +- Use namespacing to avoid name clashes. Delimit the namespaces using a dot + character. For example `service.version` denotes the service version where + `service` is the namespace and `version` is an attribute in that namespace. + +- Namespaces can be nested. For example `telemetry.sdk` is a namespace inside + top-level `telemetry` namespace and `telemetry.sdk.name` is an attribute + inside `telemetry.sdk` namespace. + +- For each multi-word dot-delimited component of the attribute name separate the + words by underscores (i.e. use snake_case). For example `http.status_code` + denotes the status code in the http namespace. + +- Attribute names SHOULD NOT coincide with namespaces. For example if + `service.instance.id` is an attribute name then it is no longer valid to have + an attribute named `service.instance` because `service.instance` is already a + namespace. Because of this rule be careful when choosing attribute names: + every existing attribute name prohibits existence of an equally named + namespace in the future, and vice versa: any existing namespace prohibits + existence of an equally named attribute in the future. + +#### Recommendations for OpenTelemetry Authors + +- All attribute names that are part of OpenTelemetry semantic conventions + SHOULD be part of a namespace. + +- When coming up with a new convention make sure to check existing namespaces + for + [Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), + [Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), + and + [Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions) + to see if the new attributes fits. + +- When a new namespace is necessary consider whether it should be a top-level + namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). + +- Semantic conventions MUST limit attribute names to printable Basic Latin + characters (more precisely to + [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) + subset of Unicode code points). It is recommended to further limit attribute + names to the following Unicode code points: Latin alphabet, Numeric, + Underscore, Dot (as namespace delimiter). + +#### Recommendations for Application Developers + +As an application developer when you need to record an attribute first consult +existing semantic conventions for +[Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), +[Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), +and +[Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions). +If appropriate attribute name does not exists you will need to come up with a +new name. To do that consider a few options: + +- The attribute is specific to your company and may be possibly used outside the + company as well. To avoid name clashes with attributes introduced by other + companies (in a distributed system that uses applications from multiple + vendors) it is recommended to prefix the attribute name by your company's + reverse domain name, e.g. `com.acme.shopname`. + +- The attribute is specific to your application that will be used internally + only. If you already have an internal company process that helps you to ensure + no name clashes happen then feel free to follow it. Otherwise it is + recommended to prefix the attribute name by your application name, provided + that the application name is reasonably unique within your organization (e.g. + `myuniquemapapp.longitude` is likely fine). Make sure the application name + does not clash with an existing semantic convention namespace. + +- The attribute may be generally applicable to applications in the industry. In + that case consider submitting a proposal to this specification to add a new + attribute to the semantic conventions, if necessary also to add a new + namespace for the attribute. + +It is recommended to limit attribute names to printable Basic Latin characters +(more precisely to +[U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) +subset of Unicode code points). + From 3fb086c80f761c02c4ef43d9a2ddbaa36fb2f16d Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Wed, 19 Aug 2020 13:49:28 -0400 Subject: [PATCH 067/124] Extend attribute naming rules to metric labels (#821) * Extend attribute naming rules to metric labels We earlier defined naming rules for attributes, however we do not have similar rules for metric labels. This commit extends the exact same set of rules to metric labels. This was brought up in this comment https://github.com/open-telemetry/opentelemetry-specification/pull/807#discussion_r471550053 * Address PR comments --- specification/common/common.md | 95 +++++++++++++++++++--------------- 1 file changed, 52 insertions(+), 43 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 62637d5ec9..7cedad5f3f 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -6,7 +6,7 @@ Table of Contents - [Attributes](#attributes) - - [Attribute Naming](#attribute-naming) + - [Attribute and Label Naming](#attribute-and-label-naming)
## Attributes @@ -37,12 +37,22 @@ indices that are kept in sync (e.g., two attributes `header_keys` and `header_va both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). -### Attribute Naming +## Attribute and Label Naming -Attribute name (also known as the "attribute key") MUST be a valid Unicode -sequence. +_This section applies to Resource, Span and Log attribute names (also known as +the "attribute keys") and to keys of Metric labels. For brevity within this +section when we use the term "name" without an adjective it is implied to mean +"attribute name or metric label key"._ -Attribute names SHOULD follow these rules: +Every name MUST be a valid Unicode sequence. + +_Note: we merely require that the names are represented as Unicode sequences. +This specification does not define how exactly the Unicode sequences are +encoded. The encoding can vary from one programming language to another and from +one wire format to another. Use the idiomatic way to represent Unicode in the +particular programming language or wire format._ + +Names SHOULD follow these rules: - Use namespacing to avoid name clashes. Delimit the namespaces using a dot character. For example `service.version` denotes the service version where @@ -56,68 +66,67 @@ Attribute names SHOULD follow these rules: words by underscores (i.e. use snake_case). For example `http.status_code` denotes the status code in the http namespace. -- Attribute names SHOULD NOT coincide with namespaces. For example if +- Names SHOULD NOT coincide with namespaces. For example if `service.instance.id` is an attribute name then it is no longer valid to have an attribute named `service.instance` because `service.instance` is already a - namespace. Because of this rule be careful when choosing attribute names: - every existing attribute name prohibits existence of an equally named - namespace in the future, and vice versa: any existing namespace prohibits - existence of an equally named attribute in the future. + namespace. Because of this rule be careful when choosing names: every existing + name prohibits existence of an equally named namespace in the future, and vice + versa: any existing namespace prohibits existence of an equally named + attribute or label key in the future. -#### Recommendations for OpenTelemetry Authors +### Recommendations for OpenTelemetry Authors -- All attribute names that are part of OpenTelemetry semantic conventions - SHOULD be part of a namespace. +- All names that are part of OpenTelemetry semantic conventions SHOULD be part + of a namespace. -- When coming up with a new convention make sure to check existing namespaces - for +- When coming up with a new semantic convention make sure to check existing + namespaces for [Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), [Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), and [Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions) - to see if the new attributes fits. + to see if the new name fits. - When a new namespace is necessary consider whether it should be a top-level namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). -- Semantic conventions MUST limit attribute names to printable Basic Latin - characters (more precisely to +- Semantic conventions MUST limit names to printable Basic Latin characters + (more precisely to [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) - subset of Unicode code points). It is recommended to further limit attribute - names to the following Unicode code points: Latin alphabet, Numeric, - Underscore, Dot (as namespace delimiter). + subset of Unicode code points). It is recommended to further limit names to + the following Unicode code points: Latin alphabet, Numeric, Underscore, Dot + (as namespace delimiter). -#### Recommendations for Application Developers +### Recommendations for Application Developers -As an application developer when you need to record an attribute first consult -existing semantic conventions for +As an application developer when you need to record an attribute or a label +first consult existing semantic conventions for [Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), [Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), and [Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions). -If appropriate attribute name does not exists you will need to come up with a -new name. To do that consider a few options: - -- The attribute is specific to your company and may be possibly used outside the - company as well. To avoid name clashes with attributes introduced by other - companies (in a distributed system that uses applications from multiple - vendors) it is recommended to prefix the attribute name by your company's - reverse domain name, e.g. `com.acme.shopname`. - -- The attribute is specific to your application that will be used internally - only. If you already have an internal company process that helps you to ensure - no name clashes happen then feel free to follow it. Otherwise it is - recommended to prefix the attribute name by your application name, provided - that the application name is reasonably unique within your organization (e.g. +If an appropriate name does not exists you will need to come up with a new name. +To do that consider a few options: + +- The name is specific to your company and may be possibly used outside the + company as well. To avoid clashes with names introduced by other companies (in + a distributed system that uses applications from multiple vendors) it is + recommended to prefix the new name by your company's reverse domain name, e.g. + `com.acme.shopname`. + +- The name is specific to your application that will be used internally only. If + you already have an internal company process that helps you to ensure no name + clashes happen then feel free to follow it. Otherwise it is recommended to + prefix the attribute name or label key by your application name, provided that + the application name is reasonably unique within your organization (e.g. `myuniquemapapp.longitude` is likely fine). Make sure the application name does not clash with an existing semantic convention namespace. -- The attribute may be generally applicable to applications in the industry. In - that case consider submitting a proposal to this specification to add a new - attribute to the semantic conventions, if necessary also to add a new - namespace for the attribute. +- The name may be generally applicable to applications in the industry. In that + case consider submitting a proposal to this specification to add a new name to + the semantic conventions, and if necessary also to add a new namespace. -It is recommended to limit attribute names to printable Basic Latin characters +It is recommended to limit names to printable Basic Latin characters (more precisely to [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) subset of Unicode code points). From 15097436c79462b75e4b967d83516c24f935639b Mon Sep 17 00:00:00 2001 From: Bogdan Drutu Date: Wed, 19 Aug 2020 14:43:05 -0700 Subject: [PATCH 068/124] Fix lint check in makefile, fix errors (#837) Signed-off-by: Bogdan Drutu --- specification/common/common.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/common.md b/specification/common/common.md index 7cedad5f3f..f6a917b4cf 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -7,6 +7,7 @@ Table of Contents - [Attributes](#attributes) - [Attribute and Label Naming](#attribute-and-label-naming) +
## Attributes @@ -130,4 +131,3 @@ It is recommended to limit names to printable Basic Latin characters (more precisely to [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) subset of Unicode code points). - From a15b95dafa054326ac44acd23312c0b3c001ce30 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Mon, 24 Aug 2020 13:14:37 -0400 Subject: [PATCH 069/124] Require that names and namespaces are one global space across all semantic convention areas (#832) * Require that names and namespaces are one global space across all semantic convention areas We have semantic conventions for Resources, Spans and Metrics (in the future also Logs are expected). It was not clear if the attribute names across all convention areas should be globally unique. This commit asserts that conventions are one space, they are not independent spaces with their own namespaces each. We prohibit using the same Span or Resource attribute name or metric label name but give them slightly different meanings or value sets. Resolves: https://github.com/open-telemetry/opentelemetry-specification/issues/815 * Address PR comments Co-authored-by: Bogdan Drutu --- specification/common/common.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index f6a917b4cf..44f5ccf2f5 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -91,6 +91,15 @@ Names SHOULD follow these rules: - When a new namespace is necessary consider whether it should be a top-level namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). +- Semantic conventions exist for four areas: for Resource, Span and Log + attribute names as well as Metric label keys. In addition, for spans we have + two more areas: Event and Link attribute names. Identical namespaces or names + in all these areas MUST have identical meanings. For example the `http.method` + span attribute name denotes exactly the same concept as the `http.method` + metric label, has the same data type and the same set of possible values (in + both cases it records the value of the HTTP protocol's request method as a + string). + - Semantic conventions MUST limit names to printable Basic Latin characters (more precisely to [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) From 906beb96fc8f8c38f3d2e31c9226b7efcb279067 Mon Sep 17 00:00:00 2001 From: Armin Ruech Date: Fri, 25 Sep 2020 11:49:15 +0200 Subject: [PATCH 070/124] Define `null` as an invalid value for attributes and declare attempts to set `null` as undefined behavior (#992) --- specification/common/common.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 44f5ccf2f5..974cf51821 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -25,10 +25,10 @@ Attributes SHOULD preserve the order in which they're set. Attribute values expressing a numerical value of zero, an empty string, or an empty array are considered meaningful and MUST be stored and passed on to -processors / exporters. Attribute values of `null` are considered to be not set -and get discarded as if that `Attribute` has never been created. -As an exception to this, if overwriting of values is supported, this results in -removing the attribute. +processors / exporters. + +Attribute values of `null` are not valid and attempting to set a `null` value is +undefined behavior. `null` values within arrays MUST be preserved as-is (i.e., passed on to span processors / exporters as `null`). If exporters do not support exporting `null` From 9d54f2f42b380473bd3eae52e4048363a909e4db Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20Neum=C3=BCller?= Date: Wed, 4 Nov 2020 21:29:48 +0100 Subject: [PATCH 071/124] Fix absolute links to spec (#1192) * Remove absolute links where possible. * Work around semantic conventions. * Docfx. * Fix YAML. * More docfx. --- specification/common/common.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 974cf51821..0352f819ab 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -82,10 +82,10 @@ Names SHOULD follow these rules: - When coming up with a new semantic convention make sure to check existing namespaces for - [Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), - [Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), + [Resources](../resource/semantic_conventions/README.md), + [Spans](../trace/semantic_conventions/README.md), and - [Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions) + [Metrics](../metrics/semantic_conventions/README.md) to see if the new name fits. - When a new namespace is necessary consider whether it should be a top-level @@ -111,10 +111,10 @@ Names SHOULD follow these rules: As an application developer when you need to record an attribute or a label first consult existing semantic conventions for -[Resources](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/resource/semantic_conventions), -[Spans](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/trace/semantic_conventions), +[Resources](../resource/semantic_conventions/README.md), +[Spans](../trace/semantic_conventions/README.md), and -[Metrics](https://github.com/open-telemetry/opentelemetry-specification/tree/master/specification/metrics/semantic_conventions). +[Metrics](../metrics/semantic_conventions/README.md). If an appropriate name does not exists you will need to come up with a new name. To do that consider a few options: From 2012c9e3864445c2dd9fbbef0103c1e8e32bbc8b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20Neum=C3=BCller?= Date: Tue, 10 Nov 2020 17:05:58 +0100 Subject: [PATCH 072/124] Remove ordering reqirement for attributes. (#1212) * Remove ordering for attributes. * Fill in CHANGELOG link --- specification/common/common.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 0352f819ab..fca4b007a1 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -21,8 +21,6 @@ Attributes are a list of zero or more key-value pairs. An `Attribute` MUST have i.e. it MUST NOT contain values of different types. For protocols that do not natively support array values such values SHOULD be represented as JSON strings. -Attributes SHOULD preserve the order in which they're set. - Attribute values expressing a numerical value of zero, an empty string, or an empty array are considered meaningful and MUST be stored and passed on to processors / exporters. From 501f8a108a58b0a43d3c745458affa2515b9b8f3 Mon Sep 17 00:00:00 2001 From: Przemek Maciolek <58699843+pmm-sumo@users.noreply.github.com> Date: Tue, 10 Nov 2020 21:19:52 +0100 Subject: [PATCH 073/124] Include attribute name pluralization guidelines (#1115) (#1140) --- specification/common/common.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index fca4b007a1..3a759e09e0 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -7,6 +7,9 @@ Table of Contents - [Attributes](#attributes) - [Attribute and Label Naming](#attribute-and-label-naming) + - [Name Pluralization Guidelines](#name-pluralization-guidelines) + - [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) + - [Recommendations for Application Developers](#recommendations-for-application-developers) @@ -72,6 +75,19 @@ Names SHOULD follow these rules: name prohibits existence of an equally named namespace in the future, and vice versa: any existing namespace prohibits existence of an equally named attribute or label key in the future. + +### Name Pluralization guidelines + +- When an attribute represents a single entity, the attribute name SHOULD be singular. + Examples: `host.name`, `db.user`, `container.id`. + +- When attribute can represent multiple entities, the attribute name SHOULD be pluralized + and the value type SHOULD be an array. E.g. `process.command_args` might include multiple + values: the executable name and command arguments. + +- When an attribute represents a measurement, + [Metric Name Pluralization Guidelines](../metrics/semantic_conventions/README.md#pluralization) + SHOULD be followed for the attribute name. ### Recommendations for OpenTelemetry Authors From 82b14e151c25401d0b6c660b67e1f7da32914ea3 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Tue, 10 Nov 2020 15:51:16 -0500 Subject: [PATCH 074/124] Add guidance on when to use and not use nested namespaces (#1197) This topic has come up at least 3 times now. I believe a clarification is warranted. The clarification is aligned with previous recommendations: https://github.com/open-telemetry/opentelemetry-specification/pull/789#issuecomment-675506855 https://github.com/open-telemetry/opentelemetry-specification/pull/882#issuecomment-690945900 https://github.com/open-telemetry/opentelemetry-specification/pull/1194#issuecomment-722431982 --- specification/common/common.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index 3a759e09e0..15d25304c4 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -63,6 +63,10 @@ Names SHOULD follow these rules: - Namespaces can be nested. For example `telemetry.sdk` is a namespace inside top-level `telemetry` namespace and `telemetry.sdk.name` is an attribute inside `telemetry.sdk` namespace. + Note: the fact that an entity is located within another entity is typically + not a sufficient reason for forming nested namespaces. The purpose of a + namespace is to avoid name clashes, not to indicate entity hierarchies. This + purpose should primarily drive the decision about forming nested namespaces. - For each multi-word dot-delimited component of the attribute name separate the words by underscores (i.e. use snake_case). For example `http.status_code` From ba5580577ab0a7f40001c68f7b1d676b45b74109 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20Neum=C3=BCller?= Date: Wed, 11 Nov 2020 02:15:41 +0100 Subject: [PATCH 075/124] Nulls SHOULD NOT be allowed in arrays. (#1214) * Nulls SHOULD NOT be allowed in arrays. * Fill in CHANGELOG link Co-authored-by: Armin Ruech --- specification/common/common.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index 15d25304c4..f1526b9518 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -31,6 +31,9 @@ processors / exporters. Attribute values of `null` are not valid and attempting to set a `null` value is undefined behavior. +`null` values SHOULD NOT be allowed in arrays. However, if it is impossible to +make sure that no `null` values are accepted +(e.g. in languages that do not have appropriate compile-time type checking), `null` values within arrays MUST be preserved as-is (i.e., passed on to span processors / exporters as `null`). If exporters do not support exporting `null` values, they MAY replace those values by 0, `false`, or empty strings. From 4c23074b10495846f67c449e8648045748203dfd Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Tue, 17 Nov 2020 16:41:56 -0500 Subject: [PATCH 076/124] Declare freeze of Trace API Specification 1.0 (#1121) * Declare freeze of Trace Specification 1.0 We want to freeze Trace specification 1.0 so that we no longer accept substantial changes (unless a fundamental problem is found in the spec). Resolves https://github.com/open-telemetry/opentelemetry-specification/issues/1120 --- .../common/attribute-and-label-naming.md | 130 ++++++++++++++++++ specification/common/common.md | 126 +---------------- 2 files changed, 133 insertions(+), 123 deletions(-) create mode 100644 specification/common/attribute-and-label-naming.md diff --git a/specification/common/attribute-and-label-naming.md b/specification/common/attribute-and-label-naming.md new file mode 100644 index 0000000000..d84eab4c62 --- /dev/null +++ b/specification/common/attribute-and-label-naming.md @@ -0,0 +1,130 @@ +# Attribute and Label Naming + +
+ +Table of Contents + + +- [Name Pluralization Guidelines](#name-pluralization-guidelines) +- [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) +- [Recommendations for Application Developers](#recommendations-for-application-developers) + +
+ +_This section applies to Resource, Span and Log attribute names (also known as +the "attribute keys") and to keys of Metric labels. For brevity within this +section when we use the term "name" without an adjective it is implied to mean +"attribute name or metric label key"._ + +Every name MUST be a valid Unicode sequence. + +_Note: we merely require that the names are represented as Unicode sequences. +This specification does not define how exactly the Unicode sequences are +encoded. The encoding can vary from one programming language to another and from +one wire format to another. Use the idiomatic way to represent Unicode in the +particular programming language or wire format._ + +Names SHOULD follow these rules: + +- Use namespacing to avoid name clashes. Delimit the namespaces using a dot + character. For example `service.version` denotes the service version where + `service` is the namespace and `version` is an attribute in that namespace. + +- Namespaces can be nested. For example `telemetry.sdk` is a namespace inside + top-level `telemetry` namespace and `telemetry.sdk.name` is an attribute + inside `telemetry.sdk` namespace. + Note: the fact that an entity is located within another entity is typically + not a sufficient reason for forming nested namespaces. The purpose of a + namespace is to avoid name clashes, not to indicate entity hierarchies. This + purpose should primarily drive the decision about forming nested namespaces. + +- For each multi-word dot-delimited component of the attribute name separate the + words by underscores (i.e. use snake_case). For example `http.status_code` + denotes the status code in the http namespace. + +- Names SHOULD NOT coincide with namespaces. For example if + `service.instance.id` is an attribute name then it is no longer valid to have + an attribute named `service.instance` because `service.instance` is already a + namespace. Because of this rule be careful when choosing names: every existing + name prohibits existence of an equally named namespace in the future, and vice + versa: any existing namespace prohibits existence of an equally named + attribute or label key in the future. + +## Name Pluralization guidelines + +- When an attribute represents a single entity, the attribute name SHOULD be singular. + Examples: `host.name`, `db.user`, `container.id`. + +- When attribute can represent multiple entities, the attribute name SHOULD be pluralized + and the value type SHOULD be an array. E.g. `process.command_args` might include multiple + values: the executable name and command arguments. + +- When an attribute represents a measurement, + [Metric Name Pluralization Guidelines](../metrics/semantic_conventions/README.md#pluralization) + SHOULD be followed for the attribute name. + +## Recommendations for OpenTelemetry Authors + +- All names that are part of OpenTelemetry semantic conventions SHOULD be part + of a namespace. + +- When coming up with a new semantic convention make sure to check existing + namespaces for + [Resources](../resource/semantic_conventions/README.md), + [Spans](../trace/semantic_conventions/README.md), + and + [Metrics](../metrics/semantic_conventions/README.md) + to see if the new name fits. + +- When a new namespace is necessary consider whether it should be a top-level + namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). + +- Semantic conventions exist for four areas: for Resource, Span and Log + attribute names as well as Metric label keys. In addition, for spans we have + two more areas: Event and Link attribute names. Identical namespaces or names + in all these areas MUST have identical meanings. For example the `http.method` + span attribute name denotes exactly the same concept as the `http.method` + metric label, has the same data type and the same set of possible values (in + both cases it records the value of the HTTP protocol's request method as a + string). + +- Semantic conventions MUST limit names to printable Basic Latin characters + (more precisely to + [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) + subset of Unicode code points). It is recommended to further limit names to + the following Unicode code points: Latin alphabet, Numeric, Underscore, Dot + (as namespace delimiter). + +## Recommendations for Application Developers + +As an application developer when you need to record an attribute or a label +first consult existing semantic conventions for +[Resources](../resource/semantic_conventions/README.md), +[Spans](../trace/semantic_conventions/README.md), +and +[Metrics](../metrics/semantic_conventions/README.md). +If an appropriate name does not exists you will need to come up with a new name. +To do that consider a few options: + +- The name is specific to your company and may be possibly used outside the + company as well. To avoid clashes with names introduced by other companies (in + a distributed system that uses applications from multiple vendors) it is + recommended to prefix the new name by your company's reverse domain name, e.g. + `com.acme.shopname`. + +- The name is specific to your application that will be used internally only. If + you already have an internal company process that helps you to ensure no name + clashes happen then feel free to follow it. Otherwise it is recommended to + prefix the attribute name or label key by your application name, provided that + the application name is reasonably unique within your organization (e.g. + `myuniquemapapp.longitude` is likely fine). Make sure the application name + does not clash with an existing semantic convention namespace. + +- The name may be generally applicable to applications in the industry. In that + case consider submitting a proposal to this specification to add a new name to + the semantic conventions, and if necessary also to add a new namespace. + +It is recommended to limit names to printable Basic Latin characters +(more precisely to +[U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) +subset of Unicode code points). diff --git a/specification/common/common.md b/specification/common/common.md index f1526b9518..51ce946c48 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -1,15 +1,13 @@ # Common specification concepts +**Status**: [Feature-freeze](../document-status.md). +
Table of Contents - [Attributes](#attributes) - - [Attribute and Label Naming](#attribute-and-label-naming) - - [Name Pluralization Guidelines](#name-pluralization-guidelines) - - [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) - - [Recommendations for Application Developers](#recommendations-for-application-developers)
@@ -42,122 +40,4 @@ indices that are kept in sync (e.g., two attributes `header_keys` and `header_va both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). -## Attribute and Label Naming - -_This section applies to Resource, Span and Log attribute names (also known as -the "attribute keys") and to keys of Metric labels. For brevity within this -section when we use the term "name" without an adjective it is implied to mean -"attribute name or metric label key"._ - -Every name MUST be a valid Unicode sequence. - -_Note: we merely require that the names are represented as Unicode sequences. -This specification does not define how exactly the Unicode sequences are -encoded. The encoding can vary from one programming language to another and from -one wire format to another. Use the idiomatic way to represent Unicode in the -particular programming language or wire format._ - -Names SHOULD follow these rules: - -- Use namespacing to avoid name clashes. Delimit the namespaces using a dot - character. For example `service.version` denotes the service version where - `service` is the namespace and `version` is an attribute in that namespace. - -- Namespaces can be nested. For example `telemetry.sdk` is a namespace inside - top-level `telemetry` namespace and `telemetry.sdk.name` is an attribute - inside `telemetry.sdk` namespace. - Note: the fact that an entity is located within another entity is typically - not a sufficient reason for forming nested namespaces. The purpose of a - namespace is to avoid name clashes, not to indicate entity hierarchies. This - purpose should primarily drive the decision about forming nested namespaces. - -- For each multi-word dot-delimited component of the attribute name separate the - words by underscores (i.e. use snake_case). For example `http.status_code` - denotes the status code in the http namespace. - -- Names SHOULD NOT coincide with namespaces. For example if - `service.instance.id` is an attribute name then it is no longer valid to have - an attribute named `service.instance` because `service.instance` is already a - namespace. Because of this rule be careful when choosing names: every existing - name prohibits existence of an equally named namespace in the future, and vice - versa: any existing namespace prohibits existence of an equally named - attribute or label key in the future. - -### Name Pluralization guidelines - -- When an attribute represents a single entity, the attribute name SHOULD be singular. - Examples: `host.name`, `db.user`, `container.id`. - -- When attribute can represent multiple entities, the attribute name SHOULD be pluralized - and the value type SHOULD be an array. E.g. `process.command_args` might include multiple - values: the executable name and command arguments. - -- When an attribute represents a measurement, - [Metric Name Pluralization Guidelines](../metrics/semantic_conventions/README.md#pluralization) - SHOULD be followed for the attribute name. - -### Recommendations for OpenTelemetry Authors - -- All names that are part of OpenTelemetry semantic conventions SHOULD be part - of a namespace. - -- When coming up with a new semantic convention make sure to check existing - namespaces for - [Resources](../resource/semantic_conventions/README.md), - [Spans](../trace/semantic_conventions/README.md), - and - [Metrics](../metrics/semantic_conventions/README.md) - to see if the new name fits. - -- When a new namespace is necessary consider whether it should be a top-level - namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). - -- Semantic conventions exist for four areas: for Resource, Span and Log - attribute names as well as Metric label keys. In addition, for spans we have - two more areas: Event and Link attribute names. Identical namespaces or names - in all these areas MUST have identical meanings. For example the `http.method` - span attribute name denotes exactly the same concept as the `http.method` - metric label, has the same data type and the same set of possible values (in - both cases it records the value of the HTTP protocol's request method as a - string). - -- Semantic conventions MUST limit names to printable Basic Latin characters - (more precisely to - [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) - subset of Unicode code points). It is recommended to further limit names to - the following Unicode code points: Latin alphabet, Numeric, Underscore, Dot - (as namespace delimiter). - -### Recommendations for Application Developers - -As an application developer when you need to record an attribute or a label -first consult existing semantic conventions for -[Resources](../resource/semantic_conventions/README.md), -[Spans](../trace/semantic_conventions/README.md), -and -[Metrics](../metrics/semantic_conventions/README.md). -If an appropriate name does not exists you will need to come up with a new name. -To do that consider a few options: - -- The name is specific to your company and may be possibly used outside the - company as well. To avoid clashes with names introduced by other companies (in - a distributed system that uses applications from multiple vendors) it is - recommended to prefix the new name by your company's reverse domain name, e.g. - `com.acme.shopname`. - -- The name is specific to your application that will be used internally only. If - you already have an internal company process that helps you to ensure no name - clashes happen then feel free to follow it. Otherwise it is recommended to - prefix the attribute name or label key by your application name, provided that - the application name is reasonably unique within your organization (e.g. - `myuniquemapapp.longitude` is likely fine). Make sure the application name - does not clash with an existing semantic convention namespace. - -- The name may be generally applicable to applications in the industry. In that - case consider submitting a proposal to this specification to add a new name to - the semantic conventions, and if necessary also to add a new namespace. - -It is recommended to limit names to printable Basic Latin characters -(more precisely to -[U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) -subset of Unicode code points). +See [Attribute and Label Naming](attribute-and-label-naming.md) for naming guidelines. From 8de36bfb30587f84109db6cc4daa27f9a6e818a9 Mon Sep 17 00:00:00 2001 From: Ted Young Date: Thu, 4 Feb 2021 06:06:46 -0800 Subject: [PATCH 077/124] Add lifecycle statuses to all documents (#1385) --- specification/common/attribute-and-label-naming.md | 2 ++ specification/common/common.md | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/specification/common/attribute-and-label-naming.md b/specification/common/attribute-and-label-naming.md index d84eab4c62..0a9e3356e8 100644 --- a/specification/common/attribute-and-label-naming.md +++ b/specification/common/attribute-and-label-naming.md @@ -1,5 +1,7 @@ # Attribute and Label Naming +**Status**: [Experimental](../document-status.md) +
Table of Contents diff --git a/specification/common/common.md b/specification/common/common.md index 51ce946c48..203a25b912 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -1,6 +1,6 @@ # Common specification concepts -**Status**: [Feature-freeze](../document-status.md). +**Status**: [Stable, Feature-freeze](../document-status.md)
From cb6ead3c7a2eb3e523bf7196a33253b6653ac2d0 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Wed, 28 Apr 2021 10:38:46 -0400 Subject: [PATCH 078/124] Clarify usage of "otel." attribute namespace (#1640) I noticed developers adding their own attributes to this namespace without going through the specification. We need to regulate this namespace through the specification, just like we do it for other semantic conventions. --- specification/common/attribute-and-label-naming.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/specification/common/attribute-and-label-naming.md b/specification/common/attribute-and-label-naming.md index 0a9e3356e8..6ebe80ed2f 100644 --- a/specification/common/attribute-and-label-naming.md +++ b/specification/common/attribute-and-label-naming.md @@ -130,3 +130,17 @@ It is recommended to limit names to printable Basic Latin characters (more precisely to [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) subset of Unicode code points). + +## otel.* Namespace + +Attribute and label names that start with `otel.` are reserved to be defined by +OpenTelemetry specification. These are typically used to express OpenTelemetry +concepts in formats that don't have a corresponding concept. + +For example, the `otel.library.name` attribute is used to record the +instrumentation library name, which is an OpenTelemetry concept that is natively +represented in OTLP, but does not have an equivalent in other telemetry formats +and protocols. + +Any additions to the `otel.*` namespace MUST be approved as part of +OpenTelemetry specification. From c37549e1d8997e07c158219540d115c225f6a50f Mon Sep 17 00:00:00 2001 From: Jakub Malinowski Date: Thu, 1 Jul 2021 00:11:54 +0200 Subject: [PATCH 079/124] Rename Metrics labels to attributes (#1775) --- ...nd-label-naming.md => attribute-naming.md} | 36 +++++++++---------- specification/common/common.md | 2 +- 2 files changed, 18 insertions(+), 20 deletions(-) rename specification/common/{attribute-and-label-naming.md => attribute-naming.md} (84%) diff --git a/specification/common/attribute-and-label-naming.md b/specification/common/attribute-naming.md similarity index 84% rename from specification/common/attribute-and-label-naming.md rename to specification/common/attribute-naming.md index 6ebe80ed2f..6599f6b239 100644 --- a/specification/common/attribute-and-label-naming.md +++ b/specification/common/attribute-naming.md @@ -1,4 +1,4 @@ -# Attribute and Label Naming +# Attribute Naming **Status**: [Experimental](../document-status.md) @@ -13,10 +13,9 @@ Table of Contents
-_This section applies to Resource, Span and Log attribute names (also known as -the "attribute keys") and to keys of Metric labels. For brevity within this -section when we use the term "name" without an adjective it is implied to mean -"attribute name or metric label key"._ +_This section applies to Resource, Span, Log, and Metric attribute names (also +known as the "attribute keys"). For brevity within this section when we use the +term "name" without an adjective it is implied to mean "attribute name"._ Every name MUST be a valid Unicode sequence. @@ -50,8 +49,8 @@ Names SHOULD follow these rules: namespace. Because of this rule be careful when choosing names: every existing name prohibits existence of an equally named namespace in the future, and vice versa: any existing namespace prohibits existence of an equally named - attribute or label key in the future. - + attribute key in the future. + ## Name Pluralization guidelines - When an attribute represents a single entity, the attribute name SHOULD be singular. @@ -81,14 +80,13 @@ Names SHOULD follow these rules: - When a new namespace is necessary consider whether it should be a top-level namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). -- Semantic conventions exist for four areas: for Resource, Span and Log - attribute names as well as Metric label keys. In addition, for spans we have - two more areas: Event and Link attribute names. Identical namespaces or names - in all these areas MUST have identical meanings. For example the `http.method` - span attribute name denotes exactly the same concept as the `http.method` - metric label, has the same data type and the same set of possible values (in - both cases it records the value of the HTTP protocol's request method as a - string). +- Semantic conventions exist for four areas: for Resource, Span, Log, and Metric + attribute names. In addition, for spans we have two more areas: Event and Link + attribute names. Identical namespaces or names in all these areas MUST have + identical meanings. For example the `http.method` span attribute name denotes + exactly the same concept as the `http.method` metric attribute, has the same + data type and the same set of possible values (in both cases it records the + value of the HTTP protocol's request method as a string). - Semantic conventions MUST limit names to printable Basic Latin characters (more precisely to @@ -99,8 +97,8 @@ Names SHOULD follow these rules: ## Recommendations for Application Developers -As an application developer when you need to record an attribute or a label -first consult existing semantic conventions for +As an application developer when you need to record an attribute first consult +existing semantic conventions for [Resources](../resource/semantic_conventions/README.md), [Spans](../trace/semantic_conventions/README.md), and @@ -117,7 +115,7 @@ To do that consider a few options: - The name is specific to your application that will be used internally only. If you already have an internal company process that helps you to ensure no name clashes happen then feel free to follow it. Otherwise it is recommended to - prefix the attribute name or label key by your application name, provided that + prefix the attribute name by your application name, provided that the application name is reasonably unique within your organization (e.g. `myuniquemapapp.longitude` is likely fine). Make sure the application name does not clash with an existing semantic convention namespace. @@ -133,7 +131,7 @@ subset of Unicode code points). ## otel.* Namespace -Attribute and label names that start with `otel.` are reserved to be defined by +Attribute names that start with `otel.` are reserved to be defined by OpenTelemetry specification. These are typically used to express OpenTelemetry concepts in formats that don't have a corresponding concept. diff --git a/specification/common/common.md b/specification/common/common.md index 203a25b912..1b34c4f9f8 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -40,4 +40,4 @@ indices that are kept in sync (e.g., two attributes `header_keys` and `header_va both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). -See [Attribute and Label Naming](attribute-and-label-naming.md) for naming guidelines. +See [Attribute Naming](attribute-naming.md) for naming guidelines. From 43e0a08c24a2eef9894c0086e1fe404202b7b245 Mon Sep 17 00:00:00 2001 From: Jakub Malinowski Date: Wed, 4 Aug 2021 00:26:10 +0200 Subject: [PATCH 080/124] Add an option to limit length of values of attributes and metric values (#1130) --- specification/common/common.md | 51 ++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index 1b34c4f9f8..6cb26c12f9 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -8,6 +8,8 @@ Table of Contents - [Attributes](#attributes) + - [Attribute Limits](#attribute-limits) + - [Exempt Entities](#exempt-entities)
@@ -41,3 +43,52 @@ both containing an array of strings to represent a mapping `header_keys[i] -> header_values[i]`). See [Attribute Naming](attribute-naming.md) for naming guidelines. + +### Attribute Limits + +Execution of erroneous code can result in unintended attributes. If there are no +limits placed on attributes, they can quickly exhaust available memory, resulting +in crashes that are difficult to recover from safely. + +By default an SDK SHOULD apply truncation as per the list of +[configurable parameters](#attribute-limits-configuration) below. + +If an SDK provides a way to: + +- set an attribute value length limit such that for each + attribute value: + - if it is a string, if it exceeds that limit (counting any character in it as + 1), SDKs MUST truncate that value, so that its length is at most equal + to the limit, + - if it is an array of strings, then apply the above rule to each of the + values separately, + - otherwise a value MUST NOT be truncated; +- set a limit of unique attribute keys such that: + - for each unique attributes key, addition of which would result in exceeding + the limit, SDK MUST discard that key/value pair. + +There MAY be a log emitted to indicate to the user that an attribute was +truncated or discarded. To prevent excessive logging, the log MUST NOT be +emitted more than once per record on which an attribute is set. + +If the SDK implements the limits above, it MUST provide a way to change these +limits programmatically. Names of the configuration options SHOULD be the same as +in the list below. + +An SDK MAY implement model-specific limits, for example +`SpanAttributeCountLimit`. If both a general and a model-specific limit are +implemented, then the SDK MUST first attempt to use the model-specific limit, if +it isn't set and doesn't have a default, then the SDK MUST attempt to use the +general limit. + + +**Configurable parameters:** + +* `AttributeCountLimit` (Default=128) - Maximum allowed attribute count per record; +* `AttributeValueLengthLimit` (Default=Infinity) - Maximum allowed attribute value length; + +#### Exempt Entities + +Attributes, which belong to Metrics, are exempt from the limits described above +at this time, as discussed in +[Metrics Attribute Limits](../metrics/sdk.md#attribute-limits). From 36ebbcad536af547f8f0da1a010433f7823cf276 Mon Sep 17 00:00:00 2001 From: Owais Lone Date: Tue, 14 Sep 2021 16:40:45 +0530 Subject: [PATCH 081/124] Prefer global limit over model-specific default (#1893) --- specification/common/common.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 6cb26c12f9..c3f434aa89 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -78,8 +78,9 @@ in the list below. An SDK MAY implement model-specific limits, for example `SpanAttributeCountLimit`. If both a general and a model-specific limit are implemented, then the SDK MUST first attempt to use the model-specific limit, if -it isn't set and doesn't have a default, then the SDK MUST attempt to use the -general limit. +it isn't set, then the SDK MUST attempt to use the general limit. If neither are +defined, then the SDK MUST try to use the model-specific limit default value, +followed by the global limit default value. **Configurable parameters:** From 19e4040e36a8d5d10459baf339879eefe51b2919 Mon Sep 17 00:00:00 2001 From: Owais Lone Date: Thu, 23 Sep 2021 05:19:00 +0530 Subject: [PATCH 082/124] Exempt resources from attribute limits (#1892) Resources are not susceptible to scenarios where excessive attributes can be recorded unlike Spans. Resources are also immutable and it can be hard for some SDKs to apply the limits at source at the time the attributes are added to a resource. Furthermore, limits and Resources both are generally defined and passed on to a TracerProvider which forces a TracerProvider to either mutate the resource or generate a new one with duplicate attributes in order to apply the limits to it. Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> --- specification/common/common.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index c3f434aa89..b0779ad652 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -90,6 +90,14 @@ followed by the global limit default value. #### Exempt Entities +Resource attributes SHOULD be exempt from the limits described above as resources +are not susceptible to the scenarios (auto-instrumentation) that result in +excessive attributes count or size. Resources are also sent only once per batch +instead of per span so it is relatively cheaper to have more/larger attributes +on them. Resources are also immutable by design and they are generally passed +down to TracerProvider along with limits. This makes it awkward to implement +attribute limits for Resources. + Attributes, which belong to Metrics, are exempt from the limits described above at this time, as discussed in [Metrics Attribute Limits](../metrics/sdk.md#attribute-limits). From 6c59f11a4aef0ee0506a4f6cda2e18cea9f33b23 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Tue, 16 Nov 2021 10:05:01 -0500 Subject: [PATCH 083/124] Provide a normative definition of Attribute (singular) rather than Attributes (plural) (#2123) * Provide a normative definition of Attribute (singular) - Providing a normative definition of **attribute** (singular) - Other copyedits /cc @austinlparker @tedsuo * Move anchor to make markdown checker happy --- specification/common/common.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index b0779ad652..19d3d87eb7 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -7,21 +7,23 @@ Table of Contents -- [Attributes](#attributes) +- [Attribute](#attribute) - [Attribute Limits](#attribute-limits) - [Exempt Entities](#exempt-entities) -## Attributes +## Attribute -Attributes are a list of zero or more key-value pairs. An `Attribute` MUST have the following properties: + -- The attribute key, which MUST be a non-`null` and non-empty string. -- The attribute value, which is either: +An `Attribute` is a key-value pair, which MUST have the following properties: + +- The attribute key MUST be a non-`null` and non-empty string. +- The attribute value is either: - A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer. - An array of primitive type values. The array MUST be homogeneous, - i.e. it MUST NOT contain values of different types. For protocols that do + i.e., it MUST NOT contain values of different types. For protocols that do not natively support array values such values SHOULD be represented as JSON strings. Attribute values expressing a numerical value of zero, an empty string, or an @@ -64,7 +66,7 @@ If an SDK provides a way to: values separately, - otherwise a value MUST NOT be truncated; - set a limit of unique attribute keys such that: - - for each unique attributes key, addition of which would result in exceeding + - for each unique attribute key, addition of which would result in exceeding the limit, SDK MUST discard that key/value pair. There MAY be a log emitted to indicate to the user that an attribute was From 70eaeee2aa18d7fae80e01038fade6a0791c4fe1 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Wed, 24 Nov 2021 09:57:38 -0500 Subject: [PATCH 084/124] Ensure all ToCs are generated using markdown-toc (#2146) --- specification/common/attribute-naming.md | 11 +++++++---- specification/common/common.md | 12 +++++++----- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index 6599f6b239..1cf593b9f7 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -3,13 +3,16 @@ **Status**: [Experimental](../document-status.md)
- -Table of Contents - +Table of Contents -- [Name Pluralization Guidelines](#name-pluralization-guidelines) + + +- [Name Pluralization guidelines](#name-pluralization-guidelines) - [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) - [Recommendations for Application Developers](#recommendations-for-application-developers) +- [otel.* Namespace](#otel-namespace) + +
diff --git a/specification/common/common.md b/specification/common/common.md index 19d3d87eb7..3e5f170456 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -3,13 +3,15 @@ **Status**: [Stable, Feature-freeze](../document-status.md)
- -Table of Contents - +Table of Contents + + - [Attribute](#attribute) - - [Attribute Limits](#attribute-limits) - - [Exempt Entities](#exempt-entities) + * [Attribute Limits](#attribute-limits) + + [Exempt Entities](#exempt-entities) + +
From 22b6b073e3c50eb0adeac22fde7d61d465cd1f76 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Mon, 13 Dec 2021 13:04:41 -0500 Subject: [PATCH 085/124] Prohibit usage of retired names in semantic conventions (#2191) * Prohibit usage of retired names in semantic conventions This change adds a prohibition clause that requires that no old metric or attribute name is used for a new attribute. This is important to ensure reversibility of schema transformation (converting from a new version to an old version of schema). Without this restriction the following is possible: Schema version 1. Attribute A exists. Schema version 2. Attribute A is renamed to B. Appropriate schema file is created. Schema version 3. Attribute A is introduced (a completely different new attribute). Now attempting to go from Version 3 to version 1 is impossible since it requires renaming B to A (for the change in version 2), but a different attribute A already exists. * Fix based on comments * Add changelog entry Co-authored-by: Carlos Alberto Cortez --- specification/common/attribute-naming.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index 1cf593b9f7..9cd2f187b5 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -8,6 +8,7 @@ - [Name Pluralization guidelines](#name-pluralization-guidelines) +- [Name Reuse Prohibition](#name-reuse-prohibition) - [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) - [Recommendations for Application Developers](#recommendations-for-application-developers) - [otel.* Namespace](#otel-namespace) @@ -67,6 +68,15 @@ Names SHOULD follow these rules: [Metric Name Pluralization Guidelines](../metrics/semantic_conventions/README.md#pluralization) SHOULD be followed for the attribute name. +## Name Reuse Prohibition + +A new attribute MUST NOT be added with the same name as an attribute that +existed in the past but was renamed (with a corresponding schema file). + +When introducing a new attribute name check all existing schema files to make +sure the name does not appear as a key of any "rename_attributes" section (keys +denote old attribute names in rename operations). + ## Recommendations for OpenTelemetry Authors - All names that are part of OpenTelemetry semantic conventions SHOULD be part From c948e38726d43e8ddbbd10d95f111118fe9d0522 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Wed, 26 Jan 2022 16:47:43 -0500 Subject: [PATCH 086/124] Clarify that attribute keys are unique in collections (#2248) Attributes keys must be unique. The key/value pair collections in the specification was always intended to model a map. There was a recent confusion about this. This change clarifies the spec. Resolves https://github.com/open-telemetry/opentelemetry-specification/issues/2245 --- specification/common/common.md | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/specification/common/common.md b/specification/common/common.md index 3e5f170456..f098255a22 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -10,6 +10,7 @@ - [Attribute](#attribute) * [Attribute Limits](#attribute-limits) + [Exempt Entities](#exempt-entities) +- [Attribute Collections](#attribute-collections) @@ -105,3 +106,35 @@ attribute limits for Resources. Attributes, which belong to Metrics, are exempt from the limits described above at this time, as discussed in [Metrics Attribute Limits](../metrics/sdk.md#attribute-limits). + +## Attribute Collections + +[Resources](../resource/sdk.md), Metrics +[data points](../metrics/datamodel.md#metric-points), +[Spans](../trace/api.md#set-attributes), Span +[Events](../trace/api.md#add-events), Span +[Links](../trace/api.md#specifying-links) and +[Log Records](../logs/data-model.md) may contain a collection of attributes. The +keys in each such collection are unique, i.e. there MUST NOT exist more than one +key-value pair with the same key. The enforcement of uniqueness may be performed +in a variety of ways as it best fits the limitations of the particular +implementation. + +Normally for the telemetry generated using OpenTelemetry SDKs the attribute +key-value pairs are set via an API that either accepts a single key-value pair +or a collection of key-value pairs. Setting an attribute with the same key as an +existing attribute SHOULD overwrite the existing attribute's value. See for +example Span's [SetAttribute](../trace/api.md#set-attributes) API. + +A typical implementation of [SetAttribute](../trace/api.md#set-attributes) API +will enforce the uniqueness by overwriting any existing attribute values pending +to be exported, so that when the Span is eventually exported the exporters see +only unique attributes. The OTLP format in particular requires that exported +Resources, Spans, Metric data points and Log Records contain only unique +attributes. + +Some other implementations may use a streaming approach where every +[SetAttribute](../trace/api.md#set-attributes) API call immediately results in +that individual attribute value being exported using a streaming wire protocol. +In such cases the enforcement of uniqueness will likely be the responsibility of +the recipient of this data. From 132fd6aa149ca1fe2e3cc0e89e65260922f8b31c Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Tue, 15 Feb 2022 11:34:11 -0500 Subject: [PATCH 087/124] Clarify that Trace/Meter are associated with Instrumentation Scope (#2276) * Clarify that Trace/Meter are associated with Instrumentation Scope This is a slightly different take on https://github.com/open-telemetry/opentelemetry-specification/issues/2203 Instead of renaming instrumentation library to instrumentation scope everywhere this PR suggests targetted editing of wording of the Trace/Meter obtaining API to allow not just instrumentation library but other instrumentation scopes to be used as a parameter. This change does not force renaming of API parameters and is not a breaking change. We consider it a clarification of usage to match the intent that we had for the "name" field. If this PR is accepted there will be a follow up PR that will suggest to rename InstrumentationLibrary* messages in OTLP proto to InstrumentationScope* message. Such a change will not be a breaking change for the OTLP wire format and is acceptable. If this PR is accepted we will also close https://github.com/open-telemetry/opentelemetry-specification/pull/1236 since it will be no longer necessary. The logger name will be recorded as the instrumentation scope. This clarification will be a follow up PR that clarifies the behavior in https://github.com/open-telemetry/oteps/blob/main/text/logs/0150-logging-library-sdk.md --- specification/common/attribute-naming.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index 9cd2f187b5..fecbf3b21b 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -148,8 +148,8 @@ Attribute names that start with `otel.` are reserved to be defined by OpenTelemetry specification. These are typically used to express OpenTelemetry concepts in formats that don't have a corresponding concept. -For example, the `otel.library.name` attribute is used to record the -instrumentation library name, which is an OpenTelemetry concept that is natively +For example, the `otel.scope.name` attribute is used to record the +instrumentation scope name, which is an OpenTelemetry concept that is natively represented in OTLP, but does not have an equivalent in other telemetry formats and protocols. From 5f93caf5bbe8a04a203ccc1158b600baa5ff0f8e Mon Sep 17 00:00:00 2001 From: David Ashpole Date: Thu, 3 Mar 2022 00:30:53 -0500 Subject: [PATCH 088/124] Describe how to convert non-string primitives for protocols which only support strings (#2343) * Describe how to handle converting non-string primitives for protocols that only support strings * update wording to make clear that only non-string values are converted to strings * unify language * Update specification/common/common.md Co-authored-by: Joshua MacDonald Co-authored-by: Joshua MacDonald Co-authored-by: Bogdan Drutu --- specification/common/common.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index f098255a22..80a13012f4 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -26,8 +26,9 @@ An `Attribute` is a key-value pair, which MUST have the following properties: - The attribute value is either: - A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer. - An array of primitive type values. The array MUST be homogeneous, - i.e., it MUST NOT contain values of different types. For protocols that do - not natively support array values such values SHOULD be represented as JSON strings. + i.e., it MUST NOT contain values of different types. + +For protocols that do not natively support non-string values, non-string values SHOULD be represented as JSON-encoded strings. For example, the expression `int64(100)` will be encoded as `100`, `float64(1.5)` will be encoded as `1.5`, and an empty array of any type will be encoded as `[]`. Attribute values expressing a numerical value of zero, an empty string, or an empty array are considered meaningful and MUST be stored and passed on to From ec4cee89aef97ffa7f807fc7dcc04d1046e52c7c Mon Sep 17 00:00:00 2001 From: Reiley Yang Date: Mon, 21 Mar 2022 23:05:45 -0700 Subject: [PATCH 089/124] Fix links (#2426) --- specification/common/common.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specification/common/common.md b/specification/common/common.md index 80a13012f4..774c236130 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -9,6 +9,7 @@ - [Attribute](#attribute) * [Attribute Limits](#attribute-limits) + + [Configurable Parameters](#configurable-parameters) + [Exempt Entities](#exempt-entities) - [Attribute Collections](#attribute-collections) @@ -57,7 +58,7 @@ limits placed on attributes, they can quickly exhaust available memory, resultin in crashes that are difficult to recover from safely. By default an SDK SHOULD apply truncation as per the list of -[configurable parameters](#attribute-limits-configuration) below. +[configurable parameters](#configurable-parameters) below. If an SDK provides a way to: @@ -88,8 +89,7 @@ it isn't set, then the SDK MUST attempt to use the general limit. If neither are defined, then the SDK MUST try to use the model-specific limit default value, followed by the global limit default value. - -**Configurable parameters:** +#### Configurable Parameters * `AttributeCountLimit` (Default=128) - Maximum allowed attribute count per record; * `AttributeValueLengthLimit` (Default=Infinity) - Maximum allowed attribute value length; From b5306f3db2de7b46a746bbba4f3893bc5cb6f3f5 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Tue, 22 Mar 2022 08:59:37 -0400 Subject: [PATCH 090/124] Implement OTEP 0178: Mapping external data to AnyValue (#2385) OTEP 0178 [0] proposed this mapping. This PR merges the proposal into specification. The content of this PR is mostly copy/paste of the text of the OTEP minus the irrelevant sections such as "Alternates", etc. 0 - https://github.com/open-telemetry/oteps/blob/main/text/0178-mapping-to-otlp-anyvalue.md --- .../common/attribute-type-mapping.md | 255 ++++++++++++++++++ specification/common/common.md | 3 + 2 files changed, 258 insertions(+) create mode 100644 specification/common/attribute-type-mapping.md diff --git a/specification/common/attribute-type-mapping.md b/specification/common/attribute-type-mapping.md new file mode 100644 index 0000000000..46a25c1504 --- /dev/null +++ b/specification/common/attribute-type-mapping.md @@ -0,0 +1,255 @@ +# Mapping Arbitrary Data to OTLP AnyValue + +**Status**: [Experimental](../document-status.md) + +
+Table of Contents + + + +- [Converting to AnyValue](#converting-to-anyvalue) + * [Primitive Values](#primitive-values) + + [Integer Values](#integer-values) + + [Enumerations](#enumerations) + + [Floating Point Values](#floating-point-values) + + [String Values](#string-values) + + [Byte Sequences](#byte-sequences) + * [Composite Values](#composite-values) + + [Array Values](#array-values) + + [Associative Arrays With Unique Keys](#associative-arrays-with-unique-keys) + + [Associative Arrays With Non-Unique Keys](#associative-arrays-with-non-unique-keys) + + [Sets](#sets) + * [Other Values](#other-values) + * [Empty Values](#empty-values) + + + +
+ +This document defines how to map (convert) arbitrary data (e.g. in-memory +objects) to OTLP's [AnyValue](https://github.com/open-telemetry/opentelemetry-proto/blob/cc4ed55c082cb75e084d40b4ddf3805eda099f97/opentelemetry/proto/common/v1/common.proto#L27). + +The mapping is needed when OpenTelemetry needs to convert a value produced outside +OpenTelemetry into a value that can be exported using an OTLP exporter, or otherwise be +converted to be used inside OpenTelemetry boundaries. Example use cases are the following: + +- In [Logging Library SDK](../logs/logging-library-sdk.md)s, to convert values received + from logging libraries into OpenTelemetry representation. +- In the Collector, to convert values received from various data sources into + [pdata](https://github.com/open-telemetry/opentelemetry-collector/blob/4998703dadd19fa91a88eabf7ccc68d728bee3fd/model/pdata/common.go#L84) + internal representation. + +## Converting to AnyValue + +[AnyValue](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L27) +is capable of representing primitive and structured data of certain types. + +Implementations that have source data in any form, such as in-memory objects +or data coming from other formats that needs to be converted to AnyValue SHOULD +follow the rules described below. + +### Primitive Values + +#### Integer Values + +Integer values which are within the range of 64 bit signed numbers +[-2^63..2^63-1] SHOULD be converted to AnyValue's +[int_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L33) +field. + +Integer values which are outside the range of 64 bit signed numbers SHOULD be +converted to AnyValue's +[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) +field using decimal representation. + +#### Enumerations + +Values, which belong to a limited enumerated set (e.g. a Java +[enum](https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html)), SHOULD be +converted to AnyValue's +[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) +field with the value of the string set to the symbolic name of the enumeration. + +If it is not possible to obtain the symbolic name of the enumeration, the +implementation SHOULD map enumeration values to AnyValue's +[int_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L33) +field set to the enum's ordinal value, when such an ordinal number is naturally +obtainable. + +If it is also not possible to obtain the ordinal value, the enumeration SHOULD be +converted to AnyValue's +[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) +field in any manner that the implementation deems reasonable. + +#### Floating Point Values + +Floating point values which are within the range and precision of IEEE 754 +64-bit floating point numbers (including IEEE 32-bit floating point values) +SHOULD be converted to AnyValue's +[double_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L34) +field. + +Floating point values which are outside the range or precision of IEEE 754 +64-bit floating point numbers (e.g. IEEE 128-bit floating bit values) SHOULD be +converted to AnyValue's +[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) +field using decimal floating point representation. + +#### String Values + +String values which are valid UTF-8 sequences SHOULD be converted to +AnyValue's +[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) +field. + +String values which are not valid Unicode sequences SHOULD be converted to +AnyValue's +[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) +with the bytes representing the string in the original order and format of the +source string. + +#### Byte Sequences + +Byte sequences (e.g. Go's `[]byte` slice or raw byte content of a file) SHOULD +be converted to AnyValue's +[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) +field. + +### Composite Values + +#### Array Values + +Values that represent ordered sequences of other values (such as +[arrays](https://docs.oracle.com/javase/specs/jls/se7/html/jls-10.html), +[vectors](https://en.cppreference.com/w/cpp/container/vector), ordered +[lists](https://docs.python.org/3/tutorial/datastructures.html#more-on-lists), +[slices](https://golang.org/ref/spec#Slice_types)) SHOULD be converted to +AnyValue's +[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) +field. String values and byte sequences are an exception from this rule (see +above). + +The rules described in this document should be applied recursively to each element +of the array. + +#### Associative Arrays With Unique Keys + +Values that represent associative arrays with unique keys (also often known +as maps, dictionaries or key-value stores) SHOULD be converted to AnyValue's +[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) +field. + +If the keys of the source array are not strings, they MUST be converted to +strings by any means available, often via a toString() or stringify functions +available in programming languages. The conversion function MUST be chosen in a +way that ensures that the resulting string keys are unique in the target array. + +The value part of each element of the source array SHOULD be converted to +AnyValue recursively. + +For example a JSON object `{"a": 123, "b": "def"}` SHOULD be converted to + +``` +AnyValue{ + kvlist_value:KeyValueList{ + values:[ + KeyValue{key:"a",value:AnyValue{int_value:123}}, + KeyValue{key:"b",value:AnyValue{string_value:"def"}}, + ] + } +} +``` + +The rules described in this document should be applied recursively to each value +of the associative array. + +#### Associative Arrays With Non-Unique Keys + +Values that represent an associative arrays with non-unique keys where multiple values may be associated with the same key (also sometimes known +as multimaps, multidicts) SHOULD be converted to AnyValue's +[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) +field. + +The resulting +[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) +field MUST list each key only once and the value of each element of +[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) +field MUST be an array represented using AnyValue's +[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) +field, each element of the +[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) +representing one value of source array associated with the given key. + +For example an associative array shown in the following table: + +|Key|Value| +|---|---| +|"abc"|123| +|"def"|"foo"| +|"def"|"bar"| + +SHOULD be converted to: + +``` +AnyValue{ + kvlist_value:KeyValueList{ + values:[ + KeyValue{ + key:"abc", + value:AnyValue{array_value:ArrayValue{values[ + AnyValue{int_value:123} + ]}} + }, + KeyValue{ + key:"def", + value:AnyValue{array_value:ArrayValue{values[ + AnyValue{string_value:"foo"}, + AnyValue{string_value:"bar"} + ]}} + }, + ] + } +} +``` + +The rules described in this document should be applied recursively to each value +of the associative array. + +#### Sets + +Unordered collections of unique values (such as +[Java Sets](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Set.html), +[C++ sets](https://en.cppreference.com/w/cpp/container/set), +[Python Sets](https://docs.python.org/3/tutorial/datastructures.html#sets)) SHOULD be +converted to AnyValue's +[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) +field, where each element of the set becomes an element of the array. + +The rules described in this document should be applied recursively to each value +of the set. + +### Other Values + +Any other values not listed above SHOULD be converted to AnyValue's +[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) +field if the source data can be serialized to a string (can be stringified) +using toString() or stringify functions available in programming languages. + +If the source data cannot be serialized to a string then the value SHOULD be +converted AnyValue's +[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) +field by serializing it into a byte sequence by any means available. + +If the source data cannot be serialized neither to a string nor to a byte +sequence then it SHOULD by converted to an empty AnyValue. + +### Empty Values + +If the source data has no type associated with it and is empty, null, nil or +otherwise indicates absence of data it SHOULD be converted to an +[empty](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L29) +AnyValue, where all the fields are unset. + +Empty values which has a type associated with them (e.g. empty associative +array) SHOULD be converted using the corresponding rules defined for the types +above. diff --git a/specification/common/common.md b/specification/common/common.md index 774c236130..6c3e80a527 100644 --- a/specification/common/common.md +++ b/specification/common/common.md @@ -51,6 +51,9 @@ both containing an array of strings to represent a mapping See [Attribute Naming](attribute-naming.md) for naming guidelines. +See [this document](attribute-type-mapping.md) to find out how to map values obtained +outside OpenTelemetry into OpenTelemetry attribute values. + ### Attribute Limits Execution of erroneous code can result in unintended attributes. If there are no From 97d6ebffccafadacb9b64a8f7a7bab6916b1ff3d Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Fri, 8 Apr 2022 13:07:21 -0400 Subject: [PATCH 091/124] Ensure common section has a README (#2479) --- specification/common/{common.md => README.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename specification/common/{common.md => README.md} (100%) diff --git a/specification/common/common.md b/specification/common/README.md similarity index 100% rename from specification/common/common.md rename to specification/common/README.md From 37675886662f0f200ca6f3c4090fddb64f34ce93 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 16 May 2022 10:16:33 -0700 Subject: [PATCH 092/124] Define attribute requirement levels (#2522) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * nits * review comments * No exceptions for Required attributes, clarifications on performance * Conditional clarifications * nits * Conditional -> required conditionally and minor fixes * Align requirement levels with RFC levels better * Clarify Note on required attributes * Update specification/common/attribute-requirement-level.md Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> * Clarify Note on required attributes * Remove performance from conditionally required attributes * Clarify Conditionally Required case when condition is false * Apply suggestions from code review Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Christian Neumüller * headings for levels and moving things around * nits: formatting Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Co-authored-by: Christian Neumüller Co-authored-by: Reiley Yang --- specification/common/README.md | 2 + .../common/attribute-requirement-level.md | 69 +++++++++++++++++++ 2 files changed, 71 insertions(+) create mode 100644 specification/common/attribute-requirement-level.md diff --git a/specification/common/README.md b/specification/common/README.md index 6c3e80a527..18d46fe00c 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -51,6 +51,8 @@ both containing an array of strings to represent a mapping See [Attribute Naming](attribute-naming.md) for naming guidelines. +See [Requirement Level](attribute-requirement-level.md) for requirement levels guidelines. + See [this document](attribute-type-mapping.md) to find out how to map values obtained outside OpenTelemetry into OpenTelemetry attribute values. diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md new file mode 100644 index 0000000000..5a7d6e323d --- /dev/null +++ b/specification/common/attribute-requirement-level.md @@ -0,0 +1,69 @@ +# Attribute Requirement Levels for Semantic Conventions + +**Status**: [Experimental](../document-status.md) + +
+Table of Contents + + + +- [Required](#required) +- [Conditionally Required](#conditionally-required) +- [Recommended](#recommended) +- [Optional](#optional) +- [Performance suggestions](#performance-suggestions) + + + +
+ +_This section applies to Log, Metric, Resource, and Span, and describes requirement levels for attributes defined in semantic conventions._ + +The following attribute requirement levels are specified: + +- [Required](#required) +- [Conditionally Required](#conditionally-required) +- [Recommended](#recommended) +- [Optional](#optional) + +The requirement level for attribute is defined by semantic conventions depending on attribute availability across instrumented entities, performance, security, and other factors. When defining requirement levels, semantic conventions MUST take into account signal-specific requirements. + +For example, Metric attributes that may have high cardinality can only be defined with `Optional` level. + +Semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. + +For example, [Database semantic convention](../trace/semantic_conventions/database.md) references `net.transport` attribute defined in [General attributes](../trace/semantic_conventions/span-general.md) with `Conditionally Required` level on it. + +## Required + +All instrumentations MUST populate the attribute. Semantic convention defining a Required attribute expects that an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, can ensure cardinality, security, and other requirements specific to signal defined by the convention. `http.method` is an example of a Required attribute. + +_Note: Consumers of telemetry can detect if telemetry item follows a specific semantic convention by checking the presence of a `Required` attribute defined by such convention. For example, the presence of `db.system` attribute on a span can be used as an indication that the span follows database semantics._ + +## Conditionally Required + +All instrumentations MUST add the attribute when given condition is satisfied. Semantic convention of a `Conditionally Required` level of an attribute MUST clarify the condition under which the attribute is expected to be populated. + +`http.route` is an example of a conditionally required attribute to be populated when instrumented HTTP framework provides route information for the instrumented request. Some low-level HTTP server implementations do not support routing and corresponding instrumentations can't populate the attribute. + +When the condition on `Conditionally Required` attribute is not satisfied and there is no requirement to populate attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Optional` requirement level on the attribute. + +For example, `net.peer.name` is `Conditionally Required` by [Database convention](../trace/semantic_conventions/database.md) when available. When only `net.peer.ip` is available, instrumentation can do a DNS lookup, cache and populate `net.peer.name` but only if user explicitly enables instrumentation to do so, considering performance issues the DNS lookup introduces. + +## Recommended + +Instrumentations SHOULD add the attribute by default if it's readily available and can be [efficiently populated](#performance-suggestions). Instrumentations MAY offer a configuration option to disable Recommended attributes. + +Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD use the `Optional` requirement level on them if the attributes are logically applicable. + +## Optional + +Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Optional` attributes. + +## Performance suggestions + +Here are several examples of expensive operations to be avoided by default: + +- DNS lookup to populate `net.peer.name` if only IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. +- forcing `http.route` calculation before HTTP framework calculates it +- reading response stream to find `http.response_content_length` when `Content-Length` header is not available From 8781d7df056c4f8f59f653d98c0ef35079077ef5 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Fri, 20 May 2022 02:12:03 -0400 Subject: [PATCH 093/124] Add missing READMEs to section (#2559) --- specification/common/README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/specification/common/README.md b/specification/common/README.md index 18d46fe00c..3d1379fde4 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -1,3 +1,6 @@ + # Common specification concepts **Status**: [Stable, Feature-freeze](../document-status.md) From 9eaea0af14eae15dbd02a5823efa131cf97bac2c Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Thu, 2 Jun 2022 03:25:38 -0400 Subject: [PATCH 094/124] Move non-otlp.md to common directory (#2587) --- specification/common/mapping-to-non-otlp.md | 76 +++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 specification/common/mapping-to-non-otlp.md diff --git a/specification/common/mapping-to-non-otlp.md b/specification/common/mapping-to-non-otlp.md new file mode 100644 index 0000000000..cb80f685ca --- /dev/null +++ b/specification/common/mapping-to-non-otlp.md @@ -0,0 +1,76 @@ +# OpenTelemetry Transformation to non-OTLP Formats + +**Status**: [Stable](../document-status.md) + +All OpenTelemetry concepts and data recorded using OpenTelemetry API can be +directly and precisely represented using corresponding messages and fields of +OTLP format. However, for other formats this is not always the case. Sometimes a +format will not have a native way to represent a particular OpenTelemetry +concept or a field of a concept. + +This document defines the transformation between OpenTelemetry and formats other +than OTLP, for OpenTelemetry fields and concepts that have no direct semantic +equivalent in those other formats. + +Note: when a format has a direct semantic equivalent for a particular field or +concept then the recommendation in this document MUST be ignored. + +See also additional specific transformation rules for +[Jaeger](../trace/sdk_exporters/jaeger.md) and [Zipkin](../trace/sdk_exporters/zipkin.md). +The specific rules for Jaeger and Zipkin take precedence over the generic rules defined +in this document. + +## Mappings + +### InstrumentationScope + +OpenTelemetry `InstrumentationScope`'s fields MUST be reported as key-value +pairs associated with the Span, Metric Data Point or LogRecord using the following mapping: + +| OpenTelemetry InstrumentationScope Field | non-OTLP Key | Notes | +| ------------------- | --- | --- | +| `InstrumentationScope.name`|`otel.scope.name`|since 1.10.0| +| `InstrumentationScope.version`|`otel.scope.version`|since 1.10.0| + +The following deprecated aliases MUST also be reported with exact same values for +backward compatibility reasons: + +| non-OTLP Key | Alias for | Notes | +| --- | --- | --- | +|`otel.library.name`|`otel.scope.name`|deprecated since 1.10.0| +|`otel.library.version`|`otel.scope.version`|deprecated since 1.10.0| + +### Span Status + +Span `Status` MUST be reported as key-value pairs associated with the Span, +unless the `Status` is `UNSET`. In the latter case it MUST NOT be reported. + +The following table defines the OpenTelemetry `Status`'s mapping to Span's +key-value pairs: + +|OpenTelemetry Status Field|non-OTLP Key|non-OTLP Value| +|--|--|--| +|Code | `otel.status_code` | Name of the code, either `OK` or `ERROR`. MUST NOT be set if the code is `UNSET`. | +|Description | `otel.status_description` | Description of the `Status` if it has a value otherwise not set. | + +### Dropped Attributes Count + +OpenTelemetry dropped attributes count MUST be reported as a key-value +pair associated with the corresponding data entity (e.g. Span, Span Link, Span Event, +Metric data point, LogRecord, etc). The key name MUST be `otel.dropped_attributes_count`. + +This key-value pair should only be recorded when it contains a non-zero value. + +### Dropped Events Count + +OpenTelemetry Span's dropped events count MUST be reported as a key-value pair +associated with the Span. The key name MUST be `otel.dropped_events_count`. + +This key-value pair should only be recorded when it contains a non-zero value. + +### Dropped Links Count + +OpenTelemetry Span's dropped links count MUST be reported as a key-value pair +associated with the Span. The key name MUST be `otel.dropped_links_count`. + +This key-value pair should only be recorded when it contains a non-zero value. From e67da74c1fba3a78f8b781f0216de0569f5af674 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Fri, 10 Jun 2022 22:19:33 -0400 Subject: [PATCH 095/124] Introduce Instrumentation Scope Attributes (#2579) --- specification/common/mapping-to-non-otlp.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/specification/common/mapping-to-non-otlp.md b/specification/common/mapping-to-non-otlp.md index cb80f685ca..656b37fa35 100644 --- a/specification/common/mapping-to-non-otlp.md +++ b/specification/common/mapping-to-non-otlp.md @@ -74,3 +74,9 @@ OpenTelemetry Span's dropped links count MUST be reported as a key-value pair associated with the Span. The key name MUST be `otel.dropped_links_count`. This key-value pair should only be recorded when it contains a non-zero value. + +### Instrumentation Scope Attributes + +Exporters to formats that don't have a concept that is equivalent to the Scope +SHOULD record the attributes at the most suitable place in their corresponding format, +typically at the Span, Metric or LogRecord equivalent. From 894b49059be3930df120f1e056e921a49c1df13c Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Mon, 13 Jun 2022 12:00:00 -0400 Subject: [PATCH 096/124] Use consistent file name for data-model.md (#2586) --- specification/common/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/README.md b/specification/common/README.md index 3d1379fde4..35d6264b2a 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -119,7 +119,7 @@ at this time, as discussed in ## Attribute Collections [Resources](../resource/sdk.md), Metrics -[data points](../metrics/datamodel.md#metric-points), +[data points](../metrics/data-model.md#metric-points), [Spans](../trace/api.md#set-attributes), Span [Events](../trace/api.md#add-events), Span [Links](../trace/api.md#specifying-links) and From 6cd23747866ccaf171926fb941b163177e9f1d90 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Fri, 8 Jul 2022 03:12:12 -0400 Subject: [PATCH 097/124] Add note to Hugo front matter (#2651) --- specification/common/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/README.md b/specification/common/README.md index 35d6264b2a..2b377a731c 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -1,4 +1,4 @@ - # Common specification concepts From 5f11d9f4f7ed0efabb7ae5df8d9ee5d881285988 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Wed, 13 Jul 2022 10:01:44 -0700 Subject: [PATCH 098/124] Define net.sock attributes and clarify logical net.peer|host.name attributes (#2614) --- specification/common/attribute-requirement-level.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index 5a7d6e323d..e015ef89eb 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -48,7 +48,7 @@ All instrumentations MUST add the attribute when given condition is satisfied. S When the condition on `Conditionally Required` attribute is not satisfied and there is no requirement to populate attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Optional` requirement level on the attribute. -For example, `net.peer.name` is `Conditionally Required` by [Database convention](../trace/semantic_conventions/database.md) when available. When only `net.peer.ip` is available, instrumentation can do a DNS lookup, cache and populate `net.peer.name` but only if user explicitly enables instrumentation to do so, considering performance issues the DNS lookup introduces. +For example, `net.peer.name` is `Conditionally Required` by [Database convention](../trace/semantic_conventions/database.md) when available. When only `net.sock.peer.addr` is available, instrumentation can do a DNS lookup, cache and populate `net.sock.peer.name` but only if user explicitly enables instrumentation to do so, considering performance issues the DNS lookup introduces. ## Recommended From 3fd804bd853d584efeace2018f9d7f9244116bd9 Mon Sep 17 00:00:00 2001 From: jack-berg <34418638+jack-berg@users.noreply.github.com> Date: Tue, 13 Sep 2022 09:45:58 -0500 Subject: [PATCH 099/124] Align log sdk naming with api (#2768) Resolves #2752. This aligns log SDK and API concepts which have diverged after the merged of #2676. This PR brings alignment to the log API and SDK, and in brings the log signal into alignment with tracing and metrics where there is conceptual overlap. There shouldn't be any new concepts introduced here. - Rename `../logs/logging-library-sdk.md` to `../logs/sdk.md` - Remove wording from SDK that implies that an API doesn't exist, like [this](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/logging-library-sdk.md#L60-L62). - Move [How to Create Log4j Style Appender](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/logging-library-sdk.md#L219) to `api.md` since it describes an API use case. - Move [Implicit / Explicit Context Injection](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/logging-library-sdk.md#L270-L288) sections to `api.md` since they describe API level considerations. - Rename Logger [create](https://github.com/open-telemetry/opentelemetry-specification/blame/main/specification/logs/api.md#L133) method to be emit, to align with SDK concept of `LogRecordProcessor#onEmit(..)`. - Rename `LogProcessor`, `LogExporter` to `LogRecordProcessor`, `LogRecordExporter`. - Fill in various SDK level TODOs related to shutdown and flushing. The language from these was taken directly from the metrics / tracing SDK - no new concepts were introduced. --- specification/common/attribute-type-mapping.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-type-mapping.md b/specification/common/attribute-type-mapping.md index 46a25c1504..532d6156bc 100644 --- a/specification/common/attribute-type-mapping.md +++ b/specification/common/attribute-type-mapping.md @@ -33,7 +33,7 @@ The mapping is needed when OpenTelemetry needs to convert a value produced outsi OpenTelemetry into a value that can be exported using an OTLP exporter, or otherwise be converted to be used inside OpenTelemetry boundaries. Example use cases are the following: -- In [Logging Library SDK](../logs/logging-library-sdk.md)s, to convert values received +- In the [Logging SDK](../logs/sdk.md)s, to convert values received from logging libraries into OpenTelemetry representation. - In the Collector, to convert values received from various data sources into [pdata](https://github.com/open-telemetry/opentelemetry-collector/blob/4998703dadd19fa91a88eabf7ccc68d728bee3fd/model/pdata/common.go#L84) From d1eb5e2e74f28590022c267379b9ff9b374cc02f Mon Sep 17 00:00:00 2001 From: Bogdan Drutu Date: Tue, 25 Oct 2022 08:46:06 -0700 Subject: [PATCH 100/124] Define semantic conventions yaml for non-otlp conventions (#2850) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Define semantic conventions yaml for non-otlp conventions Signed-off-by: Bogdan Drutu * Update semantic_conventions/trace/exporter/exporter.yaml Co-authored-by: Christian Neumüller * Update semantic_conventions/scope/exporter/exporter.yaml Co-authored-by: Joao Grassi * Rename otel to otel_span Signed-off-by: Bogdan Drutu Signed-off-by: Bogdan Drutu Co-authored-by: Christian Neumüller Co-authored-by: Joao Grassi --- specification/common/mapping-to-non-otlp.md | 37 ++++++++++++++------- 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/specification/common/mapping-to-non-otlp.md b/specification/common/mapping-to-non-otlp.md index 656b37fa35..a82dac3e63 100644 --- a/specification/common/mapping-to-non-otlp.md +++ b/specification/common/mapping-to-non-otlp.md @@ -27,18 +27,22 @@ in this document. OpenTelemetry `InstrumentationScope`'s fields MUST be reported as key-value pairs associated with the Span, Metric Data Point or LogRecord using the following mapping: -| OpenTelemetry InstrumentationScope Field | non-OTLP Key | Notes | -| ------------------- | --- | --- | -| `InstrumentationScope.name`|`otel.scope.name`|since 1.10.0| -| `InstrumentationScope.version`|`otel.scope.version`|since 1.10.0| + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `otel.scope.name` | string | The name of the instrumentation scope - (`InstrumentationScope.Name` in OTLP). | `io.opentelemetry.contrib.mongodb` | Recommended | +| `otel.scope.version` | string | The version of the instrumentation scope - (`InstrumentationScope.Version` in OTLP). | `1.0.0` | Recommended | + The following deprecated aliases MUST also be reported with exact same values for backward compatibility reasons: -| non-OTLP Key | Alias for | Notes | -| --- | --- | --- | -|`otel.library.name`|`otel.scope.name`|deprecated since 1.10.0| -|`otel.library.version`|`otel.scope.version`|deprecated since 1.10.0| + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `otel.library.name` | string | Deprecated, use the `otel.scope.name` attribute. | `io.opentelemetry.contrib.mongodb` | Recommended | +| `otel.library.version` | string | Deprecated, use the `otel.scope.version` attribute. | `1.0.0` | Recommended | + ### Span Status @@ -48,10 +52,19 @@ unless the `Status` is `UNSET`. In the latter case it MUST NOT be reported. The following table defines the OpenTelemetry `Status`'s mapping to Span's key-value pairs: -|OpenTelemetry Status Field|non-OTLP Key|non-OTLP Value| -|--|--|--| -|Code | `otel.status_code` | Name of the code, either `OK` or `ERROR`. MUST NOT be set if the code is `UNSET`. | -|Description | `otel.status_description` | Description of the `Status` if it has a value otherwise not set. | + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `otel.status_code` | string | Name of the code, either "OK" or "ERROR". MUST NOT be set if the status code is UNSET. | `OK` | Recommended | +| `otel.status_description` | string | Description of the Status if it has a value, otherwise not set. | `resource not found` | Recommended | + +`otel.status_code` MUST be one of the following: + +| Value | Description | +|---|---| +| `OK` | The operation has been validated by an Application developer or Operator to have completed successfully. | +| `ERROR` | The operation contains an error. | + ### Dropped Attributes Count From 99f6917344dd6f3427cae5c45056dc42c1cce501 Mon Sep 17 00:00:00 2001 From: Alan West <3676547+alanwest@users.noreply.github.com> Date: Thu, 26 Jan 2023 16:22:06 -0800 Subject: [PATCH 101/124] Add log attribute limit configuration (#2861) Fixes #2860 Adds log attribute limit configuration. These new environment variables bring more consistency between spans and logs. --- specification/common/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/specification/common/README.md b/specification/common/README.md index 2b377a731c..f36cb92b85 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -91,11 +91,11 @@ limits programmatically. Names of the configuration options SHOULD be the same a in the list below. An SDK MAY implement model-specific limits, for example -`SpanAttributeCountLimit`. If both a general and a model-specific limit are -implemented, then the SDK MUST first attempt to use the model-specific limit, if -it isn't set, then the SDK MUST attempt to use the general limit. If neither are -defined, then the SDK MUST try to use the model-specific limit default value, -followed by the global limit default value. +`SpanAttributeCountLimit` or `LogRecordAttributeCountLimit`. If both a general +and a model-specific limit are implemented, then the SDK MUST first attempt to +use the model-specific limit, if it isn't set, then the SDK MUST attempt to use +the general limit. If neither are defined, then the SDK MUST try to use the +model-specific limit default value, followed by the global limit default value. #### Configurable Parameters From 35e27bcb99d51cd1db10757f4b87f3d687af670e Mon Sep 17 00:00:00 2001 From: Josh Suereth Date: Thu, 23 Feb 2023 11:38:55 -0500 Subject: [PATCH 102/124] Mark Attribute naming conventions as stable. (#3220) --- specification/common/attribute-naming.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index fecbf3b21b..d761b5f9fd 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -1,6 +1,6 @@ # Attribute Naming -**Status**: [Experimental](../document-status.md) +**Status**: [Stable](../document-status.md)
Table of Contents From 58c415c1ba87ef72640d2ec7257f27b5d3cc17f4 Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Wed, 1 Mar 2023 15:13:24 -0800 Subject: [PATCH 103/124] Rename Optional attribute requirement level to Opt-In (#3228) --- .../common/attribute-requirement-level.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index e015ef89eb..c030159409 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -10,7 +10,7 @@ - [Required](#required) - [Conditionally Required](#conditionally-required) - [Recommended](#recommended) -- [Optional](#optional) +- [Opt-In](#opt-in) - [Performance suggestions](#performance-suggestions) @@ -24,11 +24,11 @@ The following attribute requirement levels are specified: - [Required](#required) - [Conditionally Required](#conditionally-required) - [Recommended](#recommended) -- [Optional](#optional) +- [Opt-In](#opt-in) The requirement level for attribute is defined by semantic conventions depending on attribute availability across instrumented entities, performance, security, and other factors. When defining requirement levels, semantic conventions MUST take into account signal-specific requirements. -For example, Metric attributes that may have high cardinality can only be defined with `Optional` level. +For example, Metric attributes that may have high cardinality can only be defined with `Opt-In` level. Semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. @@ -46,7 +46,7 @@ All instrumentations MUST add the attribute when given condition is satisfied. S `http.route` is an example of a conditionally required attribute to be populated when instrumented HTTP framework provides route information for the instrumented request. Some low-level HTTP server implementations do not support routing and corresponding instrumentations can't populate the attribute. -When the condition on `Conditionally Required` attribute is not satisfied and there is no requirement to populate attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Optional` requirement level on the attribute. +When the condition on `Conditionally Required` attribute is not satisfied and there is no requirement to populate attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. For example, `net.peer.name` is `Conditionally Required` by [Database convention](../trace/semantic_conventions/database.md) when available. When only `net.sock.peer.addr` is available, instrumentation can do a DNS lookup, cache and populate `net.sock.peer.name` but only if user explicitly enables instrumentation to do so, considering performance issues the DNS lookup introduces. @@ -54,11 +54,11 @@ For example, `net.peer.name` is `Conditionally Required` by [Database convention Instrumentations SHOULD add the attribute by default if it's readily available and can be [efficiently populated](#performance-suggestions). Instrumentations MAY offer a configuration option to disable Recommended attributes. -Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD use the `Optional` requirement level on them if the attributes are logically applicable. +Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD use the `Opt-In` requirement level on them if the attributes are logically applicable. -## Optional +## Opt-In -Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Optional` attributes. +Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Opt-In` attributes. ## Performance suggestions From dba3c1c9a33049fb24ea721709af9cbf98f9d4ca Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Mon, 6 Mar 2023 10:20:11 -0800 Subject: [PATCH 104/124] Proofread of attribute requirement levels in preparation for stability (#3270) --- .../common/attribute-requirement-level.md | 25 +++++++++++-------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index c030159409..59432e6096 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -26,44 +26,47 @@ The following attribute requirement levels are specified: - [Recommended](#recommended) - [Opt-In](#opt-in) -The requirement level for attribute is defined by semantic conventions depending on attribute availability across instrumented entities, performance, security, and other factors. When defining requirement levels, semantic conventions MUST take into account signal-specific requirements. +The requirement level for an attribute is specified by semantic conventions depending on attribute availability across instrumented entities, performance, security, and other factors. When specifying requirement levels, a semantic convention MUST take into account signal-specific requirements. For example, Metric attributes that may have high cardinality can only be defined with `Opt-In` level. -Semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. +A semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. For example, [Database semantic convention](../trace/semantic_conventions/database.md) references `net.transport` attribute defined in [General attributes](../trace/semantic_conventions/span-general.md) with `Conditionally Required` level on it. ## Required -All instrumentations MUST populate the attribute. Semantic convention defining a Required attribute expects that an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, can ensure cardinality, security, and other requirements specific to signal defined by the convention. `http.method` is an example of a Required attribute. +All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.method` is an example of a Required attribute. -_Note: Consumers of telemetry can detect if telemetry item follows a specific semantic convention by checking the presence of a `Required` attribute defined by such convention. For example, the presence of `db.system` attribute on a span can be used as an indication that the span follows database semantics._ +_Note: Consumers of telemetry can detect if a telemetry item follows a specific semantic convention by checking for the presence of a `Required` attribute defined by such convention. For example, the presence of the `db.system` attribute on a span can be used as an indication that the span follows database semantics._ ## Conditionally Required -All instrumentations MUST add the attribute when given condition is satisfied. Semantic convention of a `Conditionally Required` level of an attribute MUST clarify the condition under which the attribute is expected to be populated. +All instrumentations MUST populate the attribute when the given condition is satisfied. The semantic convention of a `Conditionally Required` attribute MUST clarify the condition under which the attribute is to be populated. -`http.route` is an example of a conditionally required attribute to be populated when instrumented HTTP framework provides route information for the instrumented request. Some low-level HTTP server implementations do not support routing and corresponding instrumentations can't populate the attribute. +`http.route` is an example of a conditionally required attribute that is populated when the instrumented HTTP framework provides route information for the instrumented request. Some low-level HTTP server implementations do not support routing and corresponding instrumentations can't populate the attribute. -When the condition on `Conditionally Required` attribute is not satisfied and there is no requirement to populate attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. +When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. -For example, `net.peer.name` is `Conditionally Required` by [Database convention](../trace/semantic_conventions/database.md) when available. When only `net.sock.peer.addr` is available, instrumentation can do a DNS lookup, cache and populate `net.sock.peer.name` but only if user explicitly enables instrumentation to do so, considering performance issues the DNS lookup introduces. +For example, `net.peer.name` is `Conditionally Required` by the [Database convention](../trace/semantic_conventions/database.md) when available. When `net.sock.peer.addr` is available instead, instrumentation can do a DNS lookup, cache and populate `net.peer.name`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. ## Recommended Instrumentations SHOULD add the attribute by default if it's readily available and can be [efficiently populated](#performance-suggestions). Instrumentations MAY offer a configuration option to disable Recommended attributes. -Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD use the `Opt-In` requirement level on them if the attributes are logically applicable. +Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD allow for users to +opt-in to emit them as defined for the `Opt-In` requirement level (if the attributes are logically applicable). ## Opt-In Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Opt-In` attributes. +This attribute requirement level is recommended for attributes that are particularly expensive to retrieve or might pose a security or privacy risk. These should therefore only be enabled enabled explicitly by a user making an informed decision. + ## Performance suggestions Here are several examples of expensive operations to be avoided by default: -- DNS lookup to populate `net.peer.name` if only IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. -- forcing `http.route` calculation before HTTP framework calculates it +- DNS lookups to populate `net.peer.name` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. +- forcing an `http.route` calculation before the HTTP framework calculates it - reading response stream to find `http.response_content_length` when `Content-Length` header is not available From b5f419fee375fc694bb684f3ee3230d35870b7a7 Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Tue, 7 Mar 2023 11:22:32 -0800 Subject: [PATCH 105/124] Attribute requirement level follow-up edits (#3293) --- specification/common/attribute-requirement-level.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index 59432e6096..c7fb99345c 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -36,7 +36,7 @@ For example, [Database semantic convention](../trace/semantic_conventions/databa ## Required -All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.method` is an example of a Required attribute. +All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, and can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.method` is an example of a Required attribute. _Note: Consumers of telemetry can detect if a telemetry item follows a specific semantic convention by checking for the presence of a `Required` attribute defined by such convention. For example, the presence of the `db.system` attribute on a span can be used as an indication that the span follows database semantics._ @@ -61,7 +61,7 @@ opt-in to emit them as defined for the `Opt-In` requirement level (if the attrib Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Opt-In` attributes. -This attribute requirement level is recommended for attributes that are particularly expensive to retrieve or might pose a security or privacy risk. These should therefore only be enabled enabled explicitly by a user making an informed decision. +This attribute requirement level is recommended for attributes that are particularly expensive to retrieve or might pose a security or privacy risk. These should therefore only be enabled explicitly by a user making an informed decision. ## Performance suggestions From eadd2af3a16345b216fe88a600375b7c92e5e09f Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Thu, 16 Mar 2023 06:31:51 -0700 Subject: [PATCH 106/124] Clarify that attribute requirement levels apply to instrumentation libraries (#3289) Based on discussion in semconv stability WG Closes #3283 ## Changes Clarifies that attribute requirement levels apply to instrumentation. And that, because users can transform their telemetry in a number of ways (e.g. metric views, span processors, and collector transformations), these requirement levels cannot be relied on by telemetry consumers. --------- Co-authored-by: Liudmila Molkova Co-authored-by: Bogdan Drutu --- specification/common/attribute-requirement-level.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index c7fb99345c..3543970a29 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -19,6 +19,8 @@ _This section applies to Log, Metric, Resource, and Span, and describes requirement levels for attributes defined in semantic conventions._ +Attribute requirement levels apply to the [instrumentation library](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/glossary.md#instrumentation-library). + The following attribute requirement levels are specified: - [Required](#required) From 899a24adb154ce7f57f9eed9a37be33e65af970d Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Thu, 6 Apr 2023 21:56:17 -0700 Subject: [PATCH 107/124] Mark attribute requirement levels as stable (#3368) --- specification/common/attribute-requirement-level.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index 3543970a29..e9da7f0702 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -1,6 +1,6 @@ # Attribute Requirement Levels for Semantic Conventions -**Status**: [Experimental](../document-status.md) +**Status**: [Stable](../document-status.md)
Table of Contents From 079ef0c249cd19c9371b1f8b0e74cedffbfa5879 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Fri, 7 Apr 2023 15:55:06 -0400 Subject: [PATCH 108/124] Use path, not external URL, for link into glossary (#3375) - Contributes to https://github.com/open-telemetry/opentelemetry.io/issues/2429 - This is part of an effort to normalize links, for improved link checking on the OTel website --- specification/common/attribute-requirement-level.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index e9da7f0702..cfbc22b9da 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -19,7 +19,7 @@ _This section applies to Log, Metric, Resource, and Span, and describes requirement levels for attributes defined in semantic conventions._ -Attribute requirement levels apply to the [instrumentation library](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/glossary.md#instrumentation-library). +Attribute requirement levels apply to the [instrumentation library](../glossary.md#instrumentation-library). The following attribute requirement levels are specified: From 3d4e7b452af76e7f08a9e41575ce5ff623430139 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 8 May 2023 16:19:51 -0700 Subject: [PATCH 109/124] BREAKING: Replace `net.peer.*`/`net.host.*` with `client.*`/`server.*` (and `source.*`/`destination.*`) (#3402) --- specification/common/attribute-requirement-level.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index cfbc22b9da..aca07f487c 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -50,7 +50,7 @@ All instrumentations MUST populate the attribute when the given condition is sat When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. -For example, `net.peer.name` is `Conditionally Required` by the [Database convention](../trace/semantic_conventions/database.md) when available. When `net.sock.peer.addr` is available instead, instrumentation can do a DNS lookup, cache and populate `net.peer.name`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. +For example, `server.address` is `Conditionally Required` by the [Database convention](../trace/semantic_conventions/database.md) when available. When `server.socket.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. ## Recommended @@ -69,6 +69,6 @@ This attribute requirement level is recommended for attributes that are particul Here are several examples of expensive operations to be avoided by default: -- DNS lookups to populate `net.peer.name` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. +- DNS lookups to populate `server.address` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. - forcing an `http.route` calculation before the HTTP framework calculates it - reading response stream to find `http.response_content_length` when `Content-Length` header is not available From 70ffed7002dec16bbf839ad2619c6aee712507f4 Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Mon, 8 May 2023 17:46:55 -0700 Subject: [PATCH 110/124] BREAKING: Rename remaining network attributes from `net.*` to `network.*` and align definitions with ECS (#3426) --- specification/common/attribute-requirement-level.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index aca07f487c..fc3f5b89d3 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -34,7 +34,7 @@ For example, Metric attributes that may have high cardinality can only be define A semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. -For example, [Database semantic convention](../trace/semantic_conventions/database.md) references `net.transport` attribute defined in [General attributes](../trace/semantic_conventions/span-general.md) with `Conditionally Required` level on it. +For example, [Database semantic convention](../trace/semantic_conventions/database.md) references `network.transport` attribute defined in [General attributes](../trace/semantic_conventions/span-general.md) with `Conditionally Required` level on it. ## Required From 92a7f35fc07818475c7e838c36f1344f56025f91 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 8 May 2023 19:55:53 -0700 Subject: [PATCH 111/124] BREAKING: Introduce common `url.*` attributes, and improve use of namespacing under `http.*` (#3355) --- specification/common/attribute-naming.md | 6 +-- .../common/attribute-requirement-level.md | 4 +- specification/common/url.md | 45 +++++++++++++++++++ 3 files changed, 50 insertions(+), 5 deletions(-) create mode 100644 specification/common/url.md diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index d761b5f9fd..67aee0ad09 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -44,7 +44,7 @@ Names SHOULD follow these rules: purpose should primarily drive the decision about forming nested namespaces. - For each multi-word dot-delimited component of the attribute name separate the - words by underscores (i.e. use snake_case). For example `http.status_code` + words by underscores (i.e. use snake_case). For example `http.response.status_code` denotes the status code in the http namespace. - Names SHOULD NOT coincide with namespaces. For example if @@ -96,8 +96,8 @@ denote old attribute names in rename operations). - Semantic conventions exist for four areas: for Resource, Span, Log, and Metric attribute names. In addition, for spans we have two more areas: Event and Link attribute names. Identical namespaces or names in all these areas MUST have - identical meanings. For example the `http.method` span attribute name denotes - exactly the same concept as the `http.method` metric attribute, has the same + identical meanings. For example the `http.request.method` span attribute name denotes + exactly the same concept as the `http.request.method` metric attribute, has the same data type and the same set of possible values (in both cases it records the value of the HTTP protocol's request method as a string). diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index fc3f5b89d3..52f8e34f26 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -38,7 +38,7 @@ For example, [Database semantic convention](../trace/semantic_conventions/databa ## Required -All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, and can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.method` is an example of a Required attribute. +All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, and can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.request.method` is an example of a Required attribute. _Note: Consumers of telemetry can detect if a telemetry item follows a specific semantic convention by checking for the presence of a `Required` attribute defined by such convention. For example, the presence of the `db.system` attribute on a span can be used as an indication that the span follows database semantics._ @@ -71,4 +71,4 @@ Here are several examples of expensive operations to be avoided by default: - DNS lookups to populate `server.address` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. - forcing an `http.route` calculation before the HTTP framework calculates it -- reading response stream to find `http.response_content_length` when `Content-Length` header is not available +- reading response stream to find `http.response.body.size` when `Content-Length` header is not available diff --git a/specification/common/url.md b/specification/common/url.md new file mode 100644 index 0000000000..bbe041ba67 --- /dev/null +++ b/specification/common/url.md @@ -0,0 +1,45 @@ +# Semantic conventions for URL + +**Status**: [Experimental](../document-status.md) + +This document defines semantic conventions that describe URL and its components. + +
+Table of Contents + + + +- [Attributes](#attributes) +- [Sensitive information](#sensitive-information) + + + +
+ +## Attributes + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `url.scheme` | string | The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol. | `https`; `ftp`; `telnet` | Recommended | +| `url.full` | string | Absolute URL describing a network resource according to [RFC3986](https://www.rfc-editor.org/rfc/rfc3986) [1] | `https://www.foo.bar/search?q=OpenTelemetry#SemConv`; `//localhost` | Recommended | +| `url.path` | string | The [URI path](https://www.rfc-editor.org/rfc/rfc3986#section-3.3) component [2] | `/search` | Recommended | +| `url.query` | string | The [URI query](https://www.rfc-editor.org/rfc/rfc3986#section-3.4) component [3] | `q=OpenTelemetry` | Recommended | +| `url.fragment` | string | The [URI fragment](https://www.rfc-editor.org/rfc/rfc3986#section-3.5) component | `SemConv` | Recommended | + +**[1]:** For network calls, URL usually has `scheme://host[:port][path][?query][#fragment]` format, where the fragment is not transmitted over HTTP, but if it is known, it should be included nevertheless. +`url.full` MUST NOT contain credentials passed via URL in form of `https://username:password@www.example.com/`. In such case username and password should be redacted and attribute's value should be `https://REDACTED:REDACTED@www.example.com/`. +`url.full` SHOULD capture the absolute URL when it is available (or can be reconstructed) and SHOULD NOT be validated or modified except for sanitizing purposes. + +**[2]:** When missing, the value is assumed to be `/` + +**[3]:** Sensitive content provided in query string SHOULD be scrubbed when instrumentations can identify it. + + +## Sensitive information + +Capturing URL and its components MAY impose security risk. User and password information, when they are provided in [User Information](https://datatracker.ietf.org/doc/html/rfc3986#section-3.2.1) subcomponent, MUST NOT be recorded. + +Instrumentations that are aware of specific sensitive query string parameters MUST scrub their values before capturing `url.query` attribute. For example, native instrumentation of a client library that passes credentials or user location in URL, must scrub corresponding properties. + +_Note: Applications and telemetry consumers should scrub sensitive information from URL attributes on collected telemetry. In systems unable to identify sensitive information, certain attribute values may be redacted entirely._ From db45b8950cd1c46192b90a22ea4c8add53e947c0 Mon Sep 17 00:00:00 2001 From: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com> Date: Fri, 26 May 2023 11:23:42 -0400 Subject: [PATCH 112/124] Explain why custom attributes are not recommended to be placed in Otel namespaces (#3507) The @open-telemetry/technical-committee discussed and decided to keep the existing recommendations but clarify them and explain the purpose. --- specification/common/attribute-naming.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index 67aee0ad09..badb9acbd0 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -133,6 +133,13 @@ To do that consider a few options: `myuniquemapapp.longitude` is likely fine). Make sure the application name does not clash with an existing semantic convention namespace. +- It is not recommended to use existing OpenTelemetry semantic convention namespace + as a prefix for a new company- or application-specific attribute name. Doing so + may result in a name clash in the future, if OpenTelemetry decides to use that + same name for a different purpose or if some other third party instrumentation + decides to use that exact same attribute name and you combine that instrumentation + with your own. + - The name may be generally applicable to applications in the industry. In that case consider submitting a proposal to this specification to add a new name to the semantic conventions, and if necessary also to add a new namespace. From 2af3a419d362e891efb2063104a2accd982cdbb6 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Wed, 12 Jul 2023 10:58:54 -0400 Subject: [PATCH 113/124] Hugo front-matter fixes for aliases and linkTitle (#3592) - Followup changes for https://github.com/open-telemetry/opentelemetry.io/issues/2793 - There are only changes to Hugo front matter - Adds `likeTitle`s for "Compatibility" pages - Adds aliases for pages that have moved or were renamed - Related: https://github.com/open-telemetry/opentelemetry.io/issues/3013 -- the `compatibility/openmetrics` spec page is in the list because it was renamed /cc @svrnm @cartermp --- specification/common/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specification/common/README.md b/specification/common/README.md index f36cb92b85..239bdf2014 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -1,6 +1,7 @@ + # Common specification concepts **Status**: [Stable, Feature-freeze](../document-status.md) From 0b3465ca3d727831418e058120c2bfed7c76cc48 Mon Sep 17 00:00:00 2001 From: Josh Suereth Date: Tue, 3 Oct 2023 12:13:38 -0400 Subject: [PATCH 114/124] Remove local stubs of semantic conventions. (#3711) --- specification/common/attribute-naming.md | 14 +++----------- .../common/attribute-requirement-level.md | 6 ++++-- 2 files changed, 7 insertions(+), 13 deletions(-) diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md index badb9acbd0..548703db6f 100644 --- a/specification/common/attribute-naming.md +++ b/specification/common/attribute-naming.md @@ -65,7 +65,7 @@ Names SHOULD follow these rules: values: the executable name and command arguments. - When an attribute represents a measurement, - [Metric Name Pluralization Guidelines](../metrics/semantic_conventions/README.md#pluralization) + [Metric Name Pluralization Guidelines](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/metrics.md#pluralization) SHOULD be followed for the attribute name. ## Name Reuse Prohibition @@ -83,11 +83,7 @@ denote old attribute names in rename operations). of a namespace. - When coming up with a new semantic convention make sure to check existing - namespaces for - [Resources](../resource/semantic_conventions/README.md), - [Spans](../trace/semantic_conventions/README.md), - and - [Metrics](../metrics/semantic_conventions/README.md) + namespaces ([Semantic Conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md)) to see if the new name fits. - When a new namespace is necessary consider whether it should be a top-level @@ -111,11 +107,7 @@ denote old attribute names in rename operations). ## Recommendations for Application Developers As an application developer when you need to record an attribute first consult -existing semantic conventions for -[Resources](../resource/semantic_conventions/README.md), -[Spans](../trace/semantic_conventions/README.md), -and -[Metrics](../metrics/semantic_conventions/README.md). +existing [semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md). If an appropriate name does not exists you will need to come up with a new name. To do that consider a few options: diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index 52f8e34f26..1866c413ca 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -34,7 +34,8 @@ For example, Metric attributes that may have high cardinality can only be define A semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. -For example, [Database semantic convention](../trace/semantic_conventions/database.md) references `network.transport` attribute defined in [General attributes](../trace/semantic_conventions/span-general.md) with `Conditionally Required` level on it. + +For example, [Database semantic convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) references `network.transport` attribute defined in [General attributes](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/README.md) with `Conditionally Required` level on it. ## Required @@ -50,7 +51,8 @@ All instrumentations MUST populate the attribute when the given condition is sat When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. -For example, `server.address` is `Conditionally Required` by the [Database convention](../trace/semantic_conventions/database.md) when available. When `server.socket.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. + +For example, `server.address` is `Conditionally Required` by the [Database convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) when available. When `server.socket.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. ## Recommended From c0513fbd9918226ef6eb5ae0e7ce1c40b508124f Mon Sep 17 00:00:00 2001 From: Carlos Alberto Cortez Date: Wed, 11 Oct 2023 01:15:24 +0200 Subject: [PATCH 115/124] Add a new AddLink() operation to Span. (#3678) Fixes #454 Related to #3337 As the Messaging SIG merged its last OTEP 222, we will be adding operations that require Links after Span creation, taking a direct route with `AddLink()`, albeit without any of the new members suggested in #3337, e.g. `timestamp` (to be discussed in a separate issue). ``` AddLink(spanContext, attributes /* optional */) ``` --- specification/common/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/README.md b/specification/common/README.md index 239bdf2014..1f17536bb6 100644 --- a/specification/common/README.md +++ b/specification/common/README.md @@ -123,7 +123,7 @@ at this time, as discussed in [data points](../metrics/data-model.md#metric-points), [Spans](../trace/api.md#set-attributes), Span [Events](../trace/api.md#add-events), Span -[Links](../trace/api.md#specifying-links) and +[Links](../trace/api.md#link) and [Log Records](../logs/data-model.md) may contain a collection of attributes. The keys in each such collection are unique, i.e. there MUST NOT exist more than one key-value pair with the same key. The enforcement of uniqueness may be performed From 04d784279e3782459e65e024da927b884263bbea Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Mon, 23 Oct 2023 08:19:08 -0700 Subject: [PATCH 116/124] Rename/replace `(client|server).socket.(address|port)` attributes with `network.(peer|local).(address|port)`. (#3713) Co-authored-by: Armin Ruech --- specification/common/attribute-requirement-level.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md index 1866c413ca..7f510fe731 100644 --- a/specification/common/attribute-requirement-level.md +++ b/specification/common/attribute-requirement-level.md @@ -52,7 +52,7 @@ All instrumentations MUST populate the attribute when the given condition is sat When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. -For example, `server.address` is `Conditionally Required` by the [Database convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) when available. When `server.socket.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. +For example, `server.address` is `Conditionally Required` by the [Database convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) when available. When `network.peer.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. ## Recommended From 7e251196c98cca076ad3ce98f91c29df1fcc00d5 Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Tue, 7 Nov 2023 14:01:24 -0600 Subject: [PATCH 117/124] Remove files to keep --- specification/common/README.md | 150 ----------- .../common/attribute-type-mapping.md | 255 ------------------ specification/common/mapping-to-non-otlp.md | 95 ------- specification/common/url.md | 45 ---- 4 files changed, 545 deletions(-) delete mode 100644 specification/common/README.md delete mode 100644 specification/common/attribute-type-mapping.md delete mode 100644 specification/common/mapping-to-non-otlp.md delete mode 100644 specification/common/url.md diff --git a/specification/common/README.md b/specification/common/README.md deleted file mode 100644 index 1f17536bb6..0000000000 --- a/specification/common/README.md +++ /dev/null @@ -1,150 +0,0 @@ - - -# Common specification concepts - -**Status**: [Stable, Feature-freeze](../document-status.md) - -
-Table of Contents - - - -- [Attribute](#attribute) - * [Attribute Limits](#attribute-limits) - + [Configurable Parameters](#configurable-parameters) - + [Exempt Entities](#exempt-entities) -- [Attribute Collections](#attribute-collections) - - - -
- -## Attribute - - - -An `Attribute` is a key-value pair, which MUST have the following properties: - -- The attribute key MUST be a non-`null` and non-empty string. -- The attribute value is either: - - A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer. - - An array of primitive type values. The array MUST be homogeneous, - i.e., it MUST NOT contain values of different types. - -For protocols that do not natively support non-string values, non-string values SHOULD be represented as JSON-encoded strings. For example, the expression `int64(100)` will be encoded as `100`, `float64(1.5)` will be encoded as `1.5`, and an empty array of any type will be encoded as `[]`. - -Attribute values expressing a numerical value of zero, an empty string, or an -empty array are considered meaningful and MUST be stored and passed on to -processors / exporters. - -Attribute values of `null` are not valid and attempting to set a `null` value is -undefined behavior. - -`null` values SHOULD NOT be allowed in arrays. However, if it is impossible to -make sure that no `null` values are accepted -(e.g. in languages that do not have appropriate compile-time type checking), -`null` values within arrays MUST be preserved as-is (i.e., passed on to span -processors / exporters as `null`). If exporters do not support exporting `null` -values, they MAY replace those values by 0, `false`, or empty strings. -This is required for map/dictionary structures represented as two arrays with -indices that are kept in sync (e.g., two attributes `header_keys` and `header_values`, -both containing an array of strings to represent a mapping -`header_keys[i] -> header_values[i]`). - -See [Attribute Naming](attribute-naming.md) for naming guidelines. - -See [Requirement Level](attribute-requirement-level.md) for requirement levels guidelines. - -See [this document](attribute-type-mapping.md) to find out how to map values obtained -outside OpenTelemetry into OpenTelemetry attribute values. - -### Attribute Limits - -Execution of erroneous code can result in unintended attributes. If there are no -limits placed on attributes, they can quickly exhaust available memory, resulting -in crashes that are difficult to recover from safely. - -By default an SDK SHOULD apply truncation as per the list of -[configurable parameters](#configurable-parameters) below. - -If an SDK provides a way to: - -- set an attribute value length limit such that for each - attribute value: - - if it is a string, if it exceeds that limit (counting any character in it as - 1), SDKs MUST truncate that value, so that its length is at most equal - to the limit, - - if it is an array of strings, then apply the above rule to each of the - values separately, - - otherwise a value MUST NOT be truncated; -- set a limit of unique attribute keys such that: - - for each unique attribute key, addition of which would result in exceeding - the limit, SDK MUST discard that key/value pair. - -There MAY be a log emitted to indicate to the user that an attribute was -truncated or discarded. To prevent excessive logging, the log MUST NOT be -emitted more than once per record on which an attribute is set. - -If the SDK implements the limits above, it MUST provide a way to change these -limits programmatically. Names of the configuration options SHOULD be the same as -in the list below. - -An SDK MAY implement model-specific limits, for example -`SpanAttributeCountLimit` or `LogRecordAttributeCountLimit`. If both a general -and a model-specific limit are implemented, then the SDK MUST first attempt to -use the model-specific limit, if it isn't set, then the SDK MUST attempt to use -the general limit. If neither are defined, then the SDK MUST try to use the -model-specific limit default value, followed by the global limit default value. - -#### Configurable Parameters - -* `AttributeCountLimit` (Default=128) - Maximum allowed attribute count per record; -* `AttributeValueLengthLimit` (Default=Infinity) - Maximum allowed attribute value length; - -#### Exempt Entities - -Resource attributes SHOULD be exempt from the limits described above as resources -are not susceptible to the scenarios (auto-instrumentation) that result in -excessive attributes count or size. Resources are also sent only once per batch -instead of per span so it is relatively cheaper to have more/larger attributes -on them. Resources are also immutable by design and they are generally passed -down to TracerProvider along with limits. This makes it awkward to implement -attribute limits for Resources. - -Attributes, which belong to Metrics, are exempt from the limits described above -at this time, as discussed in -[Metrics Attribute Limits](../metrics/sdk.md#attribute-limits). - -## Attribute Collections - -[Resources](../resource/sdk.md), Metrics -[data points](../metrics/data-model.md#metric-points), -[Spans](../trace/api.md#set-attributes), Span -[Events](../trace/api.md#add-events), Span -[Links](../trace/api.md#link) and -[Log Records](../logs/data-model.md) may contain a collection of attributes. The -keys in each such collection are unique, i.e. there MUST NOT exist more than one -key-value pair with the same key. The enforcement of uniqueness may be performed -in a variety of ways as it best fits the limitations of the particular -implementation. - -Normally for the telemetry generated using OpenTelemetry SDKs the attribute -key-value pairs are set via an API that either accepts a single key-value pair -or a collection of key-value pairs. Setting an attribute with the same key as an -existing attribute SHOULD overwrite the existing attribute's value. See for -example Span's [SetAttribute](../trace/api.md#set-attributes) API. - -A typical implementation of [SetAttribute](../trace/api.md#set-attributes) API -will enforce the uniqueness by overwriting any existing attribute values pending -to be exported, so that when the Span is eventually exported the exporters see -only unique attributes. The OTLP format in particular requires that exported -Resources, Spans, Metric data points and Log Records contain only unique -attributes. - -Some other implementations may use a streaming approach where every -[SetAttribute](../trace/api.md#set-attributes) API call immediately results in -that individual attribute value being exported using a streaming wire protocol. -In such cases the enforcement of uniqueness will likely be the responsibility of -the recipient of this data. diff --git a/specification/common/attribute-type-mapping.md b/specification/common/attribute-type-mapping.md deleted file mode 100644 index 532d6156bc..0000000000 --- a/specification/common/attribute-type-mapping.md +++ /dev/null @@ -1,255 +0,0 @@ -# Mapping Arbitrary Data to OTLP AnyValue - -**Status**: [Experimental](../document-status.md) - -
-Table of Contents - - - -- [Converting to AnyValue](#converting-to-anyvalue) - * [Primitive Values](#primitive-values) - + [Integer Values](#integer-values) - + [Enumerations](#enumerations) - + [Floating Point Values](#floating-point-values) - + [String Values](#string-values) - + [Byte Sequences](#byte-sequences) - * [Composite Values](#composite-values) - + [Array Values](#array-values) - + [Associative Arrays With Unique Keys](#associative-arrays-with-unique-keys) - + [Associative Arrays With Non-Unique Keys](#associative-arrays-with-non-unique-keys) - + [Sets](#sets) - * [Other Values](#other-values) - * [Empty Values](#empty-values) - - - -
- -This document defines how to map (convert) arbitrary data (e.g. in-memory -objects) to OTLP's [AnyValue](https://github.com/open-telemetry/opentelemetry-proto/blob/cc4ed55c082cb75e084d40b4ddf3805eda099f97/opentelemetry/proto/common/v1/common.proto#L27). - -The mapping is needed when OpenTelemetry needs to convert a value produced outside -OpenTelemetry into a value that can be exported using an OTLP exporter, or otherwise be -converted to be used inside OpenTelemetry boundaries. Example use cases are the following: - -- In the [Logging SDK](../logs/sdk.md)s, to convert values received - from logging libraries into OpenTelemetry representation. -- In the Collector, to convert values received from various data sources into - [pdata](https://github.com/open-telemetry/opentelemetry-collector/blob/4998703dadd19fa91a88eabf7ccc68d728bee3fd/model/pdata/common.go#L84) - internal representation. - -## Converting to AnyValue - -[AnyValue](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L27) -is capable of representing primitive and structured data of certain types. - -Implementations that have source data in any form, such as in-memory objects -or data coming from other formats that needs to be converted to AnyValue SHOULD -follow the rules described below. - -### Primitive Values - -#### Integer Values - -Integer values which are within the range of 64 bit signed numbers -[-2^63..2^63-1] SHOULD be converted to AnyValue's -[int_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L33) -field. - -Integer values which are outside the range of 64 bit signed numbers SHOULD be -converted to AnyValue's -[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) -field using decimal representation. - -#### Enumerations - -Values, which belong to a limited enumerated set (e.g. a Java -[enum](https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html)), SHOULD be -converted to AnyValue's -[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) -field with the value of the string set to the symbolic name of the enumeration. - -If it is not possible to obtain the symbolic name of the enumeration, the -implementation SHOULD map enumeration values to AnyValue's -[int_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L33) -field set to the enum's ordinal value, when such an ordinal number is naturally -obtainable. - -If it is also not possible to obtain the ordinal value, the enumeration SHOULD be -converted to AnyValue's -[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) -field in any manner that the implementation deems reasonable. - -#### Floating Point Values - -Floating point values which are within the range and precision of IEEE 754 -64-bit floating point numbers (including IEEE 32-bit floating point values) -SHOULD be converted to AnyValue's -[double_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L34) -field. - -Floating point values which are outside the range or precision of IEEE 754 -64-bit floating point numbers (e.g. IEEE 128-bit floating bit values) SHOULD be -converted to AnyValue's -[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) -field using decimal floating point representation. - -#### String Values - -String values which are valid UTF-8 sequences SHOULD be converted to -AnyValue's -[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) -field. - -String values which are not valid Unicode sequences SHOULD be converted to -AnyValue's -[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) -with the bytes representing the string in the original order and format of the -source string. - -#### Byte Sequences - -Byte sequences (e.g. Go's `[]byte` slice or raw byte content of a file) SHOULD -be converted to AnyValue's -[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) -field. - -### Composite Values - -#### Array Values - -Values that represent ordered sequences of other values (such as -[arrays](https://docs.oracle.com/javase/specs/jls/se7/html/jls-10.html), -[vectors](https://en.cppreference.com/w/cpp/container/vector), ordered -[lists](https://docs.python.org/3/tutorial/datastructures.html#more-on-lists), -[slices](https://golang.org/ref/spec#Slice_types)) SHOULD be converted to -AnyValue's -[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) -field. String values and byte sequences are an exception from this rule (see -above). - -The rules described in this document should be applied recursively to each element -of the array. - -#### Associative Arrays With Unique Keys - -Values that represent associative arrays with unique keys (also often known -as maps, dictionaries or key-value stores) SHOULD be converted to AnyValue's -[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) -field. - -If the keys of the source array are not strings, they MUST be converted to -strings by any means available, often via a toString() or stringify functions -available in programming languages. The conversion function MUST be chosen in a -way that ensures that the resulting string keys are unique in the target array. - -The value part of each element of the source array SHOULD be converted to -AnyValue recursively. - -For example a JSON object `{"a": 123, "b": "def"}` SHOULD be converted to - -``` -AnyValue{ - kvlist_value:KeyValueList{ - values:[ - KeyValue{key:"a",value:AnyValue{int_value:123}}, - KeyValue{key:"b",value:AnyValue{string_value:"def"}}, - ] - } -} -``` - -The rules described in this document should be applied recursively to each value -of the associative array. - -#### Associative Arrays With Non-Unique Keys - -Values that represent an associative arrays with non-unique keys where multiple values may be associated with the same key (also sometimes known -as multimaps, multidicts) SHOULD be converted to AnyValue's -[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) -field. - -The resulting -[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) -field MUST list each key only once and the value of each element of -[kvlist_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L36) -field MUST be an array represented using AnyValue's -[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) -field, each element of the -[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) -representing one value of source array associated with the given key. - -For example an associative array shown in the following table: - -|Key|Value| -|---|---| -|"abc"|123| -|"def"|"foo"| -|"def"|"bar"| - -SHOULD be converted to: - -``` -AnyValue{ - kvlist_value:KeyValueList{ - values:[ - KeyValue{ - key:"abc", - value:AnyValue{array_value:ArrayValue{values[ - AnyValue{int_value:123} - ]}} - }, - KeyValue{ - key:"def", - value:AnyValue{array_value:ArrayValue{values[ - AnyValue{string_value:"foo"}, - AnyValue{string_value:"bar"} - ]}} - }, - ] - } -} -``` - -The rules described in this document should be applied recursively to each value -of the associative array. - -#### Sets - -Unordered collections of unique values (such as -[Java Sets](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Set.html), -[C++ sets](https://en.cppreference.com/w/cpp/container/set), -[Python Sets](https://docs.python.org/3/tutorial/datastructures.html#sets)) SHOULD be -converted to AnyValue's -[array_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L35) -field, where each element of the set becomes an element of the array. - -The rules described in this document should be applied recursively to each value -of the set. - -### Other Values - -Any other values not listed above SHOULD be converted to AnyValue's -[string_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L31) -field if the source data can be serialized to a string (can be stringified) -using toString() or stringify functions available in programming languages. - -If the source data cannot be serialized to a string then the value SHOULD be -converted AnyValue's -[bytes_value](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L37) -field by serializing it into a byte sequence by any means available. - -If the source data cannot be serialized neither to a string nor to a byte -sequence then it SHOULD by converted to an empty AnyValue. - -### Empty Values - -If the source data has no type associated with it and is empty, null, nil or -otherwise indicates absence of data it SHOULD be converted to an -[empty](https://github.com/open-telemetry/opentelemetry-proto/blob/38b5b9b6e5257c6500a843f7fdacf89dd95833e8/opentelemetry/proto/common/v1/common.proto#L29) -AnyValue, where all the fields are unset. - -Empty values which has a type associated with them (e.g. empty associative -array) SHOULD be converted using the corresponding rules defined for the types -above. diff --git a/specification/common/mapping-to-non-otlp.md b/specification/common/mapping-to-non-otlp.md deleted file mode 100644 index a82dac3e63..0000000000 --- a/specification/common/mapping-to-non-otlp.md +++ /dev/null @@ -1,95 +0,0 @@ -# OpenTelemetry Transformation to non-OTLP Formats - -**Status**: [Stable](../document-status.md) - -All OpenTelemetry concepts and data recorded using OpenTelemetry API can be -directly and precisely represented using corresponding messages and fields of -OTLP format. However, for other formats this is not always the case. Sometimes a -format will not have a native way to represent a particular OpenTelemetry -concept or a field of a concept. - -This document defines the transformation between OpenTelemetry and formats other -than OTLP, for OpenTelemetry fields and concepts that have no direct semantic -equivalent in those other formats. - -Note: when a format has a direct semantic equivalent for a particular field or -concept then the recommendation in this document MUST be ignored. - -See also additional specific transformation rules for -[Jaeger](../trace/sdk_exporters/jaeger.md) and [Zipkin](../trace/sdk_exporters/zipkin.md). -The specific rules for Jaeger and Zipkin take precedence over the generic rules defined -in this document. - -## Mappings - -### InstrumentationScope - -OpenTelemetry `InstrumentationScope`'s fields MUST be reported as key-value -pairs associated with the Span, Metric Data Point or LogRecord using the following mapping: - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `otel.scope.name` | string | The name of the instrumentation scope - (`InstrumentationScope.Name` in OTLP). | `io.opentelemetry.contrib.mongodb` | Recommended | -| `otel.scope.version` | string | The version of the instrumentation scope - (`InstrumentationScope.Version` in OTLP). | `1.0.0` | Recommended | - - -The following deprecated aliases MUST also be reported with exact same values for -backward compatibility reasons: - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `otel.library.name` | string | Deprecated, use the `otel.scope.name` attribute. | `io.opentelemetry.contrib.mongodb` | Recommended | -| `otel.library.version` | string | Deprecated, use the `otel.scope.version` attribute. | `1.0.0` | Recommended | - - -### Span Status - -Span `Status` MUST be reported as key-value pairs associated with the Span, -unless the `Status` is `UNSET`. In the latter case it MUST NOT be reported. - -The following table defines the OpenTelemetry `Status`'s mapping to Span's -key-value pairs: - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `otel.status_code` | string | Name of the code, either "OK" or "ERROR". MUST NOT be set if the status code is UNSET. | `OK` | Recommended | -| `otel.status_description` | string | Description of the Status if it has a value, otherwise not set. | `resource not found` | Recommended | - -`otel.status_code` MUST be one of the following: - -| Value | Description | -|---|---| -| `OK` | The operation has been validated by an Application developer or Operator to have completed successfully. | -| `ERROR` | The operation contains an error. | - - -### Dropped Attributes Count - -OpenTelemetry dropped attributes count MUST be reported as a key-value -pair associated with the corresponding data entity (e.g. Span, Span Link, Span Event, -Metric data point, LogRecord, etc). The key name MUST be `otel.dropped_attributes_count`. - -This key-value pair should only be recorded when it contains a non-zero value. - -### Dropped Events Count - -OpenTelemetry Span's dropped events count MUST be reported as a key-value pair -associated with the Span. The key name MUST be `otel.dropped_events_count`. - -This key-value pair should only be recorded when it contains a non-zero value. - -### Dropped Links Count - -OpenTelemetry Span's dropped links count MUST be reported as a key-value pair -associated with the Span. The key name MUST be `otel.dropped_links_count`. - -This key-value pair should only be recorded when it contains a non-zero value. - -### Instrumentation Scope Attributes - -Exporters to formats that don't have a concept that is equivalent to the Scope -SHOULD record the attributes at the most suitable place in their corresponding format, -typically at the Span, Metric or LogRecord equivalent. diff --git a/specification/common/url.md b/specification/common/url.md deleted file mode 100644 index bbe041ba67..0000000000 --- a/specification/common/url.md +++ /dev/null @@ -1,45 +0,0 @@ -# Semantic conventions for URL - -**Status**: [Experimental](../document-status.md) - -This document defines semantic conventions that describe URL and its components. - -
-Table of Contents - - - -- [Attributes](#attributes) -- [Sensitive information](#sensitive-information) - - - -
- -## Attributes - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `url.scheme` | string | The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol. | `https`; `ftp`; `telnet` | Recommended | -| `url.full` | string | Absolute URL describing a network resource according to [RFC3986](https://www.rfc-editor.org/rfc/rfc3986) [1] | `https://www.foo.bar/search?q=OpenTelemetry#SemConv`; `//localhost` | Recommended | -| `url.path` | string | The [URI path](https://www.rfc-editor.org/rfc/rfc3986#section-3.3) component [2] | `/search` | Recommended | -| `url.query` | string | The [URI query](https://www.rfc-editor.org/rfc/rfc3986#section-3.4) component [3] | `q=OpenTelemetry` | Recommended | -| `url.fragment` | string | The [URI fragment](https://www.rfc-editor.org/rfc/rfc3986#section-3.5) component | `SemConv` | Recommended | - -**[1]:** For network calls, URL usually has `scheme://host[:port][path][?query][#fragment]` format, where the fragment is not transmitted over HTTP, but if it is known, it should be included nevertheless. -`url.full` MUST NOT contain credentials passed via URL in form of `https://username:password@www.example.com/`. In such case username and password should be redacted and attribute's value should be `https://REDACTED:REDACTED@www.example.com/`. -`url.full` SHOULD capture the absolute URL when it is available (or can be reconstructed) and SHOULD NOT be validated or modified except for sanitizing purposes. - -**[2]:** When missing, the value is assumed to be `/` - -**[3]:** Sensitive content provided in query string SHOULD be scrubbed when instrumentations can identify it. - - -## Sensitive information - -Capturing URL and its components MAY impose security risk. User and password information, when they are provided in [User Information](https://datatracker.ietf.org/doc/html/rfc3986#section-3.2.1) subcomponent, MUST NOT be recorded. - -Instrumentations that are aware of specific sensitive query string parameters MUST scrub their values before capturing `url.query` attribute. For example, native instrumentation of a client library that passes credentials or user location in URL, must scrub corresponding properties. - -_Note: Applications and telemetry consumers should scrub sensitive information from URL attributes on collected telemetry. In systems unable to identify sensitive information, certain attribute values may be redacted entirely._ From 55f5ce9a0c3c7fe34c48b14d418889b30f8ad907 Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Tue, 7 Nov 2023 14:02:50 -0600 Subject: [PATCH 118/124] Move attribute documents --- specification/common/attribute-naming.md | 156 ------------------ .../common/attribute-requirement-level.md | 76 --------- 2 files changed, 232 deletions(-) delete mode 100644 specification/common/attribute-naming.md delete mode 100644 specification/common/attribute-requirement-level.md diff --git a/specification/common/attribute-naming.md b/specification/common/attribute-naming.md deleted file mode 100644 index 548703db6f..0000000000 --- a/specification/common/attribute-naming.md +++ /dev/null @@ -1,156 +0,0 @@ -# Attribute Naming - -**Status**: [Stable](../document-status.md) - -
-Table of Contents - - - -- [Name Pluralization guidelines](#name-pluralization-guidelines) -- [Name Reuse Prohibition](#name-reuse-prohibition) -- [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) -- [Recommendations for Application Developers](#recommendations-for-application-developers) -- [otel.* Namespace](#otel-namespace) - - - -
- -_This section applies to Resource, Span, Log, and Metric attribute names (also -known as the "attribute keys"). For brevity within this section when we use the -term "name" without an adjective it is implied to mean "attribute name"._ - -Every name MUST be a valid Unicode sequence. - -_Note: we merely require that the names are represented as Unicode sequences. -This specification does not define how exactly the Unicode sequences are -encoded. The encoding can vary from one programming language to another and from -one wire format to another. Use the idiomatic way to represent Unicode in the -particular programming language or wire format._ - -Names SHOULD follow these rules: - -- Use namespacing to avoid name clashes. Delimit the namespaces using a dot - character. For example `service.version` denotes the service version where - `service` is the namespace and `version` is an attribute in that namespace. - -- Namespaces can be nested. For example `telemetry.sdk` is a namespace inside - top-level `telemetry` namespace and `telemetry.sdk.name` is an attribute - inside `telemetry.sdk` namespace. - Note: the fact that an entity is located within another entity is typically - not a sufficient reason for forming nested namespaces. The purpose of a - namespace is to avoid name clashes, not to indicate entity hierarchies. This - purpose should primarily drive the decision about forming nested namespaces. - -- For each multi-word dot-delimited component of the attribute name separate the - words by underscores (i.e. use snake_case). For example `http.response.status_code` - denotes the status code in the http namespace. - -- Names SHOULD NOT coincide with namespaces. For example if - `service.instance.id` is an attribute name then it is no longer valid to have - an attribute named `service.instance` because `service.instance` is already a - namespace. Because of this rule be careful when choosing names: every existing - name prohibits existence of an equally named namespace in the future, and vice - versa: any existing namespace prohibits existence of an equally named - attribute key in the future. - -## Name Pluralization guidelines - -- When an attribute represents a single entity, the attribute name SHOULD be singular. - Examples: `host.name`, `db.user`, `container.id`. - -- When attribute can represent multiple entities, the attribute name SHOULD be pluralized - and the value type SHOULD be an array. E.g. `process.command_args` might include multiple - values: the executable name and command arguments. - -- When an attribute represents a measurement, - [Metric Name Pluralization Guidelines](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/metrics.md#pluralization) - SHOULD be followed for the attribute name. - -## Name Reuse Prohibition - -A new attribute MUST NOT be added with the same name as an attribute that -existed in the past but was renamed (with a corresponding schema file). - -When introducing a new attribute name check all existing schema files to make -sure the name does not appear as a key of any "rename_attributes" section (keys -denote old attribute names in rename operations). - -## Recommendations for OpenTelemetry Authors - -- All names that are part of OpenTelemetry semantic conventions SHOULD be part - of a namespace. - -- When coming up with a new semantic convention make sure to check existing - namespaces ([Semantic Conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md)) - to see if the new name fits. - -- When a new namespace is necessary consider whether it should be a top-level - namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). - -- Semantic conventions exist for four areas: for Resource, Span, Log, and Metric - attribute names. In addition, for spans we have two more areas: Event and Link - attribute names. Identical namespaces or names in all these areas MUST have - identical meanings. For example the `http.request.method` span attribute name denotes - exactly the same concept as the `http.request.method` metric attribute, has the same - data type and the same set of possible values (in both cases it records the - value of the HTTP protocol's request method as a string). - -- Semantic conventions MUST limit names to printable Basic Latin characters - (more precisely to - [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) - subset of Unicode code points). It is recommended to further limit names to - the following Unicode code points: Latin alphabet, Numeric, Underscore, Dot - (as namespace delimiter). - -## Recommendations for Application Developers - -As an application developer when you need to record an attribute first consult -existing [semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md). -If an appropriate name does not exists you will need to come up with a new name. -To do that consider a few options: - -- The name is specific to your company and may be possibly used outside the - company as well. To avoid clashes with names introduced by other companies (in - a distributed system that uses applications from multiple vendors) it is - recommended to prefix the new name by your company's reverse domain name, e.g. - `com.acme.shopname`. - -- The name is specific to your application that will be used internally only. If - you already have an internal company process that helps you to ensure no name - clashes happen then feel free to follow it. Otherwise it is recommended to - prefix the attribute name by your application name, provided that - the application name is reasonably unique within your organization (e.g. - `myuniquemapapp.longitude` is likely fine). Make sure the application name - does not clash with an existing semantic convention namespace. - -- It is not recommended to use existing OpenTelemetry semantic convention namespace - as a prefix for a new company- or application-specific attribute name. Doing so - may result in a name clash in the future, if OpenTelemetry decides to use that - same name for a different purpose or if some other third party instrumentation - decides to use that exact same attribute name and you combine that instrumentation - with your own. - -- The name may be generally applicable to applications in the industry. In that - case consider submitting a proposal to this specification to add a new name to - the semantic conventions, and if necessary also to add a new namespace. - -It is recommended to limit names to printable Basic Latin characters -(more precisely to -[U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) -subset of Unicode code points). - -## otel.* Namespace - -Attribute names that start with `otel.` are reserved to be defined by -OpenTelemetry specification. These are typically used to express OpenTelemetry -concepts in formats that don't have a corresponding concept. - -For example, the `otel.scope.name` attribute is used to record the -instrumentation scope name, which is an OpenTelemetry concept that is natively -represented in OTLP, but does not have an equivalent in other telemetry formats -and protocols. - -Any additions to the `otel.*` namespace MUST be approved as part of -OpenTelemetry specification. diff --git a/specification/common/attribute-requirement-level.md b/specification/common/attribute-requirement-level.md deleted file mode 100644 index 7f510fe731..0000000000 --- a/specification/common/attribute-requirement-level.md +++ /dev/null @@ -1,76 +0,0 @@ -# Attribute Requirement Levels for Semantic Conventions - -**Status**: [Stable](../document-status.md) - -
-Table of Contents - - - -- [Required](#required) -- [Conditionally Required](#conditionally-required) -- [Recommended](#recommended) -- [Opt-In](#opt-in) -- [Performance suggestions](#performance-suggestions) - - - -
- -_This section applies to Log, Metric, Resource, and Span, and describes requirement levels for attributes defined in semantic conventions._ - -Attribute requirement levels apply to the [instrumentation library](../glossary.md#instrumentation-library). - -The following attribute requirement levels are specified: - -- [Required](#required) -- [Conditionally Required](#conditionally-required) -- [Recommended](#recommended) -- [Opt-In](#opt-in) - -The requirement level for an attribute is specified by semantic conventions depending on attribute availability across instrumented entities, performance, security, and other factors. When specifying requirement levels, a semantic convention MUST take into account signal-specific requirements. - -For example, Metric attributes that may have high cardinality can only be defined with `Opt-In` level. - -A semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. - - -For example, [Database semantic convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) references `network.transport` attribute defined in [General attributes](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/README.md) with `Conditionally Required` level on it. - -## Required - -All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, and can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.request.method` is an example of a Required attribute. - -_Note: Consumers of telemetry can detect if a telemetry item follows a specific semantic convention by checking for the presence of a `Required` attribute defined by such convention. For example, the presence of the `db.system` attribute on a span can be used as an indication that the span follows database semantics._ - -## Conditionally Required - -All instrumentations MUST populate the attribute when the given condition is satisfied. The semantic convention of a `Conditionally Required` attribute MUST clarify the condition under which the attribute is to be populated. - -`http.route` is an example of a conditionally required attribute that is populated when the instrumented HTTP framework provides route information for the instrumented request. Some low-level HTTP server implementations do not support routing and corresponding instrumentations can't populate the attribute. - -When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. - - -For example, `server.address` is `Conditionally Required` by the [Database convention](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/README.md) when available. When `network.peer.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. - -## Recommended - -Instrumentations SHOULD add the attribute by default if it's readily available and can be [efficiently populated](#performance-suggestions). Instrumentations MAY offer a configuration option to disable Recommended attributes. - -Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD allow for users to -opt-in to emit them as defined for the `Opt-In` requirement level (if the attributes are logically applicable). - -## Opt-In - -Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Opt-In` attributes. - -This attribute requirement level is recommended for attributes that are particularly expensive to retrieve or might pose a security or privacy risk. These should therefore only be enabled explicitly by a user making an informed decision. - -## Performance suggestions - -Here are several examples of expensive operations to be avoided by default: - -- DNS lookups to populate `server.address` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. -- forcing an `http.route` calculation before the HTTP framework calculates it -- reading response stream to find `http.response.body.size` when `Content-Length` header is not available From 48d3348fea34973b3cb48b0fd0302072e7d9ca6b Mon Sep 17 00:00:00 2001 From: Jack Berg Date: Tue, 14 Nov 2023 13:54:47 -0600 Subject: [PATCH 119/124] Format with prettier --- docs/general/attribute-naming.md | 76 +++++++++--------- docs/general/attribute-requirement-level.md | 89 ++++++++++++++++----- 2 files changed, 107 insertions(+), 58 deletions(-) diff --git a/docs/general/attribute-naming.md b/docs/general/attribute-naming.md index a800f7bd39..b84f71f03e 100644 --- a/docs/general/attribute-naming.md +++ b/docs/general/attribute-naming.md @@ -11,7 +11,7 @@ - [Name Reuse Prohibition](#name-reuse-prohibition) - [Recommendations for OpenTelemetry Authors](#recommendations-for-opentelemetry-authors) - [Recommendations for Application Developers](#recommendations-for-application-developers) -- [otel.* Namespace](#otel-namespace) +- [otel.\* Namespace](#otel-namespace) @@ -37,15 +37,15 @@ Names SHOULD follow these rules: - Namespaces can be nested. For example `telemetry.sdk` is a namespace inside top-level `telemetry` namespace and `telemetry.sdk.name` is an attribute - inside `telemetry.sdk` namespace. - Note: the fact that an entity is located within another entity is typically - not a sufficient reason for forming nested namespaces. The purpose of a - namespace is to avoid name clashes, not to indicate entity hierarchies. This - purpose should primarily drive the decision about forming nested namespaces. + inside `telemetry.sdk` namespace. Note: the fact that an entity is located + within another entity is typically not a sufficient reason for forming nested + namespaces. The purpose of a namespace is to avoid name clashes, not to + indicate entity hierarchies. This purpose should primarily drive the decision + about forming nested namespaces. - For each multi-word dot-delimited component of the attribute name separate the - words by underscores (i.e. use snake_case). For example `http.response.status_code` - denotes the status code in the http namespace. + words by underscores (i.e. use snake_case). For example + `http.response.status_code` denotes the status code in the http namespace. - Names SHOULD NOT coincide with namespaces. For example if `service.instance.id` is an attribute name then it is no longer valid to have @@ -57,16 +57,16 @@ Names SHOULD follow these rules: ## Name Pluralization guidelines -- When an attribute represents a single entity, the attribute name SHOULD be singular. - Examples: `host.name`, `db.user`, `container.id`. +- When an attribute represents a single entity, the attribute name SHOULD be + singular. Examples: `host.name`, `db.user`, `container.id`. -- When attribute can represent multiple entities, the attribute name SHOULD be pluralized - and the value type SHOULD be an array. E.g. `process.command_args` might include multiple - values: the executable name and command arguments. +- When attribute can represent multiple entities, the attribute name SHOULD be + pluralized and the value type SHOULD be an array. E.g. `process.command_args` + might include multiple values: the executable name and command arguments. - When an attribute represents a measurement, - [Metric Name Pluralization Guidelines](./metrics.md#pluralization) - SHOULD be followed for the attribute name. + [Metric Name Pluralization Guidelines](./metrics.md#pluralization) SHOULD be + followed for the attribute name. ## Name Reuse Prohibition @@ -83,8 +83,7 @@ denote old attribute names in rename operations). of a namespace. - When coming up with a new semantic convention make sure to check existing - namespaces ([Semantic Conventions](./README.md)) - to see if the new name fits. + namespaces ([Semantic Conventions](./README.md)) to see if the new name fits. - When a new namespace is necessary consider whether it should be a top-level namespace (e.g. `service`) or a nested namespace (e.g. `service.instance`). @@ -92,14 +91,14 @@ denote old attribute names in rename operations). - Semantic conventions exist for four areas: for Resource, Span, Log, and Metric attribute names. In addition, for spans we have two more areas: Event and Link attribute names. Identical namespaces or names in all these areas MUST have - identical meanings. For example the `http.request.method` span attribute name denotes - exactly the same concept as the `http.request.method` metric attribute, has the same - data type and the same set of possible values (in both cases it records the - value of the HTTP protocol's request method as a string). + identical meanings. For example the `http.request.method` span attribute name + denotes exactly the same concept as the `http.request.method` metric + attribute, has the same data type and the same set of possible values (in both + cases it records the value of the HTTP protocol's request method as a string). - Semantic conventions MUST limit names to printable Basic Latin characters (more precisely to - [U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) + [U+0021 .. U+007E]() subset of Unicode code points). It is recommended to further limit names to the following Unicode code points: Latin alphabet, Numeric, Underscore, Dot (as namespace delimiter). @@ -107,9 +106,9 @@ denote old attribute names in rename operations). ## Recommendations for Application Developers As an application developer when you need to record an attribute first consult -existing [semantic conventions](./README.md). -If an appropriate name does not exists you will need to come up with a new name. -To do that consider a few options: +existing [semantic conventions](./README.md). If an appropriate name does not +exists you will need to come up with a new name. To do that consider a few +options: - The name is specific to your company and may be possibly used outside the company as well. To avoid clashes with names introduced by other companies (in @@ -120,28 +119,28 @@ To do that consider a few options: - The name is specific to your application that will be used internally only. If you already have an internal company process that helps you to ensure no name clashes happen then feel free to follow it. Otherwise it is recommended to - prefix the attribute name by your application name, provided that - the application name is reasonably unique within your organization (e.g. + prefix the attribute name by your application name, provided that the + application name is reasonably unique within your organization (e.g. `myuniquemapapp.longitude` is likely fine). Make sure the application name does not clash with an existing semantic convention namespace. -- It is not recommended to use existing OpenTelemetry semantic convention namespace - as a prefix for a new company- or application-specific attribute name. Doing so - may result in a name clash in the future, if OpenTelemetry decides to use that - same name for a different purpose or if some other third party instrumentation - decides to use that exact same attribute name and you combine that instrumentation - with your own. +- It is not recommended to use existing OpenTelemetry semantic convention + namespace as a prefix for a new company- or application-specific attribute + name. Doing so may result in a name clash in the future, if OpenTelemetry + decides to use that same name for a different purpose or if some other third + party instrumentation decides to use that exact same attribute name and you + combine that instrumentation with your own. - The name may be generally applicable to applications in the industry. In that case consider submitting a proposal to this specification to add a new name to the semantic conventions, and if necessary also to add a new namespace. -It is recommended to limit names to printable Basic Latin characters -(more precisely to -[U+0021 .. U+007E](https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)#Table_of_characters) +It is recommended to limit names to printable Basic Latin characters (more +precisely to +[U+0021 .. U+007E]() subset of Unicode code points). -## otel.* Namespace +## otel.\* Namespace Attribute names that start with `otel.` are reserved to be defined by OpenTelemetry specification. These are typically used to express OpenTelemetry @@ -155,4 +154,5 @@ and protocols. Any additions to the `otel.*` namespace MUST be approved as part of OpenTelemetry specification. -[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/document-status.md +[DocumentStatus]: + https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/document-status.md diff --git a/docs/general/attribute-requirement-level.md b/docs/general/attribute-requirement-level.md index eb89336df3..4fe6ddb9dd 100644 --- a/docs/general/attribute-requirement-level.md +++ b/docs/general/attribute-requirement-level.md @@ -17,9 +17,11 @@
-_This section applies to Log, Metric, Resource, and Span, and describes requirement levels for attributes defined in semantic conventions._ +_This section applies to Log, Metric, Resource, and Span, and describes +requirement levels for attributes defined in semantic conventions._ -Attribute requirement levels apply to the [instrumentation library](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/glossary.md#instrumentation-library). +Attribute requirement levels apply to the +[instrumentation library](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/glossary.md#instrumentation-library). The following attribute requirement levels are specified: @@ -28,51 +30,98 @@ The following attribute requirement levels are specified: - [Recommended](#recommended) - [Opt-In](#opt-in) -The requirement level for an attribute is specified by semantic conventions depending on attribute availability across instrumented entities, performance, security, and other factors. When specifying requirement levels, a semantic convention MUST take into account signal-specific requirements. +The requirement level for an attribute is specified by semantic conventions +depending on attribute availability across instrumented entities, performance, +security, and other factors. When specifying requirement levels, a semantic +convention MUST take into account signal-specific requirements. -For example, Metric attributes that may have high cardinality can only be defined with `Opt-In` level. +For example, Metric attributes that may have high cardinality can only be +defined with `Opt-In` level. -A semantic convention that refers to an attribute from another semantic convention MAY modify the requirement level within its own scope. Otherwise, requirement level from the referred semantic convention applies. +A semantic convention that refers to an attribute from another semantic +convention MAY modify the requirement level within its own scope. Otherwise, +requirement level from the referred semantic convention applies. -For example, [Database semantic convention](../database/README.md) references `network.transport` attribute defined in [General attributes](./README.md) with `Conditionally Required` level on it. + +For example, [Database semantic convention](../database/README.md) references +`network.transport` attribute defined in [General attributes](./README.md) with +`Conditionally Required` level on it. ## Required -All instrumentations MUST populate the attribute. A semantic convention defining a Required attribute expects an absolute majority of instrumentation libraries and applications are able to efficiently retrieve and populate it, and can additionally meet requirements for cardinality, security, and any others specific to the signal defined by the convention. `http.request.method` is an example of a Required attribute. +All instrumentations MUST populate the attribute. A semantic convention defining +a Required attribute expects an absolute majority of instrumentation libraries +and applications are able to efficiently retrieve and populate it, and can +additionally meet requirements for cardinality, security, and any others +specific to the signal defined by the convention. `http.request.method` is an +example of a Required attribute. -_Note: Consumers of telemetry can detect if a telemetry item follows a specific semantic convention by checking for the presence of a `Required` attribute defined by such convention. For example, the presence of the `db.system` attribute on a span can be used as an indication that the span follows database semantics._ +_Note: Consumers of telemetry can detect if a telemetry item follows a specific +semantic convention by checking for the presence of a `Required` attribute +defined by such convention. For example, the presence of the `db.system` +attribute on a span can be used as an indication that the span follows database +semantics._ ## Conditionally Required -All instrumentations MUST populate the attribute when the given condition is satisfied. The semantic convention of a `Conditionally Required` attribute MUST clarify the condition under which the attribute is to be populated. +All instrumentations MUST populate the attribute when the given condition is +satisfied. The semantic convention of a `Conditionally Required` attribute MUST +clarify the condition under which the attribute is to be populated. -`http.route` is an example of a conditionally required attribute that is populated when the instrumented HTTP framework provides route information for the instrumented request. Some low-level HTTP server implementations do not support routing and corresponding instrumentations can't populate the attribute. +`http.route` is an example of a conditionally required attribute that is +populated when the instrumented HTTP framework provides route information for +the instrumented request. Some low-level HTTP server implementations do not +support routing and corresponding instrumentations can't populate the attribute. -When a `Conditionally Required` attribute's condition is not satisfied, and there is no requirement to populate the attribute, semantic conventions MAY provide special instructions on how to handle it. If no instructions are given and if instrumentation can populate the attribute, instrumentation SHOULD use the `Opt-In` requirement level on the attribute. +When a `Conditionally Required` attribute's condition is not satisfied, and +there is no requirement to populate the attribute, semantic conventions MAY +provide special instructions on how to handle it. If no instructions are given +and if instrumentation can populate the attribute, instrumentation SHOULD use +the `Opt-In` requirement level on the attribute. -For example, `server.address` is `Conditionally Required` by the [Database convention](../database/README.md) when available. When `network.peer.address` is available instead, instrumentation can do a DNS lookup, cache and populate `server.address`, but only if the user explicitly enables the instrumentation to do so, considering the performance issues that DNS lookups introduce. + +For example, `server.address` is `Conditionally Required` by the +[Database convention](../database/README.md) when available. When +`network.peer.address` is available instead, instrumentation can do a DNS +lookup, cache and populate `server.address`, but only if the user explicitly +enables the instrumentation to do so, considering the performance issues that +DNS lookups introduce. ## Recommended -Instrumentations SHOULD add the attribute by default if it's readily available and can be [efficiently populated](#performance-suggestions). Instrumentations MAY offer a configuration option to disable Recommended attributes. +Instrumentations SHOULD add the attribute by default if it's readily available +and can be [efficiently populated](#performance-suggestions). Instrumentations +MAY offer a configuration option to disable Recommended attributes. -Instrumentations that decide not to populate `Recommended` attributes due to [performance](#performance-suggestions), security, privacy, or other consideration by default, SHOULD allow for users to -opt-in to emit them as defined for the `Opt-In` requirement level (if the attributes are logically applicable). +Instrumentations that decide not to populate `Recommended` attributes due to +[performance](#performance-suggestions), security, privacy, or other +consideration by default, SHOULD allow for users to opt-in to emit them as +defined for the `Opt-In` requirement level (if the attributes are logically +applicable). ## Opt-In -Instrumentations SHOULD populate the attribute if and only if the user configures the instrumentation to do so. Instrumentation that doesn't support configuration MUST NOT populate `Opt-In` attributes. +Instrumentations SHOULD populate the attribute if and only if the user +configures the instrumentation to do so. Instrumentation that doesn't support +configuration MUST NOT populate `Opt-In` attributes. -This attribute requirement level is recommended for attributes that are particularly expensive to retrieve or might pose a security or privacy risk. These should therefore only be enabled explicitly by a user making an informed decision. +This attribute requirement level is recommended for attributes that are +particularly expensive to retrieve or might pose a security or privacy risk. +These should therefore only be enabled explicitly by a user making an informed +decision. ## Performance suggestions Here are several examples of expensive operations to be avoided by default: -- DNS lookups to populate `server.address` when only an IP address is available to the instrumentation. Caching lookup results does not solve the issue for all possible cases and should be avoided by default too. +- DNS lookups to populate `server.address` when only an IP address is available + to the instrumentation. Caching lookup results does not solve the issue for + all possible cases and should be avoided by default too. - forcing an `http.route` calculation before the HTTP framework calculates it -- reading response stream to find `http.response.body.size` when `Content-Length` header is not available +- reading response stream to find `http.response.body.size` when + `Content-Length` header is not available -[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/document-status.md +[DocumentStatus]: + https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/document-status.md From 811dbab22aa5e9944988a9c1ed77b34a6ee65eeb Mon Sep 17 00:00:00 2001 From: Alexander Wert Date: Fri, 17 Nov 2023 09:46:57 +0100 Subject: [PATCH 120/124] Moved database attributes to the attributes registry (#441) Signed-off-by: Alexander Wert Co-authored-by: Joao Grassi Co-authored-by: Armin Ruech <7052238+arminru@users.noreply.github.com> --- docs/attributes-registry/README.md | 1 + docs/attributes-registry/db.md | 222 +++++++++++++++ docs/database/cassandra.md | 18 +- docs/database/cosmosdb.md | 18 +- docs/database/couchdb.md | 4 +- docs/database/database-spans.md | 32 ++- docs/database/dynamodb.md | 2 +- docs/database/elasticsearch.md | 27 +- docs/database/hbase.md | 4 +- docs/database/mongodb.md | 4 +- docs/database/mssql.md | 6 +- docs/database/redis.md | 6 +- docs/database/sql.md | 4 +- model/registry/db.yaml | 424 +++++++++++++++++++++++++++ model/trace/database.yaml | 444 +++-------------------------- 15 files changed, 770 insertions(+), 446 deletions(-) create mode 100644 docs/attributes-registry/db.md create mode 100644 model/registry/db.yaml diff --git a/docs/attributes-registry/README.md b/docs/attributes-registry/README.md index 607e9cfc88..c8a0702d3e 100644 --- a/docs/attributes-registry/README.md +++ b/docs/attributes-registry/README.md @@ -31,6 +31,7 @@ Currently, the following namespaces exist: * [Cloud](cloud.md) * [Code](code.md) * [Container](container.md) +* [DB](db.md) (database) * [Destination](destination.md) * [Device](device.md) * [Error](error.md) diff --git a/docs/attributes-registry/db.md b/docs/attributes-registry/db.md new file mode 100644 index 0000000000..1964f71282 --- /dev/null +++ b/docs/attributes-registry/db.md @@ -0,0 +1,222 @@ + + +# Database + + + +- [Generic Database Attributes](#generic-database-attributes) +- [Cassandra Attributes](#cassandra-attributes) +- [CosmosDB Attributes](#cosmosdb-attributes) +- [Elasticsearch Attributes](#elasticsearch-attributes) +- [JDBC Attributes](#jdbc-attributes) +- [MongoDB Attributes](#mongodb-attributes) +- [MSSQL Attributes](#mssql-attributes) +- [Redis Attributes](#redis-attributes) +- [SQL Attributes](#sql-attributes) + + + +## Generic Database Attributes + + +| Attribute | Type | Description | Examples | +|---|---|---|---| +| `db.connection_string` | string | The connection string used to connect to the database. It is recommended to remove embedded credentials. | `Server=(localdb)\v11.0;Integrated Security=true;` | +| `db.name` | string | This attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). [1] | `customers`; `main` | +| `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`, or the SQL keyword. [2] | `findAndModify`; `HMSET`; `SELECT` | +| `db.statement` | string | The database statement being executed. | `SELECT * FROM wuser_table`; `SET mykey "WuValue"` | +| `db.system` | string | An identifier for the database management system (DBMS) product being used. See below for a list of well-known identifiers. | `other_sql` | +| `db.user` | string | Username for accessing the database. | `readonly_user`; `reporting_user` | + +**[1]:** In some SQL databases, the database name to be used is called "schema name". In case there are multiple layers that could be considered for database name (e.g. Oracle instance name and schema name), the database name to be used is the more specific layer (e.g. Oracle schema name). + +**[2]:** When setting this to an SQL keyword, it is not recommended to attempt any client-side parsing of `db.statement` just to get this property, but it should be set if the operation name is provided by the library being instrumented. If the SQL statement has an ambiguous operation, or performs more than one operation, this value may be omitted. + +`db.system` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `other_sql` | Some other SQL database. Fallback only. See notes. | +| `mssql` | Microsoft SQL Server | +| `mssqlcompact` | Microsoft SQL Server Compact | +| `mysql` | MySQL | +| `oracle` | Oracle Database | +| `db2` | IBM Db2 | +| `postgresql` | PostgreSQL | +| `redshift` | Amazon Redshift | +| `hive` | Apache Hive | +| `cloudscape` | Cloudscape | +| `hsqldb` | HyperSQL DataBase | +| `progress` | Progress Database | +| `maxdb` | SAP MaxDB | +| `hanadb` | SAP HANA | +| `ingres` | Ingres | +| `firstsql` | FirstSQL | +| `edb` | EnterpriseDB | +| `cache` | InterSystems Caché | +| `adabas` | Adabas (Adaptable Database System) | +| `firebird` | Firebird | +| `derby` | Apache Derby | +| `filemaker` | FileMaker | +| `informix` | Informix | +| `instantdb` | InstantDB | +| `interbase` | InterBase | +| `mariadb` | MariaDB | +| `netezza` | Netezza | +| `pervasive` | Pervasive PSQL | +| `pointbase` | PointBase | +| `sqlite` | SQLite | +| `sybase` | Sybase | +| `teradata` | Teradata | +| `vertica` | Vertica | +| `h2` | H2 | +| `coldfusion` | ColdFusion IMQ | +| `cassandra` | Apache Cassandra | +| `hbase` | Apache HBase | +| `mongodb` | MongoDB | +| `redis` | Redis | +| `couchbase` | Couchbase | +| `couchdb` | CouchDB | +| `cosmosdb` | Microsoft Azure Cosmos DB | +| `dynamodb` | Amazon DynamoDB | +| `neo4j` | Neo4j | +| `geode` | Apache Geode | +| `elasticsearch` | Elasticsearch | +| `memcached` | Memcached | +| `cockroachdb` | CockroachDB | +| `opensearch` | OpenSearch | +| `clickhouse` | ClickHouse | +| `spanner` | Cloud Spanner | +| `trino` | Trino | + + +## Cassandra Attributes + + +| Attribute | Type | Description | Examples | +|---|---|---|---| +| `db.cassandra.consistency_level` | string | The consistency level of the query. Based on consistency values from [CQL](https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/dml/dmlConfigConsistency.html). | `all` | +| `db.cassandra.coordinator.dc` | string | The data center of the coordinating node for a query. | `us-west-2` | +| `db.cassandra.coordinator.id` | string | The ID of the coordinating node for a query. | `be13faa2-8574-4d71-926d-27f16cf8a7af` | +| `db.cassandra.idempotence` | boolean | Whether or not the query is idempotent. | | +| `db.cassandra.page_size` | int | The fetch size used for paging, i.e. how many rows will be returned at once. | `5000` | +| `db.cassandra.speculative_execution_count` | int | The number of times a query was speculatively executed. Not set or `0` if the query was not executed speculatively. | `0`; `2` | +| `db.cassandra.table` | string | The name of the primary Cassandra table that the operation is acting upon, including the keyspace name (if applicable). [1] | `mytable` | + +**[1]:** This mirrors the db.sql.table attribute but references cassandra rather than sql. It is not recommended to attempt any client-side parsing of `db.statement` just to get this property, but it should be set if it is provided by the library being instrumented. If the operation is acting upon an anonymous table, or more than one table, this value MUST NOT be set. + +`db.cassandra.consistency_level` MUST be one of the following: + +| Value | Description | +|---|---| +| `all` | all | +| `each_quorum` | each_quorum | +| `quorum` | quorum | +| `local_quorum` | local_quorum | +| `one` | one | +| `two` | two | +| `three` | three | +| `local_one` | local_one | +| `any` | any | +| `serial` | serial | +| `local_serial` | local_serial | + + +## CosmosDB Attributes + + +| Attribute | Type | Description | Examples | +|---|---|---|---| +| `db.cosmosdb.client_id` | string | Unique Cosmos client instance id. | `3ba4827d-4422-483f-b59f-85b74211c11d` | +| `db.cosmosdb.connection_mode` | string | Cosmos client connection mode. | `gateway` | +| `db.cosmosdb.container` | string | Cosmos DB container name. | `anystring` | +| `db.cosmosdb.operation_type` | string | CosmosDB Operation Type. | `Invalid` | +| `db.cosmosdb.request_charge` | double | RU consumed for that operation | `46.18`; `1.0` | +| `db.cosmosdb.request_content_length` | int | Request payload size in bytes | | +| `db.cosmosdb.status_code` | int | Cosmos DB status code. | `200`; `201` | +| `db.cosmosdb.sub_status_code` | int | Cosmos DB sub status code. | `1000`; `1002` | + +`db.cosmosdb.connection_mode` MUST be one of the following: + +| Value | Description | +|---|---| +| `gateway` | Gateway (HTTP) connections mode | +| `direct` | Direct connection. | + +`db.cosmosdb.operation_type` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `Invalid` | invalid | +| `Create` | create | +| `Patch` | patch | +| `Read` | read | +| `ReadFeed` | read_feed | +| `Delete` | delete | +| `Replace` | replace | +| `Execute` | execute | +| `Query` | query | +| `Head` | head | +| `HeadFeed` | head_feed | +| `Upsert` | upsert | +| `Batch` | batch | +| `QueryPlan` | query_plan | +| `ExecuteJavaScript` | execute_javascript | + + +## Elasticsearch Attributes + + +| Attribute | Type | Description | Examples | +|---|---|---|---| +| `db.elasticsearch.cluster.name` | string | Represents the identifier of an Elasticsearch cluster. | `e9106fc68e3044f0b1475b04bf4ffd5f` | +| `db.elasticsearch.node.name` | string | Represents the human-readable identifier of the node/instance to which a request was routed. | `instance-0000000001` | +| `db.elasticsearch.path_parts.` | string | A dynamic value in the url path. [1] | `db.elasticsearch.path_parts.index=test-index`; `db.elasticsearch.path_parts.doc_id=123` | + +**[1]:** Many Elasticsearch url paths allow dynamic values. These SHOULD be recorded in span attributes in the format `db.elasticsearch.path_parts.`, where `` is the url path part name. The implementation SHOULD reference the [elasticsearch schema](https://raw.githubusercontent.com/elastic/elasticsearch-specification/main/output/schema/schema.json) in order to map the path part values to their names. + + +## JDBC Attributes + + +| Attribute | Type | Description | Examples | +|---|---|---|---| +| `db.jdbc.driver_classname` | string | The fully-qualified class name of the [Java Database Connectivity (JDBC)](https://docs.oracle.com/javase/8/docs/technotes/guides/jdbc/) driver used to connect. | `org.postgresql.Driver`; `com.microsoft.sqlserver.jdbc.SQLServerDriver` | + + +## MongoDB Attributes + + +| Attribute | Type | Description | Examples | +|---|---|---|---| +| `db.mongodb.collection` | string | The MongoDB collection being accessed within the database stated in `db.name`. | `customers`; `products` | + + +## MSSQL Attributes + + +| Attribute | Type | Description | Examples | +|---|---|---|---| +| `db.mssql.instance_name` | string | The Microsoft SQL Server [instance name](https://docs.microsoft.com/sql/connect/jdbc/building-the-connection-url?view=sql-server-ver15) connecting to. This name is used to determine the port of a named instance. [1] | `MSSQLSERVER` | + +**[1]:** If setting a `db.mssql.instance_name`, `server.port` is no longer required (but still recommended if non-standard). + + +## Redis Attributes + + +| Attribute | Type | Description | Examples | +|---|---|---|---| +| `db.redis.database_index` | int | The index of the database being accessed as used in the [`SELECT` command](https://redis.io/commands/select), provided as an integer. To be used instead of the generic `db.name` attribute. | `0`; `1`; `15` | + + +## SQL Attributes + + +| Attribute | Type | Description | Examples | +|---|---|---|---| +| `db.sql.table` | string | The name of the primary table that the operation is acting upon, including the database name (if applicable). [1] | `public.users`; `customers` | + +**[1]:** It is not recommended to attempt any client-side parsing of `db.statement` just to get this property, but it should be set if it is provided by the library being instrumented. If the operation is acting upon an anonymous table, or more than one table, this value MUST NOT be set. + diff --git a/docs/database/cassandra.md b/docs/database/cassandra.md index b6f160457c..0fcc385c82 100644 --- a/docs/database/cassandra.md +++ b/docs/database/cassandra.md @@ -14,17 +14,17 @@ described on this page. ## Call-level attributes - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `db.cassandra.consistency_level` | string | The consistency level of the query. Based on consistency values from [CQL](https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/dml/dmlConfigConsistency.html). | `all` | Recommended | -| `db.cassandra.coordinator.dc` | string | The data center of the coordinating node for a query. | `us-west-2` | Recommended | -| `db.cassandra.coordinator.id` | string | The ID of the coordinating node for a query. | `be13faa2-8574-4d71-926d-27f16cf8a7af` | Recommended | -| `db.cassandra.idempotence` | boolean | Whether or not the query is idempotent. | | Recommended | -| `db.cassandra.page_size` | int | The fetch size used for paging, i.e. how many rows will be returned at once. | `5000` | Recommended | -| `db.cassandra.speculative_execution_count` | int | The number of times a query was speculatively executed. Not set or `0` if the query was not executed speculatively. | `0`; `2` | Recommended | -| `db.cassandra.table` | string | The name of the primary table that the operation is acting upon, including the keyspace name (if applicable). [1] | `mytable` | Recommended | -| [`db.name`](database-spans.md) | string | The keyspace name in Cassandra. [2] | `mykeyspace` | Conditionally Required: If applicable. | +| [`db.cassandra.consistency_level`](../attributes-registry/db.md) | string | The consistency level of the query. Based on consistency values from [CQL](https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/dml/dmlConfigConsistency.html). | `all` | Recommended | +| [`db.cassandra.coordinator.dc`](../attributes-registry/db.md) | string | The data center of the coordinating node for a query. | `us-west-2` | Recommended | +| [`db.cassandra.coordinator.id`](../attributes-registry/db.md) | string | The ID of the coordinating node for a query. | `be13faa2-8574-4d71-926d-27f16cf8a7af` | Recommended | +| [`db.cassandra.idempotence`](../attributes-registry/db.md) | boolean | Whether or not the query is idempotent. | | Recommended | +| [`db.cassandra.page_size`](../attributes-registry/db.md) | int | The fetch size used for paging, i.e. how many rows will be returned at once. | `5000` | Recommended | +| [`db.cassandra.speculative_execution_count`](../attributes-registry/db.md) | int | The number of times a query was speculatively executed. Not set or `0` if the query was not executed speculatively. | `0`; `2` | Recommended | +| [`db.cassandra.table`](../attributes-registry/db.md) | string | The name of the primary Cassandra table that the operation is acting upon, including the keyspace name (if applicable). [1] | `mytable` | Recommended | +| [`db.name`](../attributes-registry/db.md) | string | The keyspace name in Cassandra. [2] | `mykeyspace` | Conditionally Required: If applicable. | **[1]:** This mirrors the db.sql.table attribute but references cassandra rather than sql. It is not recommended to attempt any client-side parsing of `db.statement` just to get this property, but it should be set if it is provided by the library being instrumented. If the operation is acting upon an anonymous table, or more than one table, this value MUST NOT be set. diff --git a/docs/database/cosmosdb.md b/docs/database/cosmosdb.md index 3c2669879c..1862a1efa7 100644 --- a/docs/database/cosmosdb.md +++ b/docs/database/cosmosdb.md @@ -17,17 +17,17 @@ described on this page. Cosmos DB instrumentation includes call-level (public API) surface spans and network spans. Depending on the connection mode (Gateway or Direct), network-level spans may also be created. - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `db.cosmosdb.client_id` | string | Unique Cosmos client instance id. | `3ba4827d-4422-483f-b59f-85b74211c11d` | Recommended | -| `db.cosmosdb.connection_mode` | string | Cosmos client connection mode. | `gateway` | Conditionally Required: if not `direct` (or pick gw as default) | -| `db.cosmosdb.container` | string | Cosmos DB container name. | `anystring` | Conditionally Required: if available | -| `db.cosmosdb.operation_type` | string | CosmosDB Operation Type. | `Invalid` | Conditionally Required: [1] | -| `db.cosmosdb.request_charge` | double | RU consumed for that operation | `46.18`; `1.0` | Conditionally Required: when available | -| `db.cosmosdb.request_content_length` | int | Request payload size in bytes | | Recommended | -| `db.cosmosdb.status_code` | int | Cosmos DB status code. | `200`; `201` | Conditionally Required: if response was received | -| `db.cosmosdb.sub_status_code` | int | Cosmos DB sub status code. | `1000`; `1002` | Conditionally Required: [2] | +| [`db.cosmosdb.client_id`](../attributes-registry/db.md) | string | Unique Cosmos client instance id. | `3ba4827d-4422-483f-b59f-85b74211c11d` | Recommended | +| [`db.cosmosdb.connection_mode`](../attributes-registry/db.md) | string | Cosmos client connection mode. | `gateway` | Conditionally Required: if not `direct` (or pick gw as default) | +| [`db.cosmosdb.container`](../attributes-registry/db.md) | string | Cosmos DB container name. | `anystring` | Conditionally Required: if available | +| [`db.cosmosdb.operation_type`](../attributes-registry/db.md) | string | CosmosDB Operation Type. | `Invalid` | Conditionally Required: [1] | +| [`db.cosmosdb.request_charge`](../attributes-registry/db.md) | double | RU consumed for that operation | `46.18`; `1.0` | Conditionally Required: when available | +| [`db.cosmosdb.request_content_length`](../attributes-registry/db.md) | int | Request payload size in bytes | | Recommended | +| [`db.cosmosdb.status_code`](../attributes-registry/db.md) | int | Cosmos DB status code. | `200`; `201` | Conditionally Required: if response was received | +| [`db.cosmosdb.sub_status_code`](../attributes-registry/db.md) | int | Cosmos DB sub status code. | `1000`; `1002` | Conditionally Required: [2] | | [`user_agent.original`](../attributes-registry/user-agent.md) | string | Full user-agent string is generated by Cosmos DB SDK [3] | `cosmos-netstandard-sdk/3.23.0\|3.23.1\|1\|X64\|Linux 5.4.0-1098-azure 104 18\|.NET Core 3.1.32\|S\|` | Recommended | **[1]:** when performing one of the operations in this list diff --git a/docs/database/couchdb.md b/docs/database/couchdb.md index 3ee335d759..6702a92b40 100644 --- a/docs/database/couchdb.md +++ b/docs/database/couchdb.md @@ -14,10 +14,10 @@ described on this page. ## Call-level attributes - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| [`db.operation`](database-spans.md) | string | The HTTP method + the target REST route. [1] | `GET /{db}/{docid}` | Conditionally Required: If `db.statement` is not applicable. | +| [`db.operation`](../attributes-registry/db.md) | string | The HTTP method + the target REST route. [1] | `GET /{db}/{docid}` | Conditionally Required: If `db.statement` is not applicable. | **[1]:** In **CouchDB**, `db.operation` should be set to the HTTP method + the target REST route according to the API reference documentation. For example, when retrieving a document, `db.operation` would be set to (literally, i.e., without replacing the placeholders with concrete values): [`GET /{db}/{docid}`](http://docs.couchdb.org/en/stable/api/document/common.html#get--db-docid). diff --git a/docs/database/database-spans.md b/docs/database/database-spans.md index 1d3f6471f9..b6cf27a20f 100644 --- a/docs/database/database-spans.md +++ b/docs/database/database-spans.md @@ -61,12 +61,12 @@ When it's otherwise impossible to get any meaningful span name, `db.name` or the These attributes will usually be the same for all operations performed over the same database connection. Some database systems may allow a connection to switch to a different `db.user`, for example, and other database systems may not even have the concept of a connection at all. - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `db.connection_string` | string | The connection string used to connect to the database. It is recommended to remove embedded credentials. | `Server=(localdb)\v11.0;Integrated Security=true;` | Recommended | -| `db.system` | string | An identifier for the database management system (DBMS) product being used. See below for a list of well-known identifiers. | `other_sql` | Required | -| `db.user` | string | Username for accessing the database. | `readonly_user`; `reporting_user` | Recommended | +| [`db.connection_string`](../attributes-registry/db.md) | string | The connection string used to connect to the database. It is recommended to remove embedded credentials. | `Server=(localdb)\v11.0;Integrated Security=true;` | Recommended | +| [`db.system`](../attributes-registry/db.md) | string | An identifier for the database management system (DBMS) product being used. See below for a list of well-known identifiers. | `other_sql` | Required | +| [`db.user`](../attributes-registry/db.md) | string | Username for accessing the database. | `readonly_user`; `reporting_user` | Recommended | | [`network.peer.address`](../attributes-registry/network.md) | string | Peer address of the network connection - IP address or Unix domain socket name. | `10.1.2.80`; `/tmp/my.sock` | Recommended | | [`network.peer.port`](../attributes-registry/network.md) | int | Peer port number of the network connection. | `65123` | Recommended: If `network.peer.address` is set. | | [`network.transport`](../attributes-registry/network.md) | string | [OSI transport layer](https://osi-model.com/transport-layer/) or [inter-process communication method](https://wikipedia.org/wiki/Inter-process_communication). [1] | `tcp`; `udp` | Recommended | @@ -144,6 +144,22 @@ different processes could be listening on TCP port 12345 and UDP port 12345. | `clickhouse` | ClickHouse | | `spanner` | Cloud Spanner | | `trino` | Trino | + +`network.transport` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `tcp` | TCP | +| `udp` | UDP | +| `pipe` | Named or anonymous pipe. | +| `unix` | Unix domain socket | + +`network.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `ipv4` | IPv4 | +| `ipv6` | IPv6 | ### Notes and well-known identifiers for `db.system` @@ -169,12 +185,12 @@ When additional attributes are added that only apply to a specific DBMS, its ide These attributes may be different for each operation performed, even if the same connection is used for multiple operations. Usually only one `db.name` will be used per connection though. - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `db.name` | string | This attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). [1] | `customers`; `main` | Conditionally Required: If applicable. | -| `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`, or the SQL keyword. [2] | `findAndModify`; `HMSET`; `SELECT` | Conditionally Required: If `db.statement` is not applicable. | -| `db.statement` | string | The database statement being executed. | `SELECT * FROM wuser_table`; `SET mykey "WuValue"` | Recommended: [3] | +| [`db.name`](../attributes-registry/db.md) | string | This attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). [1] | `customers`; `main` | Conditionally Required: If applicable. | +| [`db.operation`](../attributes-registry/db.md) | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`, or the SQL keyword. [2] | `findAndModify`; `HMSET`; `SELECT` | Conditionally Required: If `db.statement` is not applicable. | +| [`db.statement`](../attributes-registry/db.md) | string | The database statement being executed. | `SELECT * FROM wuser_table`; `SET mykey "WuValue"` | Recommended: [3] | **[1]:** In some SQL databases, the database name to be used is called "schema name". In case there are multiple layers that could be considered for database name (e.g. Oracle instance name and schema name), the database name to be used is the more specific layer (e.g. Oracle schema name). diff --git a/docs/database/dynamodb.md b/docs/database/dynamodb.md index 61299804a6..137ebc6f37 100644 --- a/docs/database/dynamodb.md +++ b/docs/database/dynamodb.md @@ -19,7 +19,7 @@ These attributes are filled in for all DynamoDB request types. | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| [`db.system`](database-spans.md) | string | The value `dynamodb`. | `dynamodb` | Required | +| [`db.system`](../attributes-registry/db.md) | string | The value `dynamodb`. | `dynamodb` | Required | ## DynamoDB.BatchGetItem diff --git a/docs/database/elasticsearch.md b/docs/database/elasticsearch.md index 0dc1a722fa..156e7cb15f 100644 --- a/docs/database/elasticsearch.md +++ b/docs/database/elasticsearch.md @@ -23,14 +23,14 @@ If the endpoint id is not available, the span name SHOULD be the `http.request.m ## Call-level attributes - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `db.elasticsearch.cluster.name` | string | Represents the identifier of an Elasticsearch cluster. | `e9106fc68e3044f0b1475b04bf4ffd5f` | Recommended: [1] | -| `db.elasticsearch.node.name` | string | Represents the human-readable identifier of the node/instance to which a request was routed. | `instance-0000000001` | Recommended: [2] | -| `db.elasticsearch.path_parts.` | string | A dynamic value in the url path. [3] | `db.elasticsearch.path_parts.index=test-index`; `db.elasticsearch.path_parts.doc_id=123` | Conditionally Required: when the url has dynamic values | -| [`db.operation`](database-spans.md) | string | The endpoint identifier for the request. [4] | `search`; `ml.close_job`; `cat.aliases` | Required | -| [`db.statement`](database-spans.md) | string | The request body for a [search-type query](https://www.elastic.co/guide/en/elasticsearch/reference/current/search.html), as a json string. | `"{\"query\":{\"term\":{\"user.id\":\"kimchy\"}}}"` | Recommended: [5] | +| [`db.elasticsearch.cluster.name`](../attributes-registry/db.md) | string | Represents the identifier of an Elasticsearch cluster. | `e9106fc68e3044f0b1475b04bf4ffd5f` | Recommended: [1] | +| [`db.elasticsearch.node.name`](../attributes-registry/db.md) | string | Represents the human-readable identifier of the node/instance to which a request was routed. | `instance-0000000001` | Recommended: [2] | +| [`db.elasticsearch.path_parts.`](../attributes-registry/db.md) | string | A dynamic value in the url path. [3] | `db.elasticsearch.path_parts.index=test-index`; `db.elasticsearch.path_parts.doc_id=123` | Conditionally Required: when the url has dynamic values | +| [`db.operation`](../attributes-registry/db.md) | string | The endpoint identifier for the request. [4] | `search`; `ml.close_job`; `cat.aliases` | Required | +| [`db.statement`](../attributes-registry/db.md) | string | The request body for a [search-type query](https://www.elastic.co/guide/en/elasticsearch/reference/current/search.html), as a json string. | `"{\"query\":{\"term\":{\"user.id\":\"kimchy\"}}}"` | Recommended: [5] | | [`http.request.method`](../attributes-registry/http.md) | string | HTTP request method. [6] | `GET`; `POST`; `HEAD` | Required | | [`server.address`](../attributes-registry/server.md) | string | Name of the database host. [7] | `example.com`; `10.1.2.80`; `/tmp/my.sock` | Recommended | | [`server.port`](../attributes-registry/server.md) | int | Server port number. [8] | `80`; `8080`; `443` | Conditionally Required: [9] | @@ -70,6 +70,21 @@ Tracing instrumentations that do so, MUST also set `http.request.method_original **[10]:** For network calls, URL usually has `scheme://host[:port][path][?query][#fragment]` format, where the fragment is not transmitted over HTTP, but if it is known, it SHOULD be included nevertheless. `url.full` MUST NOT contain credentials passed via URL in form of `https://username:password@www.example.com/`. In such case username and password SHOULD be redacted and attribute's value SHOULD be `https://REDACTED:REDACTED@www.example.com/`. `url.full` SHOULD capture the absolute URL when it is available (or can be reconstructed) and SHOULD NOT be validated or modified except for sanitizing purposes. + +`http.request.method` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. + +| Value | Description | +|---|---| +| `CONNECT` | CONNECT method. | +| `DELETE` | DELETE method. | +| `GET` | GET method. | +| `HEAD` | HEAD method. | +| `OPTIONS` | OPTIONS method. | +| `PATCH` | PATCH method. | +| `POST` | POST method. | +| `PUT` | PUT method. | +| `TRACE` | TRACE method. | +| `_OTHER` | Any HTTP method that the instrumentation has no prior knowledge of. | ## Example diff --git a/docs/database/hbase.md b/docs/database/hbase.md index 8cbc9d41a1..47b8acd2c5 100644 --- a/docs/database/hbase.md +++ b/docs/database/hbase.md @@ -14,10 +14,10 @@ described on this page. ## Call-level attributes - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| [`db.name`](database-spans.md) | string | The HBase namespace. [1] | `mynamespace` | Conditionally Required: If applicable. | +| [`db.name`](../attributes-registry/db.md) | string | The HBase namespace. [1] | `mynamespace` | Conditionally Required: If applicable. | **[1]:** For HBase the `db.name` should be set to the HBase namespace. diff --git a/docs/database/mongodb.md b/docs/database/mongodb.md index ef76976e27..10fdbb785e 100644 --- a/docs/database/mongodb.md +++ b/docs/database/mongodb.md @@ -14,10 +14,10 @@ described on this page. ## Call-level attributes - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `db.mongodb.collection` | string | The collection being accessed within the database stated in `db.name`. | `customers`; `products` | Required | +| [`db.mongodb.collection`](../attributes-registry/db.md) | string | The MongoDB collection being accessed within the database stated in `db.name`. | `customers`; `products` | Required | ## Example diff --git a/docs/database/mssql.md b/docs/database/mssql.md index 4873f48d11..763e7a1b12 100644 --- a/docs/database/mssql.md +++ b/docs/database/mssql.md @@ -14,11 +14,11 @@ described on this page. ## Connection-level attributes - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| [`db.jdbc.driver_classname`](database-spans.md) | string | The fully-qualified class name of the [Java Database Connectivity (JDBC)](https://docs.oracle.com/javase/8/docs/technotes/guides/jdbc/) driver used to connect. | `org.postgresql.Driver`; `com.microsoft.sqlserver.jdbc.SQLServerDriver` | Recommended | -| `db.mssql.instance_name` | string | The Microsoft SQL Server [instance name](https://docs.microsoft.com/sql/connect/jdbc/building-the-connection-url?view=sql-server-ver15) connecting to. This name is used to determine the port of a named instance. [1] | `MSSQLSERVER` | Recommended | +| [`db.jdbc.driver_classname`](../attributes-registry/db.md) | string | The fully-qualified class name of the [Java Database Connectivity (JDBC)](https://docs.oracle.com/javase/8/docs/technotes/guides/jdbc/) driver used to connect. | `org.postgresql.Driver`; `com.microsoft.sqlserver.jdbc.SQLServerDriver` | Recommended | +| [`db.mssql.instance_name`](../attributes-registry/db.md) | string | The Microsoft SQL Server [instance name](https://docs.microsoft.com/sql/connect/jdbc/building-the-connection-url?view=sql-server-ver15) connecting to. This name is used to determine the port of a named instance. [1] | `MSSQLSERVER` | Recommended | **[1]:** If setting a `db.mssql.instance_name`, `server.port` is no longer required (but still recommended if non-standard). diff --git a/docs/database/redis.md b/docs/database/redis.md index 85efb0104f..2614e494fa 100644 --- a/docs/database/redis.md +++ b/docs/database/redis.md @@ -14,11 +14,11 @@ described on this page. ## Call-level attributes - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `db.redis.database_index` | int | The index of the database being accessed as used in the [`SELECT` command](https://redis.io/commands/select), provided as an integer. To be used instead of the generic `db.name` attribute. | `0`; `1`; `15` | Conditionally Required: If other than the default database (`0`). | -| [`db.statement`](database-spans.md) | string | The full syntax of the Redis CLI command. [1] | `HMSET myhash field1 'Hello' field2 'World'` | Recommended: [2] | +| [`db.redis.database_index`](../attributes-registry/db.md) | int | The index of the database being accessed as used in the [`SELECT` command](https://redis.io/commands/select), provided as an integer. To be used instead of the generic `db.name` attribute. | `0`; `1`; `15` | Conditionally Required: If other than the default database (`0`). | +| [`db.statement`](../attributes-registry/db.md) | string | The full syntax of the Redis CLI command. [1] | `HMSET myhash field1 'Hello' field2 'World'` | Recommended: [2] | **[1]:** For **Redis**, the value provided for `db.statement` SHOULD correspond to the syntax of the Redis CLI. If, for example, the [`HMSET` command](https://redis.io/commands/hmset) is invoked, `"HMSET myhash field1 'Hello' field2 'World'"` would be a suitable value for `db.statement`. diff --git a/docs/database/sql.md b/docs/database/sql.md index 2bda71546a..6483e36374 100644 --- a/docs/database/sql.md +++ b/docs/database/sql.md @@ -12,10 +12,10 @@ described on this page. ## Call-level attributes - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `db.sql.table` | string | The name of the primary table that the operation is acting upon, including the database name (if applicable). [1] | `public.users`; `customers` | Recommended | +| [`db.sql.table`](../attributes-registry/db.md) | string | The name of the primary table that the operation is acting upon, including the database name (if applicable). [1] | `public.users`; `customers` | Recommended | **[1]:** It is not recommended to attempt any client-side parsing of `db.statement` just to get this property, but it should be set if it is provided by the library being instrumented. If the operation is acting upon an anonymous table, or more than one table, this value MUST NOT be set. diff --git a/model/registry/db.yaml b/model/registry/db.yaml new file mode 100644 index 0000000000..3ec2d84c66 --- /dev/null +++ b/model/registry/db.yaml @@ -0,0 +1,424 @@ +groups: + - id: registry.db + prefix: db + type: attribute_group + brief: > + This document defines the attributes used to describe telemetry in the context of databases. + attributes: + - id: cassandra.coordinator.dc + type: string + brief: > + The data center of the coordinating node for a query. + examples: 'us-west-2' + tag: tech-specific-cassandra + - id: cassandra.coordinator.id + type: string + brief: > + The ID of the coordinating node for a query. + examples: 'be13faa2-8574-4d71-926d-27f16cf8a7af' + tag: tech-specific-cassandra + - id: cassandra.consistency_level + brief: > + The consistency level of the query. Based on consistency values from [CQL](https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/dml/dmlConfigConsistency.html). + type: + members: + - id: all + value: 'all' + - id: each_quorum + value: 'each_quorum' + - id: quorum + value: 'quorum' + - id: local_quorum + value: 'local_quorum' + - id: one + value: 'one' + - id: two + value: 'two' + - id: three + value: 'three' + - id: local_one + value: 'local_one' + - id: any + value: 'any' + - id: serial + value: 'serial' + - id: local_serial + value: 'local_serial' + tag: tech-specific-cassandra + - id: cassandra.idempotence + type: boolean + brief: > + Whether or not the query is idempotent. + tag: tech-specific-cassandra + - id: cassandra.page_size + type: int + brief: > + The fetch size used for paging, i.e. how many rows will be returned at once. + examples: [5000] + tag: tech-specific-cassandra + - id: cassandra.speculative_execution_count + type: int + brief: > + The number of times a query was speculatively executed. Not set or `0` if the query was not executed speculatively. + examples: [0, 2] + tag: tech-specific-cassandra + - id: cassandra.table + type: string + brief: The name of the primary Cassandra table that the operation is acting upon, including the keyspace name (if applicable). + note: > + This mirrors the db.sql.table attribute but references cassandra rather than sql. + It is not recommended to attempt any client-side parsing of + `db.statement` just to get this property, but it should be set if + it is provided by the library being instrumented. + If the operation is acting upon an anonymous table, or more than one table, this + value MUST NOT be set. + examples: 'mytable' + tag: tech-specific-cassandra + - id: connection_string + type: string + brief: > + The connection string used to connect to the database. + It is recommended to remove embedded credentials. + examples: 'Server=(localdb)\v11.0;Integrated Security=true;' + tag: db-generic + - id: cosmosdb.client_id + type: string + brief: Unique Cosmos client instance id. + examples: '3ba4827d-4422-483f-b59f-85b74211c11d' + tag: tech-specific-cosmosdb + - id: cosmosdb.connection_mode + type: + allow_custom_values: false + members: + - id: gateway + value: 'gateway' + brief: Gateway (HTTP) connections mode + - id: direct + value: 'direct' + brief: Direct connection. + brief: Cosmos client connection mode. + tag: tech-specific-cosmosdb + - id: cosmosdb.container + type: string + brief: Cosmos DB container name. + examples: 'anystring' + tag: tech-specific-cosmosdb + - id: cosmosdb.operation_type + type: + allow_custom_values: true + members: + - id: invalid + value: 'Invalid' + - id: create + value: 'Create' + - id: patch + value: 'Patch' + - id: read + value: 'Read' + - id: read_feed + value: 'ReadFeed' + - id: delete + value: 'Delete' + - id: replace + value: 'Replace' + - id: execute + value: 'Execute' + - id: query + value: 'Query' + - id: head + value: 'Head' + - id: head_feed + value: 'HeadFeed' + - id: upsert + value: 'Upsert' + - id: batch + value: 'Batch' + - id: query_plan + value: 'QueryPlan' + - id: execute_javascript + value: 'ExecuteJavaScript' + brief: CosmosDB Operation Type. + tag: tech-specific-cosmosdb + - id: cosmosdb.request_charge + type: double + brief: RU consumed for that operation + examples: [46.18, 1.0] + tag: tech-specific-cosmosdb + - id: cosmosdb.request_content_length + type: int + brief: Request payload size in bytes + tag: tech-specific-cosmosdb + - id: cosmosdb.status_code + type: int + brief: Cosmos DB status code. + examples: [200, 201] + tag: tech-specific-cosmosdb + - id: cosmosdb.sub_status_code + type: int + brief: Cosmos DB sub status code. + examples: [1000, 1002] + tag: tech-specific-cosmosdb + - id: elasticsearch.cluster.name + type: string + brief: > + Represents the identifier of an Elasticsearch cluster. + examples: ["e9106fc68e3044f0b1475b04bf4ffd5f"] + tag: tech-specific-elasticsearch + - id: elasticsearch.node.name + type: string + brief: > + Represents the human-readable identifier of the node/instance to which a request was routed. + examples: ["instance-0000000001"] + tag: tech-specific-elasticsearch + - id: elasticsearch.path_parts + type: template[string] + brief: > + A dynamic value in the url path. + note: > + Many Elasticsearch url paths allow dynamic values. These SHOULD be recorded in span attributes in the format + `db.elasticsearch.path_parts.`, where `` is the url path part name. The implementation SHOULD + reference the [elasticsearch schema](https://raw.githubusercontent.com/elastic/elasticsearch-specification/main/output/schema/schema.json) + in order to map the path part values to their names. + examples: ['db.elasticsearch.path_parts.index=test-index', 'db.elasticsearch.path_parts.doc_id=123'] + tag: tech-specific-elasticsearch + - id: jdbc.driver_classname + type: string + brief: > + The fully-qualified class name of the [Java Database Connectivity (JDBC)](https://docs.oracle.com/javase/8/docs/technotes/guides/jdbc/) driver used to connect. + examples: ['org.postgresql.Driver', 'com.microsoft.sqlserver.jdbc.SQLServerDriver'] + tag: tech-specific-jdbc + - id: mongodb.collection + type: string + brief: > + The MongoDB collection being accessed within the database stated in `db.name`. + examples: [ 'customers', 'products' ] + tag: tech-specific-mongodb + - id: mssql.instance_name + type: string + note: > + If setting a `db.mssql.instance_name`, `server.port` is no longer + required (but still recommended if non-standard). + brief: > + The Microsoft SQL Server [instance name](https://docs.microsoft.com/sql/connect/jdbc/building-the-connection-url?view=sql-server-ver15) + connecting to. This name is used to determine the port of a named instance. + examples: 'MSSQLSERVER' + tag: tech-specific-mssql + - id: name + type: string + brief: > + This attribute is used to report the name of the database being accessed. + For commands that switch the database, this should be set to the target database + (even if the command fails). + note: > + In some SQL databases, the database name to be used is called "schema name". + In case there are multiple layers that could be considered for database name + (e.g. Oracle instance name and schema name), + the database name to be used is the more specific layer (e.g. Oracle schema name). + examples: [ 'customers', 'main' ] + tag: db-generic + - id: operation + type: string + brief: > + The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) + such as `findAndModify`, or the SQL keyword. + note: > + When setting this to an SQL keyword, it is not recommended to + attempt any client-side parsing of `db.statement` just to get this + property, but it should be set if the operation name is provided by + the library being instrumented. + If the SQL statement has an ambiguous operation, or performs more + than one operation, this value may be omitted. + examples: ['findAndModify', 'HMSET', 'SELECT'] + tag: db-generic + - id: redis.database_index + type: int + brief: > + The index of the database being accessed as used in the [`SELECT` command](https://redis.io/commands/select), provided as an integer. + To be used instead of the generic `db.name` attribute. + examples: [0, 1, 15] + tag: tech-specific-redis + - id: sql.table + type: string + brief: The name of the primary table that the operation is acting upon, including the database name (if applicable). + note: > + It is not recommended to attempt any client-side parsing of + `db.statement` just to get this property, but it should be set if + it is provided by the library being instrumented. + If the operation is acting upon an anonymous table, or more than one table, this + value MUST NOT be set. + examples: ['public.users', 'customers'] + tag: tech-specific-sql + - id: statement + type: string + brief: > + The database statement being executed. + examples: ['SELECT * FROM wuser_table', 'SET mykey "WuValue"'] + tag: db-generic + - id: system + brief: An identifier for the database management system (DBMS) product being used. See below for a list of well-known identifiers. + type: + allow_custom_values: true + members: + - id: other_sql + value: 'other_sql' + brief: 'Some other SQL database. Fallback only. See notes.' + - id: mssql + value: 'mssql' + brief: 'Microsoft SQL Server' + - id: mssqlcompact + value: 'mssqlcompact' + brief: 'Microsoft SQL Server Compact' + - id: mysql + value: 'mysql' + brief: 'MySQL' + - id: oracle + value: 'oracle' + brief: 'Oracle Database' + - id: db2 + value: 'db2' + brief: 'IBM Db2' + - id: postgresql + value: 'postgresql' + brief: 'PostgreSQL' + - id: redshift + value: 'redshift' + brief: 'Amazon Redshift' + - id: hive + value: 'hive' + brief: 'Apache Hive' + - id: cloudscape + value: 'cloudscape' + brief: 'Cloudscape' + - id: hsqldb + value: 'hsqldb' + brief: 'HyperSQL DataBase' + - id: progress + value: 'progress' + brief: 'Progress Database' + - id: maxdb + value: 'maxdb' + brief: 'SAP MaxDB' + - id: hanadb + value: 'hanadb' + brief: 'SAP HANA' + - id: ingres + value: 'ingres' + brief: 'Ingres' + - id: firstsql + value: 'firstsql' + brief: 'FirstSQL' + - id: edb + value: 'edb' + brief: 'EnterpriseDB' + - id: cache + value: 'cache' + brief: 'InterSystems Caché' + - id: adabas + value: 'adabas' + brief: 'Adabas (Adaptable Database System)' + - id: firebird + value: 'firebird' + brief: 'Firebird' + - id: derby + value: 'derby' + brief: 'Apache Derby' + - id: filemaker + value: 'filemaker' + brief: 'FileMaker' + - id: informix + value: 'informix' + brief: 'Informix' + - id: instantdb + value: 'instantdb' + brief: 'InstantDB' + - id: interbase + value: 'interbase' + brief: 'InterBase' + - id: mariadb + value: 'mariadb' + brief: 'MariaDB' + - id: netezza + value: 'netezza' + brief: 'Netezza' + - id: pervasive + value: 'pervasive' + brief: 'Pervasive PSQL' + - id: pointbase + value: 'pointbase' + brief: 'PointBase' + - id: sqlite + value: 'sqlite' + brief: 'SQLite' + - id: sybase + value: 'sybase' + brief: 'Sybase' + - id: teradata + value: 'teradata' + brief: 'Teradata' + - id: vertica + value: 'vertica' + brief: 'Vertica' + - id: h2 + value: 'h2' + brief: 'H2' + - id: coldfusion + value: 'coldfusion' + brief: 'ColdFusion IMQ' + - id: cassandra + value: 'cassandra' + brief: 'Apache Cassandra' + - id: hbase + value: 'hbase' + brief: 'Apache HBase' + - id: mongodb + value: 'mongodb' + brief: 'MongoDB' + - id: redis + value: 'redis' + brief: 'Redis' + - id: couchbase + value: 'couchbase' + brief: 'Couchbase' + - id: couchdb + value: 'couchdb' + brief: 'CouchDB' + - id: cosmosdb + value: 'cosmosdb' + brief: 'Microsoft Azure Cosmos DB' + - id: dynamodb + value: 'dynamodb' + brief: 'Amazon DynamoDB' + - id: neo4j + value: 'neo4j' + brief: 'Neo4j' + - id: geode + value: 'geode' + brief: 'Apache Geode' + - id: elasticsearch + value: 'elasticsearch' + brief: 'Elasticsearch' + - id: memcached + value: 'memcached' + brief: 'Memcached' + - id: cockroachdb + value: 'cockroachdb' + brief: 'CockroachDB' + - id: opensearch + value: 'opensearch' + brief: 'OpenSearch' + - id: clickhouse + value: 'clickhouse' + brief: 'ClickHouse' + - id: spanner + value: 'spanner' + brief: 'Cloud Spanner' + - id: trino + value: 'trino' + brief: 'Trino' + tag: db-generic + - id: user + type: string + brief: > + Username for accessing the database. + examples: ['readonly_user', 'reporting_user'] + tag: db-generic diff --git a/model/trace/database.yaml b/model/trace/database.yaml index 7540e58c15..ed4f57161a 100644 --- a/model/trace/database.yaml +++ b/model/trace/database.yaml @@ -1,233 +1,33 @@ groups: - id: db - prefix: db type: span brief: > This document defines the attributes used to perform database client calls. span_kind: client attributes: - - id: system + - ref: db.system tag: connection-level - brief: An identifier for the database management system (DBMS) product being used. See below for a list of well-known identifiers. requirement_level: required - type: - allow_custom_values: true - members: - - id: other_sql - value: 'other_sql' - brief: 'Some other SQL database. Fallback only. See notes.' - - id: mssql - value: 'mssql' - brief: 'Microsoft SQL Server' - - id: mssqlcompact - value: 'mssqlcompact' - brief: 'Microsoft SQL Server Compact' - - id: mysql - value: 'mysql' - brief: 'MySQL' - - id: oracle - value: 'oracle' - brief: 'Oracle Database' - - id: db2 - value: 'db2' - brief: 'IBM Db2' - - id: postgresql - value: 'postgresql' - brief: 'PostgreSQL' - - id: redshift - value: 'redshift' - brief: 'Amazon Redshift' - - id: hive - value: 'hive' - brief: 'Apache Hive' - - id: cloudscape - value: 'cloudscape' - brief: 'Cloudscape' - - id: hsqldb - value: 'hsqldb' - brief: 'HyperSQL DataBase' - - id: progress - value: 'progress' - brief: 'Progress Database' - - id: maxdb - value: 'maxdb' - brief: 'SAP MaxDB' - - id: hanadb - value: 'hanadb' - brief: 'SAP HANA' - - id: ingres - value: 'ingres' - brief: 'Ingres' - - id: firstsql - value: 'firstsql' - brief: 'FirstSQL' - - id: edb - value: 'edb' - brief: 'EnterpriseDB' - - id: cache - value: 'cache' - brief: 'InterSystems Caché' - - id: adabas - value: 'adabas' - brief: 'Adabas (Adaptable Database System)' - - id: firebird - value: 'firebird' - brief: 'Firebird' - - id: derby - value: 'derby' - brief: 'Apache Derby' - - id: filemaker - value: 'filemaker' - brief: 'FileMaker' - - id: informix - value: 'informix' - brief: 'Informix' - - id: instantdb - value: 'instantdb' - brief: 'InstantDB' - - id: interbase - value: 'interbase' - brief: 'InterBase' - - id: mariadb - value: 'mariadb' - brief: 'MariaDB' - - id: netezza - value: 'netezza' - brief: 'Netezza' - - id: pervasive - value: 'pervasive' - brief: 'Pervasive PSQL' - - id: pointbase - value: 'pointbase' - brief: 'PointBase' - - id: sqlite - value: 'sqlite' - brief: 'SQLite' - - id: sybase - value: 'sybase' - brief: 'Sybase' - - id: teradata - value: 'teradata' - brief: 'Teradata' - - id: vertica - value: 'vertica' - brief: 'Vertica' - - id: h2 - value: 'h2' - brief: 'H2' - - id: coldfusion - value: 'coldfusion' - brief: 'ColdFusion IMQ' - - id: cassandra - value: 'cassandra' - brief: 'Apache Cassandra' - - id: hbase - value: 'hbase' - brief: 'Apache HBase' - - id: mongodb - value: 'mongodb' - brief: 'MongoDB' - - id: redis - value: 'redis' - brief: 'Redis' - - id: couchbase - value: 'couchbase' - brief: 'Couchbase' - - id: couchdb - value: 'couchdb' - brief: 'CouchDB' - - id: cosmosdb - value: 'cosmosdb' - brief: 'Microsoft Azure Cosmos DB' - - id: dynamodb - value: 'dynamodb' - brief: 'Amazon DynamoDB' - - id: neo4j - value: 'neo4j' - brief: 'Neo4j' - - id: geode - value: 'geode' - brief: 'Apache Geode' - - id: elasticsearch - value: 'elasticsearch' - brief: 'Elasticsearch' - - id: memcached - value: 'memcached' - brief: 'Memcached' - - id: cockroachdb - value: 'cockroachdb' - brief: 'CockroachDB' - - id: opensearch - value: 'opensearch' - brief: 'OpenSearch' - - id: clickhouse - value: 'clickhouse' - brief: 'ClickHouse' - - id: spanner - value: 'spanner' - brief: 'Cloud Spanner' - - id: trino - value: 'trino' - brief: 'Trino' - - id: connection_string + + - ref: db.connection_string tag: connection-level - type: string - brief: > - The connection string used to connect to the database. - It is recommended to remove embedded credentials. - examples: 'Server=(localdb)\v11.0;Integrated Security=true;' - - id: user + - ref: db.user tag: connection-level - type: string - brief: > - Username for accessing the database. - examples: ['readonly_user', 'reporting_user'] - - id: jdbc.driver_classname + - ref: db.jdbc.driver_classname tag: connection-level-tech-specific - type: string - brief: > - The fully-qualified class name of the [Java Database Connectivity (JDBC)](https://docs.oracle.com/javase/8/docs/technotes/guides/jdbc/) driver used to connect. - examples: ['org.postgresql.Driver', 'com.microsoft.sqlserver.jdbc.SQLServerDriver'] - - id: name + - ref: db.name tag: call-level - type: string requirement_level: conditionally_required: If applicable. - brief: > - This attribute is used to report the name of the database being accessed. - For commands that switch the database, this should be set to the target database - (even if the command fails). - note: > - In some SQL databases, the database name to be used is called "schema name". - In case there are multiple layers that could be considered for database name - (e.g. Oracle instance name and schema name), - the database name to be used is the more specific layer (e.g. Oracle schema name). - examples: [ 'customers', 'main' ] - - id: statement + - ref: db.statement tag: call-level - type: string requirement_level: recommended: > Should be collected by default only if there is sanitization that excludes sensitive information. - brief: > - The database statement being executed. - examples: ['SELECT * FROM wuser_table', 'SET mykey "WuValue"'] - - id: operation + - ref: db.operation tag: call-level - type: string requirement_level: conditionally_required: If `db.statement` is not applicable. - brief: > - The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) - such as `findAndModify`, or the SQL keyword. - note: > - When setting this to an SQL keyword, it is not recommended to - attempt any client-side parsing of `db.statement` just to get this - property, but it should be set if the operation name is provided by - the library being instrumented. - If the SQL statement has an ambiguous operation, or performs more - than one operation, this value may be omitted. - examples: ['findAndModify', 'HMSET', 'SELECT'] - ref: server.address tag: connection-level brief: > @@ -248,25 +48,15 @@ groups: tag: connection-level - id: db.mssql - prefix: db.mssql type: span extends: db brief: > Connection-level attributes for Microsoft SQL Server attributes: - - id: instance_name + - ref: db.mssql.instance_name tag: connection-level-tech-specific - type: string - note: > - If setting a `db.mssql.instance_name`, `server.port` is no longer - required (but still recommended if non-standard). - brief: > - The Microsoft SQL Server [instance name](https://docs.microsoft.com/sql/connect/jdbc/building-the-connection-url?view=sql-server-ver15) - connecting to. This name is used to determine the port of a named instance. - examples: 'MSSQLSERVER' - id: db.cassandra - prefix: db.cassandra type: span extends: db brief: > @@ -278,79 +68,22 @@ groups: The keyspace name in Cassandra. examples: ["mykeyspace"] note: For Cassandra the `db.name` should be set to the Cassandra keyspace name. - - id: page_size - type: int + - ref: db.cassandra.page_size tag: call-level-tech-specific-cassandra - brief: > - The fetch size used for paging, i.e. how many rows will be returned at once. - examples: [5000] - - id: consistency_level + - ref: db.cassandra.consistency_level tag: call-level-tech-specific-cassandra - brief: > - The consistency level of the query. Based on consistency values from [CQL](https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/dml/dmlConfigConsistency.html). - type: - members: - - id: all - value: 'all' - - id: each_quorum - value: 'each_quorum' - - id: quorum - value: 'quorum' - - id: local_quorum - value: 'local_quorum' - - id: one - value: 'one' - - id: two - value: 'two' - - id: three - value: 'three' - - id: local_one - value: 'local_one' - - id: any - value: 'any' - - id: serial - value: 'serial' - - id: local_serial - value: 'local_serial' - - id: table - type: string + - ref: db.cassandra.table tag: call-level-tech-specific-cassandra - requirement_level: recommended - brief: The name of the primary table that the operation is acting upon, including the keyspace name (if applicable). - note: > - This mirrors the db.sql.table attribute but references cassandra rather than sql. - It is not recommended to attempt any client-side parsing of - `db.statement` just to get this property, but it should be set if - it is provided by the library being instrumented. - If the operation is acting upon an anonymous table, or more than one table, this - value MUST NOT be set. - examples: 'mytable' - - id: idempotence - type: boolean + - ref: db.cassandra.idempotence tag: call-level-tech-specific-cassandra - brief: > - Whether or not the query is idempotent. - - id: speculative_execution_count - type: int + - ref: db.cassandra.speculative_execution_count tag: call-level-tech-specific-cassandra - brief: > - The number of times a query was speculatively executed. Not set or `0` if the query was not executed speculatively. - examples: [0, 2] - - id: coordinator.id - type: string + - ref: db.cassandra.coordinator.id tag: call-level-tech-specific-cassandra - brief: > - The ID of the coordinating node for a query. - examples: 'be13faa2-8574-4d71-926d-27f16cf8a7af' - - id: coordinator.dc - type: string + - ref: db.cassandra.coordinator.dc tag: call-level-tech-specific-cassandra - brief: > - The data center of the coordinating node for a query. - examples: 'us-west-2' - id: db.hbase - prefix: db.hbase type: span extends: db brief: > @@ -364,7 +97,6 @@ groups: note: For HBase the `db.name` should be set to the HBase namespace. - id: db.couchdb - prefix: db.couchdb type: span extends: db brief: > @@ -383,21 +115,15 @@ groups: [`GET /{db}/{docid}`](http://docs.couchdb.org/en/stable/api/document/common.html#get--db-docid). - id: db.redis - prefix: db.redis type: span extends: db brief: > Call-level attributes for Redis attributes: - - id: database_index - type: int + - ref: db.redis.database_index requirement_level: conditionally_required: If other than the default database (`0`). tag: call-level-tech-specific - brief: > - The index of the database being accessed as used in the [`SELECT` command](https://redis.io/commands/select), provided as an integer. - To be used instead of the generic `db.name` attribute. - examples: [0, 1, 15] - ref: db.statement tag: call-level-tech-specific brief: > @@ -408,22 +134,16 @@ groups: If, for example, the [`HMSET` command](https://redis.io/commands/hmset) is invoked, `"HMSET myhash field1 'Hello' field2 'World'"` would be a suitable value for `db.statement`. - id: db.mongodb - prefix: db.mongodb type: span extends: db brief: > Call-level attributes for MongoDB attributes: - - id: collection - type: string + - ref: db.mongodb.collection requirement_level: required tag: call-level-tech-specific - brief: > - The collection being accessed within the database stated in `db.name`. - examples: [ 'customers', 'products' ] - id: db.elasticsearch - prefix: db.elasticsearch type: span extends: db brief: > @@ -431,13 +151,16 @@ groups: attributes: - ref: http.request.method requirement_level: required + tag: call-level-tech-specific - ref: db.operation requirement_level: required brief: The endpoint identifier for the request. examples: [ 'search', 'ml.close_job', 'cat.aliases' ] + tag: call-level-tech-specific - ref: url.full requirement_level: required examples: [ 'https://localhost:9200/index/_search?q=user.id:kimchy' ] + tag: call-level-tech-specific - ref: db.statement requirement_level: recommended: > @@ -445,59 +168,34 @@ groups: sensitive information. brief: The request body for a [search-type query](https://www.elastic.co/guide/en/elasticsearch/reference/current/search.html), as a json string. examples: [ '"{\"query\":{\"term\":{\"user.id\":\"kimchy\"}}}"' ] + tag: call-level-tech-specific - ref: server.address + tag: call-level-tech-specific - ref: server.port - - id: cluster.name - type: string + tag: call-level-tech-specific + - ref: db.elasticsearch.cluster.name requirement_level: recommended: > When communicating with an Elastic Cloud deployment, this should be collected from the "X-Found-Handling-Cluster" HTTP response header. tag: call-level-tech-specific - brief: > - Represents the identifier of an Elasticsearch cluster. - examples: ["e9106fc68e3044f0b1475b04bf4ffd5f"] - - id: node.name - type: string + - ref: db.elasticsearch.node.name requirement_level: recommended: > When communicating with an Elastic Cloud deployment, this should be collected from the "X-Found-Handling-Instance" HTTP response header. tag: call-level-tech-specific - brief: > - Represents the human-readable identifier of the node/instance to which a request was routed. - examples: ["instance-0000000001"] - - id: path_parts - type: template[string] + - ref: db.elasticsearch.path_parts requirement_level: conditionally_required: when the url has dynamic values tag: call-level-tech-specific - brief: > - A dynamic value in the url path. - note: > - Many Elasticsearch url paths allow dynamic values. These SHOULD be recorded in span attributes in the format - `db.elasticsearch.path_parts.`, where `` is the url path part name. The implementation SHOULD - reference the [elasticsearch schema](https://raw.githubusercontent.com/elastic/elasticsearch-specification/main/output/schema/schema.json) - in order to map the path part values to their names. - examples: ['db.elasticsearch.path_parts.index=test-index', 'db.elasticsearch.path_parts.doc_id=123'] - id: db.sql - prefix: 'db.sql' type: span extends: 'db' brief: > Call-level attributes for SQL databases attributes: - - id: table + - ref: db.sql.table tag: call-level-tech-specific - type: string - requirement_level: recommended - brief: The name of the primary table that the operation is acting upon, including the database name (if applicable). - note: > - It is not recommended to attempt any client-side parsing of - `db.statement` just to get this property, but it should be set if - it is provided by the library being instrumented. - If the operation is acting upon an anonymous table, or more than one table, this - value MUST NOT be set. - examples: ['public.users', 'customers'] - id: db.cosmosdb type: span @@ -506,47 +204,12 @@ groups: brief: > Call-level attributes for Cosmos DB. attributes: - - id: client_id - type: string - brief: Unique Cosmos client instance id. - examples: '3ba4827d-4422-483f-b59f-85b74211c11d' - - id: operation_type - type: - allow_custom_values: true - members: - - id: invalid - value: 'Invalid' - - id: create - value: 'Create' - - id: patch - value: 'Patch' - - id: read - value: 'Read' - - id: read_feed - value: 'ReadFeed' - - id: delete - value: 'Delete' - - id: replace - value: 'Replace' - - id: execute - value: 'Execute' - - id: query - value: 'Query' - - id: head - value: 'Head' - - id: head_feed - value: 'HeadFeed' - - id: upsert - value: 'Upsert' - - id: batch - value: 'Batch' - - id: query_plan - value: 'QueryPlan' - - id: execute_javascript - value: 'ExecuteJavaScript' - brief: CosmosDB Operation Type. + - ref: db.cosmosdb.client_id + tag: call-level-tech-specific + - ref: db.cosmosdb.operation_type requirement_level: conditionally_required: when performing one of the operations in this list + tag: call-level-tech-specific - ref: user_agent.original brief: 'Full user-agent string is generated by Cosmos DB SDK' note: > @@ -561,46 +224,29 @@ groups: Format Reg-{D (Disabled discovery)}-S(application region)|L(List of preferred regions)|N(None, user did not configure it). Default value is "NS". examples: ['cosmos-netstandard-sdk/3.23.0\|3.23.1\|1\|X64\|Linux 5.4.0-1098-azure 104 18\|.NET Core 3.1.32\|S\|'] - - id: connection_mode - type: - allow_custom_values: false - members: - - id: gateway - value: 'gateway' - brief: Gateway (HTTP) connections mode - - id: direct - value: 'direct' - brief: Direct connection. - brief: Cosmos client connection mode. + tag: call-level-tech-specific + - ref: db.cosmosdb.connection_mode requirement_level: conditionally_required: if not `direct` (or pick gw as default) - - id: container - type: string - brief: Cosmos DB container name. + tag: call-level-tech-specific + - ref: db.cosmosdb.container requirement_level: conditionally_required: if available - examples: 'anystring' - - id: request_content_length - type: int - brief: Request payload size in bytes - - id: status_code - type: int - brief: Cosmos DB status code. - examples: [200, 201] + tag: call-level-tech-specific + - ref: db.cosmosdb.request_content_length + tag: call-level-tech-specific + - ref: db.cosmosdb.status_code requirement_level: conditionally_required: if response was received - - id: sub_status_code - type: int - brief: Cosmos DB sub status code. - examples: [1000, 1002] + tag: call-level-tech-specific + - ref: db.cosmosdb.sub_status_code requirement_level: conditionally_required: when response was received and contained sub-code. - - id: request_charge - type: double - brief: RU consumed for that operation - examples: [46.18, 1.0] + tag: call-level-tech-specific + - ref: db.cosmosdb.request_charge requirement_level: conditionally_required: when available + tag: call-level-tech-specific - id: db.tech type: span From 8962dbeedaaa0c99358231426f810f46eda9dcde Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Fri, 17 Nov 2023 00:57:45 -0800 Subject: [PATCH 121/124] Remove misleading pluralization wording related to count metrics (#488) Co-authored-by: Alexander Wert --- CHANGELOG.md | 3 +++ docs/general/metrics.md | 6 ++---- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 21cbd4c1bb..deffa3d7c7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,6 +27,9 @@ release. ### Fixes +- Remove misleading pluralization wording related to count metrics + ([#488](https://github.com/open-telemetry/semantic-conventions/pull/488)) + ## v1.23.0 (2023-11-03) This release marks the first where the core of HTTP semantic conventions have diff --git a/docs/general/metrics.md b/docs/general/metrics.md index 058a12e636..74699fbad0 100644 --- a/docs/general/metrics.md +++ b/docs/general/metrics.md @@ -124,12 +124,10 @@ should be pluralized, even if only a single data point is recorded. If the value being recorded represents the count of concepts signified by the namespace then the metric should be named `count` (within its namespace). -The pluralization rule does not apply in this case. -For example if we have a namespace `system.processes` which contains all metrics related +For example if we have a namespace `system.process` which contains all metrics related to the processes then to represent the count of the processes we can have a metric named -`system.processes.count`. The suffix `count` here indicates that it is the count of -`system.processes`. +`system.process.count`. #### Do not use `total` From b5b1e6a020e47ec7d3361a8293d7d291034a598e Mon Sep 17 00:00:00 2001 From: Armin Ruech <7052238+arminru@users.noreply.github.com> Date: Fri, 17 Nov 2023 16:45:33 +0100 Subject: [PATCH 122/124] [CI] Run checks.yml on all branches (#540) --- .github/workflows/checks.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml index 4ae744dcbe..a387a9db22 100644 --- a/.github/workflows/checks.yml +++ b/.github/workflows/checks.yml @@ -2,9 +2,7 @@ name: Checks on: push: - branches: [ main ] pull_request: - branches: [ main ] jobs: markdownlint: From dccfffc9bfc5480099bdcac4b99c700410112e3b Mon Sep 17 00:00:00 2001 From: Anna Levenberg Date: Sun, 19 Nov 2023 05:06:31 -0500 Subject: [PATCH 123/124] docs(messaging): add gcp_pubsub ordering key attribute (#528) Co-authored-by: Alexander Wert --- CHANGELOG.md | 4 +++- CONTRIBUTING.md | 8 ++++---- docs/attributes-registry/messaging.md | 1 + docs/messaging/gcp-pubsub.md | 9 +++++++++ model/registry/messaging.yaml | 5 +++++ model/trace/messaging.yaml | 10 ++++++++++ 6 files changed, 32 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index deffa3d7c7..64ba5be8b1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,6 +24,8 @@ release. ([#503](https://github.com/open-telemetry/semantic-conventions/pull/503)) - Add Semantic conventions for TLS/SSL encrypted communication. ([#21](https://github.com/open-telemetry/semantic-conventions/pull/21)) +- Add `messaging.gcp_pubsub.message.ordering_key` attribute. + ([#528](https://github.com/open-telemetry/semantic-conventions/pull/528)) ### Fixes @@ -123,7 +125,7 @@ stabilized. - Remove experimental Kafka metrics ([#338](https://github.com/open-telemetry/semantic-conventions/pull/338)) - Adds `session.id` and session.md to general docs and model -([#215](https://github.com/open-telemetry/semantic-conventions/pull/215)) + ([#215](https://github.com/open-telemetry/semantic-conventions/pull/215)) - Add `container.labels.` attributes. ([#125](https://github.com/open-telemetry/semantic-conventions/pull/125)) - Add `cluster.name` and `node.name` attributes to Elasticsearch semantic conventions. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3e4d9addaa..7efad36fa2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -161,14 +161,14 @@ make misspell-correction A PR (pull request) is considered to be **ready to merge** when: -* It has received at least two approvals from the [code +- It has received at least two approvals from the [code owners](./.github/CODEOWNERS) (if approvals are from only one company, they won't count). -* There is no `request changes` from the [code owners](./.github/CODEOWNERS). -* It has been at least two working days since the last modification (except for +- There is no `request changes` from the [code owners](./.github/CODEOWNERS). +- It has been at least two working days since the last modification (except for the trivial updates, such like typo, cosmetic, rebase, etc.). This gives people reasonable time to review. -* Trivial changes (typos, cosmetic changes, CI improvements, etc.) don't have to +- Trivial changes (typos, cosmetic changes, CI improvements, etc.) don't have to wait for two days. Any [maintainer](./README.md#contributing) can merge the PR once it is **ready diff --git a/docs/attributes-registry/messaging.md b/docs/attributes-registry/messaging.md index fe5523c85d..626d8cc59f 100644 --- a/docs/attributes-registry/messaging.md +++ b/docs/attributes-registry/messaging.md @@ -16,6 +16,7 @@ | `messaging.destination.temporary` | boolean | A boolean that is true if the message destination is temporary and might not exist anymore after messages are processed. | | | `messaging.destination_publish.anonymous` | boolean | A boolean that is true if the publish message destination is anonymous (could be unnamed or have auto-generated name). | | | `messaging.destination_publish.name` | string | The name of the original destination the message was published to [4] | `MyQueue`; `MyTopic` | +| `messaging.gcp_pubsub.message.ordering_key` | string | The ordering key for a given message. If the attribute is not present, the message does not have an ordering key. | `ordering_key` | | `messaging.kafka.consumer.group` | string | Name of the Kafka Consumer Group that is handling the message. Only applies to consumers, not producers. | `my-group` | | `messaging.kafka.destination.partition` | int | Partition the message is sent to. | `2` | | `messaging.kafka.message.key` | string | Message keys in Kafka are used for grouping alike messages to ensure they're processed on the same partition. They differ from `messaging.message.id` in that they're not unique. If the key is `null`, the attribute MUST NOT be set. [5] | `myKey` | diff --git a/docs/messaging/gcp-pubsub.md b/docs/messaging/gcp-pubsub.md index 848f144fc3..08027ce032 100644 --- a/docs/messaging/gcp-pubsub.md +++ b/docs/messaging/gcp-pubsub.md @@ -10,4 +10,13 @@ The Semantic Conventions for [Google Cloud Pub/Sub](https://cloud.google.com/pub `messaging.system` MUST be set to `"gcp_pubsub"`. +## Span attributes + +For Google Cloud Pub/Sub, the following additional attributes are defined: + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| [`messaging.gcp_pubsub.message.ordering_key`](../attributes-registry/messaging.md) | string | The ordering key for a given message. If the attribute is not present, the message does not have an ordering key. | `ordering_key` | Conditionally Required: If the message type has an ordering key set. | + + [DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/document-status.md diff --git a/model/registry/messaging.yaml b/model/registry/messaging.yaml index 64dc6fe5a0..c7ba8fd457 100644 --- a/model/registry/messaging.yaml +++ b/model/registry/messaging.yaml @@ -202,6 +202,11 @@ groups: brief: > Namespace of RocketMQ resources, resources in different namespaces are individual. examples: 'myNamespace' + - id: gcp_pubsub.message.ordering_key + type: string + brief: > + The ordering key for a given message. If the attribute is not present, the message does not have an ordering key. + examples: 'ordering_key' - id: system brief: > An identifier for the messaging system being used. See below for a list of well-known identifiers. diff --git a/model/trace/messaging.yaml b/model/trace/messaging.yaml index c14b92dfa7..9163de0b4b 100644 --- a/model/trace/messaging.yaml +++ b/model/trace/messaging.yaml @@ -170,3 +170,13 @@ groups: tag: tech-specific-rocketmq - ref: messaging.rocketmq.consumption_model tag: tech-specific-rocketmq + - id: messaging.gcp_pubsub + type: attribute_group + extends: messaging + brief: > + Attributes for Google Cloud Pub/Sub + attributes: + - ref: messaging.gcp_pubsub.message.ordering_key + tag: tech-specific-gcp-pubsub + requirement_level: + conditionally_required: If the message type has an ordering key set. From 126059d99dc29fd9a59893091f56500d4b9674b7 Mon Sep 17 00:00:00 2001 From: Trask Stalnaker Date: Mon, 20 Nov 2023 07:45:22 -0800 Subject: [PATCH 124/124] Rename metrics `jvm.memory.usage` to `jvm.memory.used` and `jvm.memory.usage_after_last_gc` to `jvm.memory.used_after_last_gc` (#536) Co-authored-by: Alexander Wert Co-authored-by: Joao Grassi --- CHANGELOG.md | 3 +++ docs/runtime/jvm-metrics.md | 20 ++++++++++---------- model/metrics/jvm-metrics.yaml | 8 ++++---- schema-next.yaml | 4 ++++ 4 files changed, 21 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 64ba5be8b1..0e6f2f2257 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,9 @@ release. ([#495](https://github.com/open-telemetry/semantic-conventions/issues/495)) - Changed `messaging.system` attribute type to an open enum ([#517](https://github.com/open-telemetry/semantic-conventions/pull/517)) +- Rename metrics `jvm.memory.usage` to `jvm.memory.used` and `jvm.memory.usage_after_last_gc` + to `jvm.memory.used_after_last_gc` + ([#536](https://github.com/open-telemetry/semantic-conventions/pull/536)) ### Features diff --git a/docs/runtime/jvm-metrics.md b/docs/runtime/jvm-metrics.md index 4e7825c857..41e2d38ac4 100644 --- a/docs/runtime/jvm-metrics.md +++ b/docs/runtime/jvm-metrics.md @@ -13,10 +13,10 @@ This document describes semantic conventions for JVM metrics in OpenTelemetry. - [JVM Memory](#jvm-memory) - * [Metric: `jvm.memory.usage`](#metric-jvmmemoryusage) + * [Metric: `jvm.memory.used`](#metric-jvmmemoryused) * [Metric: `jvm.memory.committed`](#metric-jvmmemorycommitted) * [Metric: `jvm.memory.limit`](#metric-jvmmemorylimit) - * [Metric: `jvm.memory.usage_after_last_gc`](#metric-jvmmemoryusage_after_last_gc) + * [Metric: `jvm.memory.used_after_last_gc`](#metric-jvmmemoryused_after_last_gc) - [JVM Garbage Collection](#jvm-garbage-collection) * [Metric: `jvm.gc.duration`](#metric-jvmgcduration) - [JVM Threads](#jvm-threads) @@ -43,18 +43,18 @@ This document describes semantic conventions for JVM metrics in OpenTelemetry. **Description:** Java Virtual Machine (JVM) metrics captured under the namespace `jvm.memory.*` -### Metric: `jvm.memory.usage` +### Metric: `jvm.memory.used` This metric is [recommended][MetricRecommended]. This metric is obtained from [`MemoryPoolMXBean#getUsage()`](https://docs.oracle.com/javase/8/docs/api/java/lang/management/MemoryPoolMXBean.html#getUsage--). - + | Name | Instrument Type | Unit (UCUM) | Description | | -------- | --------------- | ----------- | -------------- | -| `jvm.memory.usage` | UpDownCounter | `By` | Measure of memory used. | +| `jvm.memory.used` | UpDownCounter | `By` | Measure of memory used. | - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| | `jvm.memory.pool.name` | string | Name of the memory pool. [1] | `G1 Old Gen`; `G1 Eden space`; `G1 Survivor Space` | Recommended | @@ -124,18 +124,18 @@ This metric is obtained from [`MemoryPoolMXBean#getUsage()`](https://docs.oracle | `non_heap` | Non-heap memory | -### Metric: `jvm.memory.usage_after_last_gc` +### Metric: `jvm.memory.used_after_last_gc` This metric is [recommended][MetricRecommended]. This metric is obtained from [`MemoryPoolMXBean#getCollectionUsage()`](https://docs.oracle.com/javase/8/docs/api/java/lang/management/MemoryPoolMXBean.html#getCollectionUsage--). - + | Name | Instrument Type | Unit (UCUM) | Description | | -------- | --------------- | ----------- | -------------- | -| `jvm.memory.usage_after_last_gc` | UpDownCounter | `By` | Measure of memory used, as measured after the most recent garbage collection event on this pool. | +| `jvm.memory.used_after_last_gc` | UpDownCounter | `By` | Measure of memory used, as measured after the most recent garbage collection event on this pool. | - + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| | `jvm.memory.pool.name` | string | Name of the memory pool. [1] | `G1 Old Gen`; `G1 Eden space`; `G1 Survivor Space` | Recommended | diff --git a/model/metrics/jvm-metrics.yaml b/model/metrics/jvm-metrics.yaml index c837145a8c..ba1f560453 100644 --- a/model/metrics/jvm-metrics.yaml +++ b/model/metrics/jvm-metrics.yaml @@ -26,9 +26,9 @@ groups: Pool names are generally obtained via [MemoryPoolMXBean#getName()](https://docs.oracle.com/en/java/javase/11/docs/api/java.management/java/lang/management/MemoryPoolMXBean.html#getName()). - - id: metric.jvm.memory.usage + - id: metric.jvm.memory.used type: metric - metric_name: jvm.memory.usage + metric_name: jvm.memory.used extends: attributes.jvm.memory brief: "Measure of memory used." instrument: updowncounter @@ -50,9 +50,9 @@ groups: instrument: updowncounter unit: "By" - - id: metric.jvm.memory.usage_after_last_gc + - id: metric.jvm.memory.used_after_last_gc type: metric - metric_name: jvm.memory.usage_after_last_gc + metric_name: jvm.memory.used_after_last_gc extends: attributes.jvm.memory brief: "Measure of memory used, as measured after the most recent garbage collection event on this pool." instrument: updowncounter diff --git a/schema-next.yaml b/schema-next.yaml index 4f618de046..ae84e381af 100644 --- a/schema-next.yaml +++ b/schema-next.yaml @@ -2,6 +2,10 @@ file_format: 1.1.0 schema_url: https://opentelemetry.io/schemas/next versions: next: + # https://github.com/open-telemetry/semantic-conventions/pull/536 + - rename_metrics: + jvm.memory.usage: jvm.memory.used + jvm.memory.usage_after_last_gc: jvm.memory.used_after_last_gc 1.23.0: metrics: changes: