Skip to content

Commit

Permalink
1st round review from @lmolkova
Browse files Browse the repository at this point in the history
  • Loading branch information
@gyliu513
gyliu513 committed Jan 16, 2025
1 parent f4c961a commit a246235
Show file tree
Hide file tree
Showing 6 changed files with 88 additions and 101 deletions.
2 changes: 1 addition & 1 deletion .chloggen/1732.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
change_type: enhancement

# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db)
component: genai
component: gen-ai

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Added AI Agent Semantic Convention
Expand Down
23 changes: 8 additions & 15 deletions docs/attributes-registry/gen-ai.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,6 @@ This document defines the attributes used to describe telemetry in the context o
| <a id="gen-ai-agent-description" href="#gen-ai-agent-description">`gen_ai.agent.description`</a> | string | Free-form description of the GenAI agent provided by the application. | `Helps with math problems`; `Generates fiction stories` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-agent-id" href="#gen-ai-agent-id">`gen_ai.agent.id`</a> | string | The unique identifier of the GenAI agent. | `asst_5j66UpCpwteGg4YSxUnt7lPY` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-agent-name" href="#gen-ai-agent-name">`gen_ai.agent.name`</a> | string | Human-readable name of the GenAI agent provided by the application. | `Math Tutor`; `Fiction Writer` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-agent-status" href="#gen-ai-agent-status">`gen_ai.agent.status`</a> | string | The status of the agent. | `completed`; `failed` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-agent-tool-call-id" href="#gen-ai-agent-tool-call-id">`gen_ai.agent.tool.call.id`</a> | string | The tool call identifier. | `call_mszuSIzqtI65i1wAUOE8w5H4` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-agent-tool-datastore-name" href="#gen-ai-agent-tool-datastore-name">`gen_ai.agent.tool.datastore.name`</a> | string | Name of the datastore of the tool utilized by the agent. | `pdf dstore`; `website dstore` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-agent-tool-extension-name" href="#gen-ai-agent-tool-extension-name">`gen_ai.agent.tool.extension.name`</a> | string | Name of the extension of the tool utilized by the agent. | `Code Interpreter` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-agent-tool-function-name" href="#gen-ai-agent-tool-function-name">`gen_ai.agent.tool.function.name`</a> | string | Name of the function of the tool utilized by the agent. | `getFlights`; `getWeather` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-agent-tool-name" href="#gen-ai-agent-tool-name">`gen_ai.agent.tool.name`</a> | string | Name of the tool utilized by the agent. | `Flights` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-agent-tool-type" href="#gen-ai-agent-tool-type">`gen_ai.agent.tool.type`</a> | string | Type of the tool utilized by the agent. | `function`; `extension`; `datastore` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-agent-tools" href="#gen-ai-agent-tools">`gen_ai.agent.tools`</a> | string[] | List of tool names available to the agent. | `["Flights", "Search", "Code"]`; `["calc"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-operation-name" href="#gen-ai-operation-name">`gen_ai.operation.name`</a> | string | The name of the operation being performed. [1] | `chat`; `text_completion`; `embeddings` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-encoding-formats" href="#gen-ai-request-encoding-formats">`gen_ai.request.encoding_formats`</a> | string[] | The encoding formats requested in an embeddings operation, if specified. [2] | `["base64"]`; `["float", "binary"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-frequency-penalty" href="#gen-ai-request-frequency-penalty">`gen_ai.request.frequency_penalty`</a> | double | The frequency penalty setting for the GenAI request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
Expand All @@ -44,6 +36,12 @@ This document defines the attributes used to describe telemetry in the context o
| <a id="gen-ai-response-model" href="#gen-ai-response-model">`gen_ai.response.model`</a> | string | The name of the model that generated the response. | `gpt-4-0613` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-system" href="#gen-ai-system">`gen_ai.system`</a> | string | The Generative AI product as identified by the client or server instrumentation. [3] | `openai` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-token-type" href="#gen-ai-token-type">`gen_ai.token.type`</a> | string | The type of token being counted. | `input`; `output` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-tool-call-id" href="#gen-ai-tool-call-id">`gen_ai.tool.call.id`</a> | string | The tool call identifier. | `call_mszuSIzqtI65i1wAUOE8w5H4` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-tool-datastore-name" href="#gen-ai-tool-datastore-name">`gen_ai.tool.datastore.name`</a> | string | Name of the datastore of the tool utilized by the agent. | `pdf dstore`; `website dstore` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-tool-extension-name" href="#gen-ai-tool-extension-name">`gen_ai.tool.extension.name`</a> | string | Name of the extension of the tool utilized by the agent. [4] | `Code Interpreter` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-tool-function-name" href="#gen-ai-tool-function-name">`gen_ai.tool.function.name`</a> | string | Name of the function of the tool utilized by the agent. | `getFlights`; `getWeather` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-tool-name" href="#gen-ai-tool-name">`gen_ai.tool.name`</a> | string | Name of the tool utilized by the agent. | `Flights` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-tool-type" href="#gen-ai-tool-type">`gen_ai.tool.type`</a> | string | Type of the tool utilized by the agent. [5] | `function`; `extension`; `datastore` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-usage-input-tokens" href="#gen-ai-usage-input-tokens">`gen_ai.usage.input_tokens`</a> | int | The number of tokens used in the GenAI input (prompt). | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-usage-output-tokens" href="#gen-ai-usage-output-tokens">`gen_ai.usage.output_tokens`</a> | int | The number of tokens used in the GenAI response (completion). | `180` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |

Expand All @@ -63,14 +61,9 @@ attribute may help identify the actual system in use for `openai`.
For custom model, a custom friendly name SHOULD be used.
If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`.

---

`gen_ai.agent.status` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
**[4] `gen_ai.tool.extension.name`:** You can refer to [langchain tools](https://python.langchain.com/docs/integrations/tools/) for more examples.

| Value | Description | Stability |
|---|---|---|
| `completed` | The agent has been completed | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `failed` | The agent was unable to complete the task due to an error. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
**[5] `gen_ai.tool.type`:** Extension: A tool executed on the agent-side to directly call external APIs, bridging the gap between the agent and real-world systems. Function: A tool executed on the client-side, where the agent generates parameters for a predefined function, and the client executes the logic. Datastore: A tool used by the agent to access and query structured or unstructured external data for retrieval-augmented tasks or knowledge updates.

---

Expand Down
42 changes: 5 additions & 37 deletions docs/gen-ai/gen-ai-agent-spans.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,8 @@ linkTitle: Generative AI traces

<!-- toc -->

- [Spans](#spans)
- [Create Agent Span](#create-agent-span)
- [Execute Tool Span](#execute-tool-span)
- [Create Agent Span](#create-agent-span)
- [Agent Execute Tool Span](#agent-execute-tool-span)

<!-- tocstop -->

Expand All @@ -24,9 +23,7 @@ It MAY be applicable to agent operations that are performed by the GenAI framewo

The semantic conventions for GenAI agents extend and override the semantic conventions for [Gen AI Spans](gen-ai-spans.md).

## Spans

### Create Agent Span
## Create Agent Span

Describes GenAI agent creation and is usually applicable when working with remote agent
services.
Expand All @@ -45,14 +42,12 @@ Semantic conventions for individual GenAI systems and frameworks MAY specify dif

| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability |
|---|---|---|---|---|---|
| [`gen_ai.agent.status`](/docs/attributes-registry/gen-ai.md) | string | The status of the agent. | `completed`; `failed` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.operation.name`](/docs/attributes-registry/gen-ai.md) | string | The name of the operation being performed. [1] | `chat`; `text_completion`; `embeddings` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.system`](/docs/attributes-registry/gen-ai.md) | string | The Generative AI product as identified by the client or server instrumentation. [2] | `openai` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`error.type`](/docs/attributes-registry/error.md) | string | Describes a class of error the operation ended with. [3] | `timeout`; `java.net.UnknownHostException`; `server_certificate_invalid`; `500` | `Conditionally Required` if the operation ended in an error | ![Stable](https://img.shields.io/badge/-stable-lightgreen) |
| [`gen_ai.agent.description`](/docs/attributes-registry/gen-ai.md) | string | Free-form description of the GenAI agent provided by the application. | `Helps with math problems`; `Generates fiction stories` | `Conditionally Required` If provided by the application. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.agent.id`](/docs/attributes-registry/gen-ai.md) | string | The unique identifier of the GenAI agent. | `asst_5j66UpCpwteGg4YSxUnt7lPY` | `Conditionally Required` if applicable. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.agent.name`](/docs/attributes-registry/gen-ai.md) | string | Human-readable name of the GenAI agent provided by the application. | `Math Tutor`; `Fiction Writer` | `Conditionally Required` If provided by the application. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.agent.tools`](/docs/attributes-registry/gen-ai.md) | string[] | List of tool names available to the agent. | `["Flights", "Search", "Code"]`; `["calc"]` | `Conditionally Required` If provided by the application. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.request.model`](/docs/attributes-registry/gen-ai.md) | string | The name of the GenAI model a request is being made to. [4] | `gpt-4` | `Conditionally Required` If provided by the application. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.request.seed`](/docs/attributes-registry/gen-ai.md) | int | Requests with same seed value more likely to return same result. | `100` | `Conditionally Required` if appliable and if the request includes a seed | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.request.temperature`](/docs/attributes-registry/gen-ai.md) | double | The temperature setting for the GenAI request. | `0.0` | `Conditionally Required` If provided by the application. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
Expand Down Expand Up @@ -108,15 +103,6 @@ Instrumentations SHOULD document the list of errors they report.

---

`gen_ai.agent.status` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

| Value | Description | Stability |
|---|---|---|
| `completed` | The agent has been completed | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `failed` | The agent was unable to complete the task due to an error. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |

---

`gen_ai.operation.name` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

| Value | Description | Stability |
Expand Down Expand Up @@ -153,26 +139,8 @@ Instrumentations SHOULD document the list of errors they report.
<!-- END AUTOGENERATED TEXT -->
<!-- endsemconv -->

### Execute Tool Span

<!-- semconv span.gen_ai.client.execute_tool -->
<!-- NOTE: THIS TEXT IS AUTOGENERATED. DO NOT EDIT BY HAND. -->
<!-- see templates/registry/markdown/snippet.md.j2 -->
<!-- prettier-ignore-start -->
<!-- markdownlint-capture -->
<!-- markdownlint-disable -->

| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability |
|---|---|---|---|---|---|
| [`gen_ai.agent.tool.datastore.name`](/docs/attributes-registry/gen-ai.md) | string | Name of the datastore of the tool utilized by the agent. | `pdf dstore`; `website dstore` | `Conditionally Required` If provided by the tool. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.agent.tool.extension.name`](/docs/attributes-registry/gen-ai.md) | string | Name of the extension of the tool utilized by the agent. | `Code Interpreter` | `Conditionally Required` If provided by the tool. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.agent.tool.function.name`](/docs/attributes-registry/gen-ai.md) | string | Name of the function of the tool utilized by the agent. | `getFlights`; `getWeather` | `Conditionally Required` If provided by the tool. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.agent.tool.call.id`](/docs/attributes-registry/gen-ai.md) | string | The tool call identifier. | `call_mszuSIzqtI65i1wAUOE8w5H4` | `Recommended` if available | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.agent.tool.name`](/docs/attributes-registry/gen-ai.md) | string | Name of the tool utilized by the agent. | `Flights` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
## Agent Execute Tool Span

<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->
<!-- END AUTOGENERATED TEXT -->
<!-- endsemconv -->
If you are using some tools in your agent, refer to [Execute Tool Span](./gen-ai-spans.md#execute-tool-span).

[DocumentStatus]: https://opentelemetry.io/docs/specs/otel/document-status
38 changes: 38 additions & 0 deletions docs/gen-ai/gen-ai-spans.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ linkTitle: Generative AI traces
- [Name](#name)
- [GenAI attributes](#genai-attributes)
- [Capturing inputs and outputs](#capturing-inputs-and-outputs)
- [Execute Tool Span](#execute-tool-span)

<!-- tocstop -->

Expand Down Expand Up @@ -139,6 +140,43 @@ Instrumentations SHOULD document the list of errors they report.
<!-- END AUTOGENERATED TEXT -->
<!-- endsemconv -->

## Execute Tool Span

<!-- semconv span.gen_ai.client.execute_tool -->
<!-- NOTE: THIS TEXT IS AUTOGENERATED. DO NOT EDIT BY HAND. -->
<!-- see templates/registry/markdown/snippet.md.j2 -->
<!-- prettier-ignore-start -->
<!-- markdownlint-capture -->
<!-- markdownlint-disable -->

| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability |
|---|---|---|---|---|---|
| [`error.type`](/docs/attributes-registry/error.md) | string | Describes a class of error the operation ended with. [1] | `timeout`; `java.net.UnknownHostException`; `server_certificate_invalid`; `500` | `Conditionally Required` if the operation ended in an error | ![Stable](https://img.shields.io/badge/-stable-lightgreen) |
| [`gen_ai.tool.datastore.name`](/docs/attributes-registry/gen-ai.md) | string | Name of the datastore of the tool utilized by the agent. | `pdf dstore`; `website dstore` | `Conditionally Required` If available. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.tool.extension.name`](/docs/attributes-registry/gen-ai.md) | string | Name of the extension of the tool utilized by the agent. [2] | `Code Interpreter` | `Conditionally Required` If available. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.tool.function.name`](/docs/attributes-registry/gen-ai.md) | string | Name of the function of the tool utilized by the agent. | `getFlights`; `getWeather` | `Conditionally Required` If available. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.tool.call.id`](/docs/attributes-registry/gen-ai.md) | string | The tool call identifier. | `call_mszuSIzqtI65i1wAUOE8w5H4` | `Recommended` if available | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.tool.name`](/docs/attributes-registry/gen-ai.md) | string | Name of the tool utilized by the agent. | `Flights` | `Recommended` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |

**[1] `error.type`:** The `error.type` SHOULD match the error code returned by the Generative AI tool provider or the
client library, the canonical name of exception that occurred, or another low-cardinality error
identifier. Instrumentations SHOULD document the list of errors they report.

**[2] `gen_ai.tool.extension.name`:** You can refer to [langchain tools](https://python.langchain.com/docs/integrations/tools/) for more examples.

---

`error.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

| Value | Description | Stability |
|---|---|---|
| `_OTHER` | A fallback error value to be used when the instrumentation doesn't define a custom value. | ![Stable](https://img.shields.io/badge/-stable-lightgreen) |

<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->
<!-- END AUTOGENERATED TEXT -->
<!-- endsemconv -->

## Capturing inputs and outputs

User inputs and model responses may be recorded as events parented to GenAI operation span. See [Semantic Conventions for GenAI events](./gen-ai-events.md) for the details.
Expand Down
Loading

0 comments on commit a246235

Please sign in to comment.