Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add remaining Kafka and Kafka consumer metrics. #2536

Merged
merged 14 commits into from
May 26, 2022
Merged

Add remaining Kafka and Kafka consumer metrics. #2536

Changes from 2 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
4e6888e
Add remaining Kafka and Kafka consumer metrics.
carlosalberto May 12, 2022
93213be
Remove leaked line.
carlosalberto May 12, 2022
ce07bcd
Update specification/metrics/semantic_conventions/instrumentation/kaf…
carlosalberto May 17, 2022
dc3ac01
Update specification/metrics/semantic_conventions/instrumentation/kaf…
carlosalberto May 17, 2022
be97569
Update specification/metrics/semantic_conventions/instrumentation/kaf…
carlosalberto May 17, 2022
57f9b2a
Update specification/metrics/semantic_conventions/instrumentation/kaf…
carlosalberto May 17, 2022
f55b204
Merge branch 'main' into kafka-metrics-additional
reyang May 19, 2022
3f95a2b
Apply feedback.
carlosalberto May 19, 2022
469f8e4
Oops, missed header section.
carlosalberto May 19, 2022
41a65db
Merge branch 'main' into kafka-metrics-additional
carlosalberto May 19, 2022
5deb8e1
Adjust name.
carlosalberto May 20, 2022
de80bff
Update CHANGELOG.
carlosalberto May 25, 2022
38068a2
Merge branch 'main' into kafka-metrics-additional
carlosalberto May 25, 2022
ecb27a1
Merge branch 'main' into kafka-metrics-additional
reyang May 26, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions specification/metrics/semantic_conventions/instrumentation/kafka.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ This document defines how to apply semantic conventions when instrumenting Kafka

- [Kafka Metrics](#kafka-metrics)
- [Kafka Producer Metrics](#kafka-producer-metrics)
- [Kafka Consumer Metrics](#kafka-consumer-metrics)

<!-- tocstop -->

Expand All @@ -30,6 +31,17 @@ This document defines how to apply semantic conventions when instrumenting Kafka
| messaging.kafka.controllers.active | UpDownCounter | Int64 | controllers | `{controllers}` | The number of active controllers in the broker. | | |
| messaging.kafka.leader.elections | Counter | Int64 | elections | `{elections}` | Leader election rate (increasing values indicates broker failures). | | |
| messaging.kafka.leader.unclean-elections | Counter | Int64 | elections | `{elections}` | Unclean leader election rate (increasing values indicates broker failures). | | |
| messaging.kafka.brokers | Gauge | Int64 | brokers | `{brokers}` | Number of brokers in the cluster. | | |
| messaging.kafka.topic.partitions | Gauge | Int64 | partitions | `{partitions}` | Number of partitions in topic. | `topic` | The ID (integer) of a topic |
carlosalberto marked this conversation as resolved.
Show resolved Hide resolved
| messaging.kafka.partition.current_offset | Gauge | Int64 | partition offset | `{partition offset}` | Current offset of partition of topic. | `topic` | The ID (integer) of a topic |
| | | | | | | `partition` | The number (integer) of the partition |

| messaging.kafka.partition.oldest_offset | Gauge | Int64 | partition offset | `{partition offset}` | Oldest offset of partition of topic | `topic` | The ID (integer) of a topic |
| | | | | | | `partition` | The number (integer) of the partition |
| messaging.kafka.partition.replicas | Gauge | Int64 | replicas | `{replicas}` | Number of replicas for partition of topic | `topic` | The ID (integer) of a topic |
carlosalberto marked this conversation as resolved.
Show resolved Hide resolved
| | | | | | | `partition` | The number (integer) of the partition |
| messaging.kafka.partition.replicas_in_sync | Gauge | Int64 | replicas | `{replicas}` | Number of synchronized replicas of partition | `topic` | The ID (integer) of a topic |
carlosalberto marked this conversation as resolved.
Show resolved Hide resolved
| | | | | | | `partition` | The number (integer) of the partition|

## Kafka Producer Metrics

Expand All @@ -49,3 +61,21 @@ This document defines how to apply semantic conventions when instrumenting Kafka
| | | | | | | `topic` | topic name |
| messaging.kafka.producer.record-sent.rate | Gauge | Double | records sent rate | `{records_sent}/s` | The average number of records sent per second for a specific topic. | `client-id` | `client-id` value |
| | | | | | | `topic` | topic name |

## Kafka Consumer Metrics

**Description:** Kafka Consumer level metrics.

| Name | Instrument | Value type | Unit | Unit ([UCUM](../README.md#instrument-units)) | Description | Attribute Key | Attribute Values |
| messaging.kafka.consumer_group.members | Gauge | Int64 | members | `{members}` | Count of members in the consumer group | `group` | The ID (string) of a consumer group |
carlosalberto marked this conversation as resolved.
Show resolved Hide resolved
| messaging.kafka.consumer_group.offset | Gauge | Int64 | offset | `{offset}` | Current offset of the consumer group at partition of topic | `group` | The ID (string) of a consumer group |
carlosalberto marked this conversation as resolved.
Show resolved Hide resolved
| | | | | | | `topic` | The ID (integer) of a topic |
| | | | | | | `partition` | The number (integer) of the partition |
| messaging.kafka.consumer_group.offset_sum | Gauge | Int64 | offset sum | `{offset sum}` | Sum of consumer group offset across partitions of topic | `group` | The ID (string) of a consumer group |
| | | | | | | `topic` | The ID (integer) of a topic |

| messaging.kafka.consumer_group.lag | Gauge | Int64 | lag | `{lag}` | Current approximate lag of consumer group at partition of topic | `group` | The ID (string) of a consumer group |
| | | | | | | `topic` | The ID (integer) of a topic |
| | | | | | | `partition` | The number (integer) of the partition |
| messaging.kafka.consumer_group.lag_sum | Gauge | Int64 | lag sum | `{lag sum}` | Current approximate sum of consumer group lag across all partitions of topic | `group` | The ID (string) of a consumer group |
| | | | | | | `topic` | The ID (integer) of a topic |