[receiver/kafka] Consume metrics published by Telegraf

[ledor473]

Component(s)

receiver/kafka

Is your feature request related to a problem? Please describe.

Currently, my metrics are collected using Telegraf and then sent to Kafka using Kafka output plugin. It's then consumed by another set of Telegraf that will write it to the storage layer.
I'd like to use a few features of the OpenTelemetry Collector but need to move to it in a phased manner. The plan would be to have the OpenTelemetry Collector consume the same messages that are published by Telegraf (using a different consumer group) and from there use the wanted features.

Describe the solution you'd like

Add a new Telegraf to the unmarshalers supported by receiver/kafka.
I'm thinking it could support only the influx format at first. Since Telegraf supports various formats, it might be best to take that into consideration while adding the features.

Describe alternatives you've considered

I looked into having Telegraf publish directly in OTLP format but it doesn't seem to be possible based on the output formats documentation.
I also looked into the OpenTelemetry output plugin, but it would skip Kafka which isn't ideal for my use case.

Additional context

Ultimately, if there's little interest in this feature request, I think I can still make it work by building an OpenTelemetry Collector and manually invoking the WithMetricsUnmarshalers method. But I figured that Telegraf is common enough that it might be interesting to host it here.

[github-actions]

Pinging code owners:

• receiver/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/kafka: @pavolloffay @MovieStoreGuy
• needs: Github issue template generation code needs this to generate the corresponding labels.

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[atoulme]

@ledor473 would you like to work on this?

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[atoulme]

@ledor473 are you offering to make this contribution? We are moving encodings to extensions, so they can be reused across components. See https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/encoding

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.