Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[statsdreceiver] support simple tags #29012

Merged
merged 6 commits into from
Dec 6, 2023

Conversation

diurnalist
Copy link
Contributor

@diurnalist diurnalist commented Nov 7, 2023

Description:

dogstatsd supports two types of tags on metrics: simple and dimensional tags1. the former is just a key, the latter is a key and a value.

with the assumption that many users of the statsdreceiver are enabling ingest of dogstatsd metrics, this makes the statsd parser more optimistic, so it can handle tags w/o a value. this functionality is gated behind a new enable_simple_tags flag.

when this happens, we set an attribute that has a zero value. so far as i know, this is allowed in the opentelemetry spec. the decision of how to handle attributes w/ zero values is best left to configuration w/in the pipeline itself, as different users may have different opinions or approaches that work best with their systems.

Testing:

added coverage to unit tests to enable parsing simple tags.

Footnotes

  1. https://www.datadoghq.com/blog/the-power-of-tagged-metrics/#whats-a-metric-tag

Copy link

linux-foundation-easycla bot commented Nov 7, 2023

CLA Signed

The committers listed above are authorized under a signed CLA.

@github-actions github-actions bot added the receiver/statsd statsd related issues label Nov 7, 2023
@diurnalist
Copy link
Contributor Author

re: attributes, the otel spec says:

Attribute values expressing a numerical value of zero, an empty string, or an empty array are considered meaningful and MUST be stored and passed on to processors / exporters.

explicit null is however not allowed. Go is a bit awkward here b/c the nil value is distinct from what null means in other languages.

@diurnalist diurnalist marked this pull request as ready for review November 7, 2023 20:56
@diurnalist diurnalist requested a review from a team November 7, 2023 20:56
@atoulme
Copy link
Contributor

atoulme commented Nov 8, 2023

At the root of your checkout, run make chlog-new to generate a changelog entry for this change. You can use this PR as the issue number.

@diurnalist diurnalist force-pushed the f/statsd-simple-tags branch 2 times, most recently from ed67ff4 to 33ce62a Compare November 8, 2023 22:57
@diurnalist
Copy link
Contributor Author

i have now put this behind a config flag as well just to be safe. the changelog includes a note about this. i'm open to it being default; we use dogstatsd so it makes sense to be on, but i'm not sure what other popular implementations of statsd are in use by others.

@diurnalist diurnalist force-pushed the f/statsd-simple-tags branch 2 times, most recently from 6468e97 to b27d979 Compare November 11, 2023 22:10
dogstatsd supports two types of tags on metrics: simple and
dimensional tags[^1]. the former is just a key, the latter is
a key and a value.

with the assumption that many users of the statsdreceiver are
enabling ingest of dogstatsd metrics, this makes the statsd
parser more optimistic, so it can handle tags w/o a value.

when this happens, we set an attribute that has a zero value.
so far as i know, this is allowed in the opentelemetry spec.
the decision of how to handle attributes w/ zero values is
best left to configuration w/in the pipeline itself, as different
users may have different opinions or approaches that work best
with their systems.

[^1]: https://www.datadoghq.com/blog/the-power-of-tagged-metrics/#whats-a-metric-tag
@atoulme atoulme added the ready to merge Code review completed; ready to merge by maintainers label Nov 21, 2023
Copy link
Contributor

github-actions bot commented Dec 5, 2023

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Dec 5, 2023
@jmacd
Copy link
Contributor

jmacd commented Dec 6, 2023

Are you interested in making the new behavior on by default? I understand being conservative, but the new functionality is not likely to break any existing functionality, right? Most users who come to this component are likely coming from a dogstatsd or similar agent.

@codeboten codeboten merged commit 4302c5b into open-telemetry:main Dec 6, 2023
76 checks passed
@github-actions github-actions bot added this to the next release milestone Dec 6, 2023
@diurnalist diurnalist deleted the f/statsd-simple-tags branch December 6, 2023 21:24
@diurnalist
Copy link
Contributor Author

thanks!

Are you interested in making the new behavior on by default?

I'm for doing this in another PR now that this is merged. I agree that leaving it off by default is perhaps too conservative, but I opted for the minimal-disruption path in order to add confidence to the patch.

jayasai470 pushed a commit to jayasai470/opentelemetry-collector-contrib that referenced this pull request Dec 8, 2023
dogstatsd supports two types of tags on metrics: simple and dimensional
tags[^1]. the former is just a key, the latter is a key and a value.

with the assumption that many users of the statsdreceiver are enabling
ingest of dogstatsd metrics, this makes the statsd parser more
optimistic, so it can handle tags w/o a value. this functionality is
gated behind a new `enable_simple_tags` flag.

when this happens, we set an attribute that has a zero value. so far as
i know, this is allowed in the opentelemetry spec. the decision of how
to handle attributes w/ zero values is best left to configuration w/in
the pipeline itself, as different users may have different opinions or
approaches that work best with their systems.

[^1]:
https://www.datadoghq.com/blog/the-power-of-tagged-metrics/#whats-a-metric-tag

**Testing:**

added coverage to unit tests to enable parsing simple tags.

---------

Co-authored-by: Alex Boten <aboten@lightstep.com>
@sirianni
Copy link
Contributor

@diurnalist thanks for this contribution!

I'm seeing errors like these from our statsdreceiver

Error aggregating metric%!(EXTRA zapcore.Field={error 26 0  invalid tag format: "account_id:"})

Unfortunately I don't have access to the full line, but based on your experience does this error message indicate use of a "simple tag"? Due to the trailing : in the error message, it's not clear to me if this is a tag with an empty value vs. a "simple tag".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready to merge Code review completed; ready to merge by maintainers receiver/statsd statsd related issues Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants