[metricbeat] fix ARN parsing in cloudwatch collector for API Gateway metrics #32358
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
For resources with names like "/apis/123-abc", the parsing logic to generate 'short' identifiers should return just the last segment of the name. However currently the split + rejoin logic ends up returning "apis/123-abc". By trimming leading separators we avoid this.
botelastic
bot
added
the
needs_team
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
💚 Flaky test reportTests succeeded. 🤖 GitHub commentsTo re-run your PR in the CI, just comment with:
|
botelastic
bot
removed
the
needs_team
…g-in-cloudwatch-metrics
tommyers-elastic
changed the title
tommyers-elastic
force-pushed
the
fix-filtering-by-tag-in-cloudwatch-metrics
branch
from
0765ef8 to
0eaf7e6
Compare
tommyers-elastic
changed the title
kaiyan-sheng
added
backport-v8.3.0
6 tasks
tommyers-elastic
changed the title
mergify bot
pushed a commit
that referenced
this pull request
Jul 21, 2022
…metrics (#32358) * Trim leading separators from ARN resource strings before parsing. For resources with names like "/apis/123-abc", the parsing logic to generate 'short' identifiers should return just the last segment of the name. However currently the split + rejoin logic ends up returning "apis/123-abc". By trimming leading separators we avoid this. * remove unused tags list from config struct * update unit tests for removal of unused tag field in metrics struct * remove usages of errors.Wrap from aws/utils.go * remove usages of fmt.Println from aws/utils_test.go * update changelog * pr comments (cherry picked from commit e3c609c) # Conflicts: # x-pack/metricbeat/module/aws/cloudwatch/cloudwatch_test.go
mergify bot
pushed a commit
that referenced
this pull request
Jul 21, 2022
…metrics (#32358) * Trim leading separators from ARN resource strings before parsing. For resources with names like "/apis/123-abc", the parsing logic to generate 'short' identifiers should return just the last segment of the name. However currently the split + rejoin logic ends up returning "apis/123-abc". By trimming leading separators we avoid this. * remove unused tags list from config struct * update unit tests for removal of unused tag field in metrics struct * remove usages of errors.Wrap from aws/utils.go * remove usages of fmt.Println from aws/utils_test.go * update changelog * pr comments (cherry picked from commit e3c609c) # Conflicts: # x-pack/metricbeat/module/aws/cloudwatch/cloudwatch_test.go
tommyers-elastic
added a commit
that referenced
this pull request
Jul 25, 2022
…llector for API Gateway metrics (#32445) * [metricbeat] fix ARN parsing in cloudwatch collector for API Gateway metrics (#32358) * Trim leading separators from ARN resource strings before parsing. For resources with names like "/apis/123-abc", the parsing logic to generate 'short' identifiers should return just the last segment of the name. However currently the split + rejoin logic ends up returning "apis/123-abc". By trimming leading separators we avoid this. * remove unused tags list from config struct * update unit tests for removal of unused tag field in metrics struct * remove usages of errors.Wrap from aws/utils.go * remove usages of fmt.Println from aws/utils_test.go * update changelog * pr comments (cherry picked from commit e3c609c) # Conflicts: # x-pack/metricbeat/module/aws/cloudwatch/cloudwatch_test.go * fix merge conflicts Co-authored-by: Tom Myers <106530686+tommyers-elastic@users.noreply.github.com>
tommyers-elastic
added a commit
that referenced
this pull request
Jul 25, 2022
…lector for API Gateway metrics (#32446) * [metricbeat] fix ARN parsing in cloudwatch collector for API Gateway metrics (#32358) * Trim leading separators from ARN resource strings before parsing. For resources with names like "/apis/123-abc", the parsing logic to generate 'short' identifiers should return just the last segment of the name. However currently the split + rejoin logic ends up returning "apis/123-abc". By trimming leading separators we avoid this. * remove unused tags list from config struct * update unit tests for removal of unused tag field in metrics struct * remove usages of errors.Wrap from aws/utils.go * remove usages of fmt.Println from aws/utils_test.go * update changelog * pr comments (cherry picked from commit e3c609c) # Conflicts: # x-pack/metricbeat/module/aws/cloudwatch/cloudwatch_test.go * update changelog with correct PR link Co-authored-by: Tom Myers <106530686+tommyers-elastic@users.noreply.github.com>
chrisberkhout
pushed a commit
that referenced
this pull request
Jun 1, 2023
…metrics (#32358) * Trim leading separators from ARN resource strings before parsing. For resources with names like "/apis/123-abc", the parsing logic to generate 'short' identifiers should return just the last segment of the name. However currently the split + rejoin logic ends up returning "apis/123-abc". By trimming leading separators we avoid this. * remove unused tags list from config struct * update unit tests for removal of unused tag field in metrics struct * remove usages of errors.Wrap from aws/utils.go * remove usages of fmt.Println from aws/utils_test.go * update changelog * pr comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Whilst investigating an issue around tag filtering not working for API Gateway metrics, it was discovered that the ARN parsing logic in the AWS cloudwatch collector did not correctly parse ARNs from cloudwatch which start with a leading
/; this caused all API gateway metrics to get filtered out. The crux of this PR is pre-strip leading slashes (or colons, just in case) from the ARN resource string before splitting. The result is that ARN resource strings like/apis/123,apis/123both result in a 'short' identifier of123.This PR also removes an used field from the internal data structure
metricsWithStatistics, which was causing some additional overhead when trying to grep the code. When updating the test case I added the 'happy path' tagging case to validate the difference between matching tags and not matching tags.Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Author's Checklist
How to test this PR locally
Related issues
Combined with #32408 and #32435, fixes #32408.
Use cases
Screenshots
Logs