-
Notifications
You must be signed in to change notification settings - Fork 529
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add block compaction delay metric #7635
Conversation
8b5ed9d
to
254817a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a first step, I'd prefer to avoid adding new field to struct representing meta.json
.
b40deef
to
7e1efd8
Compare
848ae2c
to
bfd792b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you. Left some minor comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you. (We may want to mention new metric in the changelog too)
- Change summary to a histogram - Make LastUpdated field transient - Fetch attributes only when compactor is fetching meta - Use LastUpdated when filtering jobs
- remove LastUpdated from block meta - change metric to minutes and use buckets with longer durations - only fetch attributes for compaction job filtering and recording the new metric
Co-authored-by: Peter Štibraný <pstibrany@gmail.com>
This reverts commit 55ef1ddd10b5b688a002b2cfb998055c92789b08.
ae86596
to
cabea8d
Compare
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-7635-to-r282 origin/r282
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x 96952cf7bbfbdbf5fa77f22032e834a12a8ac236
# Push it to GitHub
git push --set-upstream origin backport-7635-to-r282
git switch main
# Remove the local backport branch
git branch -D backport-7635-to-r282 Then, create a pull request where the |
Add block compaction delay summary metric Co-authored-by: Peter Štibraný <pstibrany@gmail.com> (cherry picked from commit 96952cf)
Add block compaction delay summary metric Co-authored-by: Peter Štibraný <pstibrany@gmail.com> (cherry picked from commit 96952cf)
Add block compaction delay summary metric Co-authored-by: Peter Štibraný <pstibrany@gmail.com> (cherry picked from commit 96952cf)
Add block compaction delay summary metric Co-authored-by: Peter Štibraný <pstibrany@gmail.com> (cherry picked from commit 96952cf)
What this PR does
Adds a new summary metric to compactor,
cortex_compactor_block_compaction_delay_seconds
, which is the time between a block being created/updated and it being successfully compacted. The motivation for this metric is wanting to understand when there are delays in compacting blocks, such as from compactors being or restarting, that may indirectly cause non-compacted blocks to be queried via store-gateways.Which issue(s) this PR fixes or relates to
Fixes #
Checklist
CHANGELOG.md
updated - the order of entries should be[CHANGE]
,[FEATURE]
,[ENHANCEMENT]
,[BUGFIX]
.about-versioning.md
updated with experimental features.