Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: prevent retention service creating orphaned shard files (#24530)(#24547) #24548

Merged
merged 1 commit into from
Jan 5, 2024

Conversation

davidby-influx
Copy link
Contributor

@davidby-influx davidby-influx commented Jan 5, 2024

Under certain circumstances, the retention service can fail to delete shards from the store in a timely manner. When the shard groups are pruned based on age, this leaves orphaned shard files on the disk. The retention service will then not attempt to remove the obsolete shard files because the meta store does not know about them. This can cause excessive disk space usage for some users.

This corrects that by requiring shards files be deleted before they can be removed from the meta store.

fixes: #24529
(cherry picked from commit 7bd3f89)

closes #24545

Co-authored-by: Geoffrey Wossum gwossum@influxdata.com
(cherry picked from commit 0dc48b1)

closes #24546

…#24547)

Under certain circumstances, the retention service can fail to delete shards from
the store in a timely manner. When the shard groups are pruned based on age, this
leaves orphaned shard files on the disk. The retention service will then not attempt
to remove the obsolete shard files because the meta store does not know about them.
This can cause excessive disk space usage for some users.

This corrects that by requiring shards files be deleted before they can be removed
from the meta store.

fixes: #24529
(cherry picked from commit 7bd3f89)
closes #24545

Co-authored-by: Geoffrey Wossum <gwossum@influxdata.com>
(cherry picked from commit 0dc48b1)
closes #24546
@davidby-influx davidby-influx requested a review from gwossum January 5, 2024 01:26
@davidby-influx davidby-influx self-assigned this Jan 5, 2024
Copy link
Member

@gwossum gwossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@davidby-influx davidby-influx merged commit 09a9607 into 2.7 Jan 5, 2024
28 checks passed
@davidby-influx davidby-influx deleted the DSB_retention_2.7 branch January 5, 2024 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/tsm area/2.x OSS 2.0 related issues and PRs kind/bug team/edge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants