-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Clean up shard data when dropping retention policies #5691
Conversation
ada132a
to
f57e2a0
Compare
Looks like this will fix #5394 as well. |
@@ -252,6 +261,7 @@ func (s *Store) CreateShard(database, retentionPolicy string, shardID uint64) er | |||
} | |||
|
|||
s.shards[shardID] = shard | |||
s.shardLocations[shardID] = &shardLocation{Database: database, RetentionPolicy: retentionPolicy, Shard: shard} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any reason not to just combine shards
and shardLocations
maps? shardLocation
has a pointer to the Shard
which is all that shards
has.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah that was my thinking, but I wanted to get feedback on whether it's OK to do away with shards
first. Seems I forgot to even mention that in the PR description 😄, so thanks for bring it up!
I'll refactor this morning.
9e73c57
to
be78c8c
Compare
be78c8c
to
241a92b
Compare
for _, sh := range s.shards { | ||
s.performMaintenanceOnShard(sh) | ||
for _, sl := range s.shardLocations { | ||
s.performMaintenanceOnShard(sl.Shard) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this need to check that sl.Shard
is not nil as well? It's done in other places.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess, yeah. To be honest a shardLocation
shouldn't have a nil
Shard, but better to be defensive 👍
One last item. But 👍 after that. |
241a92b
to
012b89c
Compare
for _, sh := range s.shards { | ||
if err := sh.Close(); err != nil { | ||
for _, sl := range s.shardLocations { | ||
if err := sl.Shard.Close(); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Possibly check for nil
? Although NewShard()
currently never returns nil, and that appears to be the only value that gets stored in sl.Shard
, I see that you've checked for nil
in other places for this field.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or, (*Shard).Close()
can be updated to handle a nil
receiver.
Fixes #5653 and #5394. Previously dropping retention policies did not propogate to local TSDB shards. Instead, the retention policiess would just be removed from the Meta Store. This PR adds ensures that data associated with retention policies is removed, when the retention policy is dropped. Also, it cleans up a couple of other methods in `tsdb`, including the requirement to provide (redundant) shardIDs when deleting databases.
012b89c
to
99a7341
Compare
Discussed defensive |
Clean up shard data when dropping retention policies
Fixes #5653.
tsdb.Store
for deleting shards in a retention policy, and cleaning up said retention policy.tsdb.Store.DeleteDatabase
so that it does not require the ids of the shards within the database.