feat(resharding): simple nayduck test for resharding #10162
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
wacban
commented
Nov 13, 2023
wacban
commented
- Added a simple nayduck test for resharding. It checks that the shard layout version and number of shards change at the right block.
- Added configuration for the delay while waiting for the state snapshot. Needed it, otherwise the test was all over the place.
- Refactored some common resharding test logic in nayduck.
wacban
commented
Nov 13, 2023
Comment on lines
-2519
to
-2520
| // For colour decorators to work, they need to printed directly. Otherwise the decorators get escaped, garble output and don't add colours. | ||
| debug!(target: "catchup", progress_per_shard = ?format_shard_sync_phase_per_shard(&shards_to_split, false), "Need to split states for shards"); |
There was a problem hiding this comment.
This log line doesn't make much sense.
- It is called multiple times, including after resharding is started but it doesn't actually know about the current status of it.
- There is a different log line just a bit later that prints the current status anyway.
Comment on lines
-2389
to
-2392
| // TODO(resharding) what happens to the shards_to_split here when | ||
| // catchup_state_syncs already contains an entry for the sync hash? | ||
| // Does it get overwritten? Are we guaranteed that the existing | ||
| // entry contains the same data? |
There was a problem hiding this comment.
Checked that accidentaly. It's fine.
shreyan-gupta
approved these changes
Nov 14, 2023
| Self { | ||
| batch_size: bytesize::ByteSize::kb(500), | ||
| batch_delay: Duration::from_millis(100), | ||
| retry_delay: Duration::from_secs(10), |
There was a problem hiding this comment.
Fine with any value you find appropriate. I don't think it's going to matter too much as long as it's in the domain of a couple of seconds.
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to no response for status checks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.