Don't let partial replication unnecessarily divide push commands #229
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Check-perf-impact results: (f289611903bd51c6db3f757a138e69d1) Relative execution time per category: (mean of relative medians)
|
PeterTh
approved these changes
Dec 4, 2023
fknorr
approved these changes
Dec 5, 2023
fknorr
pushed a commit
to fknorr/celerity-runtime
that referenced
this pull request
Dec 5, 2023
…e push commands
psalz
force-pushed
the
fix-replication-push-division
branch
from
a8f4fe0 to
d64b10e
Compare
psalz
force-pushed
the
fix-replication-push-division
branch
from
d64b10e to
5cb9dcf
Compare
|
Check-perf-impact results: (4c65f1399a47e0eb1340f63004745b17) 🚀 Significant speedup (<0.80x) in some microbenchmark results: building command graphs in a dedicated scheduler thread for N nodes - 1 > immediate submission to a scheduler thread / expanding tree topology Relative execution time per category: (mean of relative medians)
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This was surfaced by 2D splits, but can also be constructed (somewhat awkwardly) for 1D: The extra region map lookup into
buffer_state.replicated_regionscould sometimes cause a box with the same last writer command to be unnecessarily split into several push commands. The fix is to collect all non-replicated boxes and do a final merge before generating the push commands.