[CELEBORN-1094] Optimize mechanism of ChunkManager expired shuffle key cleanup to avoid memory leak #2053
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## main #2053 +/- ##
==========================================
+ Coverage 46.68% 46.76% +0.08%
==========================================
Files 165 165
Lines 10549 10577 +28
Branches 961 961
==========================================
+ Hits 4924 4945 +21
- Misses 5303 5310 +7
Partials 322 322
... and 3 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
SteNicholas
force-pushed
the
CELEBORN-1094
branch
8 times, most recently
from
c865e2e to
a39ad7f
Compare
|
@waitinfuture, @FMX, could you help to review this pull requst? Otherwise I should always resolve the conflicts. |
FMX
reviewed
Nov 1, 2023
…y cleanup to avoid memory leak
SteNicholas
force-pushed
the
CELEBORN-1094
branch
from
a39ad7f to
e22ef17
Compare
…y cleanup to avoid memory leak
FMX
approved these changes
Nov 2, 2023
FMX
pushed a commit
that referenced
this pull request
Nov 2, 2023
…y cleanup to avoid memory leak ### What changes were proposed in this pull request? The `cleaner` of `Worker` executes the `StorageManager#cleanupExpiredShuffleKey` to clean expired shuffle keys with daemon cached thread pool. The optimization speeds up cleaning including expired shuffle keys of ChunkManager to avoid memory leak. ### Why are the changes needed? `ChunkManager#streams` could lead memory leak when the speed of cleanup is slower than expiration for expired shuffle of worker. The behavior that `ChunkStreamManager` cleanup expired shuffle key should be optimized to avoid memory leak, which causes that the VM thread of worker is 100%. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? `WorkerSuite#clean up`. Closes #2053 from SteNicholas/CELEBORN-1094. Authored-by: SteNicholas <programgeek@163.com> Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com> (cherry picked from commit 4e8e8c2) Signed-off-by: mingji <fengmingxiao.fmx@alibaba-inc.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
The
cleanerofWorkerexecutes theStorageManager#cleanupExpiredShuffleKeyto clean expired shuffle keys with daemon cached thread pool. The optimization speeds up cleaning including expired shuffle keys of ChunkManager to avoid memory leak.Why are the changes needed?
ChunkManager#streamscould lead memory leak when the speed of cleanup is slower than expiration for expired shuffle of worker. The behavior thatChunkStreamManagercleanup expired shuffle key should be optimized to avoid memory leak, which causes that the VM thread of worker is 100%.Does this PR introduce any user-facing change?
No.
How was this patch tested?
WorkerSuite#clean up.