Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[api] Fixes memory leak in hybrid engine #1518

Merged
merged 1 commit into from
Mar 2, 2022

Conversation

frankfliu
Copy link
Contributor

Change-Id: I63f76479ea5acc55cd7725cedf13563157cac6d4

Description

Fixes #1516

Brief description of what this PR is about

  • If this change is a backward incompatible change, why must this change be made?
  • Interesting edge cases to note here

Change-Id: I63f76479ea5acc55cd7725cedf13563157cac6d4
@frankfliu frankfliu requested a review from zachgk as a code owner March 2, 2022 21:42
Copy link
Contributor

@lanking520 lanking520 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please elaborate more where did you find the memory leaks?

@codecov-commenter
Copy link

Codecov Report

Merging #1518 (3e6b94d) into master (bb5073f) will decrease coverage by 1.27%.
The diff coverage is 50.76%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1518      +/-   ##
============================================
- Coverage     72.08%   70.80%   -1.28%     
- Complexity     5126     5326     +200     
============================================
  Files           473      493      +20     
  Lines         21970    23275    +1305     
  Branches       2351     2548     +197     
============================================
+ Hits          15838    16481     +643     
- Misses         4925     5514     +589     
- Partials       1207     1280      +73     
Impacted Files Coverage Δ
api/src/main/java/ai/djl/modality/cv/Image.java 69.23% <ø> (-4.11%) ⬇️
...i/djl/modality/cv/translator/BigGANTranslator.java 21.42% <ø> (-5.24%) ⬇️
...odality/cv/translator/BigGANTranslatorFactory.java 33.33% <0.00%> (+8.33%) ⬆️
...nslator/InstanceSegmentationTranslatorFactory.java 14.28% <0.00%> (-3.90%) ⬇️
.../modality/cv/translator/YoloTranslatorFactory.java 8.33% <0.00%> (-1.67%) ⬇️
...i/djl/modality/cv/translator/YoloV5Translator.java 5.69% <0.00%> (ø)
...odality/cv/translator/YoloV5TranslatorFactory.java 8.33% <0.00%> (-1.67%) ⬇️
...pi/src/main/java/ai/djl/ndarray/BytesSupplier.java 54.54% <0.00%> (-12.13%) ⬇️
...l/training/loss/SigmoidBinaryCrossEntropyLoss.java 64.00% <0.00%> (ø)
api/src/main/java/ai/djl/translate/Translator.java 50.00% <0.00%> (-50.00%) ⬇️
... and 183 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a1441c...3e6b94d. Read the comment docs.

@frankfliu frankfliu merged commit 14fd3be into deepjavalibrary:master Mar 2, 2022
@frankfliu frankfliu deleted the hybrid branch March 2, 2022 23:22
@nguyenvietyen
Copy link

Could you please elaborate more where did you find the memory leaks?

My colleague @andreabrduque discovered it when we performed a load-test (as part of readying for production) of a new model. It causes the nodes to go OOM. She then distilled it to a ticket: #1516

@frankfliu
Copy link
Contributor Author

@nguyenvietyen
I added root cause in the ticket

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Memory issues while using Hybrid Engine
4 participants