fix: memory leak by pypdfium2 close(maybe) #11510 #11700
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This patch fix a memory leak by updating the pypdfium2 version to latest. close #11510
how to trace the memory leak:
follow the article: https://pythonspeed.com/articles/python-server-memory-leaks/
python -m filprofiler run app.py
python -m pytest api/tests/integration_tests/workflow
the memory profile show almost all memory cost from pypdfium2
search the net found these:
facebookresearch/nougat#110
https://forum.patagames.com/posts/t805-Memory-Leak-in-PDFDocument-Load-when-Pdfium-FPDF-LoadCustomDocument-returns-an-IntPtr-Zero
try to update the pypdfium2 version to latest
https://pyoven.org/package/pypdfium2?tab=versions
than do the steps again
new memory profile looks normal
Tip
Close issue syntax:
Fixes #<issue number>
orResolves #<issue number>
, see documentation for more details.Screenshots
Checklist
Important
Please review the checklist below before submitting your pull request.
dev/reformat
(backend) andcd web && npx lint-staged
(frontend) to appease the lint gods