-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: LEAP-884: avoid reading entire files to memory in export api #5612
Conversation
✅ Deploy Preview for heartex-docs canceled.
|
✅ Deploy Preview for label-studio-docs-new-theme canceled.
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #5612 +/- ##
========================================
Coverage 75.92% 75.93%
========================================
Files 158 158
Lines 13359 13363 +4
========================================
+ Hits 10143 10147 +4
Misses 3216 3216
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
With the Windows failures, we can't merge this as is. Finding a solution will require a bit more time. Update 4/24: seems to be fixed via Django's NamedTemporaryFile |
Resolves #5243
High level idea is to avoid calling
f.read()
in theExportAPI
, usingshutil.copy2
instead where we can.Tested:
pytest tests/sdk/test_tasks.py::test_export_tasks
label-studio export
, for thetasks.functions
changepytest tests/data_export.tavern.yml
, for the export mixin change.