-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH: Allow DatasetMetadataExtractor.get_required_content()
to return a generator
#361
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, thanks a lot. A few comments below
Co-authored-by: Christian Mönch <christian.moench@web.de>
Co-authored-by: Christian Mönch <christian.moench@web.de>
Co-authored-by: Christian Mönch <christian.moench@web.de>
Co-authored-by: Christian Mönch <christian.moench@web.de>
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #361 +/- ##
==========================================
- Coverage 86.42% 86.28% -0.14%
==========================================
Files 88 88
Lines 4831 4841 +10
==========================================
+ Hits 4175 4177 +2
- Misses 656 664 +8
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
…oneType return in extraction code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks @jsheunis
Great! @christian-monch feel free to merge if you're happy. |
DatasetMetadataExtractor.get_required_content()
to yield a generatorDatasetMetadataExtractor.get_required_content()
to return a generator
This PR:
extract.py
to allow either a returnedbool
or a yieldedGenerator
to be interpreted correctly: If an extractor implements logic where any result record is yielded with a failure 'status' (i.e. equal to 'impossible' or 'error') the extractor infrastructure will signal an error and the extractor's extract method will not be called.DatasetMetadataExtractor.get_required_content()
inextractors/base.by
to indicate this change, especially to developers implementing extractorsreturn
statements inextract.py
where failures are yieldedDatasetMetadataExtractor.get_required_content()
should be able toyield
datalad result dicts #356