-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add compressor option to write image #161
Conversation
this proposes a change to the write_image api to allow compressors
for more information, see https://pre-commit.ci
Codecov Report
@@ Coverage Diff @@
## master #161 +/- ##
==========================================
+ Coverage 83.53% 83.99% +0.46%
==========================================
Files 12 12
Lines 1354 1362 +8
==========================================
+ Hits 1131 1144 +13
+ Misses 223 218 -5
Continue to review full report at Codecov.
|
Thanks for this, @satra. Could you sign and return the OME CLA as described in https://ome-contributing.readthedocs.io/en/latest/cla.html? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Implementation seems simple enough, though we should probably get it integrated into a test.
As a side note, I do remember, however, not being completely happy with the chunks
argument since conceivably one might want different settings per pyramid level. The same might apply here.
I don't know if you, @satra, or anyone else at this point has a clear alternative that would keep this method cleaner. Options that come to mind include:
- passing all arguments as key words similar to fsspec's
storage_options
- and/or passing a callback function which review is given the path, the dataset, as well as the resolution level
Happy to keep pondering those after this PR though.
Brainstorming on the API questions raised in #161 (review), a solution along the lines of Do we expect the |
@sbesson - updated this to use |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @satra. We are making progress towards ome-zarr 0.3.0
which should include OME-NGFF 0.4 support. It would be great to make this new storage options available at the same time.
At minimum, could a unit test be added that covers the compression scenario? As a bonus, it looks like your storage_options
also cover the per-resolution chunks
feature mentioned in #161 (review). I suspect we will want some tests covering this scenario but this can happen as a follow-up PR depending on time.
Also could you return the CLA as mentioned in #161 (comment).
else storage_options[path] | ||
) | ||
if "chunks" not in options: | ||
options["chunks"] = chunks |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This raises the question of whether we would like to deprecate the explicit chunks
argument and only rely on the storage_options
dictionary moving forward.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if it's deprecated, i would suggest giving it a couple of cycles and perhaps add a deprecation warning with the next release. but leave this up to you.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair enough, we can gain knowledge about the storage_options
in the upcoming 0.3.x and schedule the deprecation of chunks
for 0.4. I'll create an issue
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the test @satra. All looks good from my side. Merging and we will aim to get this as a public release in the upcoming days
else storage_options[path] | ||
) | ||
if "chunks" not in options: | ||
options["chunks"] = chunks |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair enough, we can gain knowledge about the storage_options
in the upcoming 0.3.x and schedule the deprecation of chunks
for 0.4. I'll create an issue
this proposes a change to the write_image api to allow compressors. this is now more general to pass any options to the create dataset function and should handle multilevel options.
closes #160