Migrate image data to raster schema proposal #90
Conversation
| # Consolidate metadata into single .zmetadata for pyramid | ||
| # https://zarr.readthedocs.io/en/stable/tutorial.html#consolidating-metadata | ||
| z_group = zarr.open(str(zarr_path)) | ||
| zarr.consolidate_metadata(z_group.store) |
There was a problem hiding this comment.
This isn't actually going to be used, right? Like, zarr.js will still use the .zarray file for parsing? If this is only for future use, maybe a comment about that would be helpful
There was a problem hiding this comment.
I intend to use this to gather what the levels are in the pyramid. I'll be adding the convenience method openConsolidated to zarr.js (which is implemented in zarr-python) but this way we have zarr-generated information about where the levels are relative to the store base.
let data;
if (isPyramid) {
const { metadata } = await (fetch(`${url}/.zmetadata`).then(res => res.json());
const levels = Object.keys(metadata);
data = levels.map(key => zarr.openArray({ store: url, path: key});
} else {
data = zarr.openArray({ store: url });
}
const loader = ZarrLoader(data);| # Consolidate zarr metadata for easy access to array/group heirachy | ||
| # https://zarr.readthedocs.io/en/stable/tutorial.html#consolidating-metadata | ||
| z_group = zarr.open(str(zarr_path)) | ||
| zarr.consolidate_metadata(z_group.store) |
| import json | ||
| import urllib | ||
|
|
||
| from tile_zarr_base import tile_zarr |
There was a problem hiding this comment.
I'm just curious what the status of img2zarr with respect to this stuff is?
There was a problem hiding this comment.
The tile_zarr_base is essentially a port from what I've implemented in img2zarr right now. I think the efficient converters to zarr base are really immature and neither Heath or myself have the bandwidth to take that on at the moment. I'd prefer to write custom in vitessce-data until that matures more.
| return bool((np.log2(shape) >= 12).any()) | ||
|
|
||
|
|
||
| class OmeTiffReader: |
There was a problem hiding this comment.
Maybe add a comment that this only reads tiff and outputs zarr, and does not actually do the tiling/pyramid generation?
I think the plumbing is mostly in place now. I have updated the fixtures and tried to consolidate some of the code. Now readers use the
tile_zarr_basemodule to generate the image pyramids.