cannot upload zarr files #1307

gmazzamuto · 2023-06-30T15:38:08Z

Hi all,

using the latest version I can't upload zarr data because an IsADirectoryError exception is raised. See below.

2023-06-30T17:19:50+0200 [INFO    ] dandi 3077715:140494520747840 dandi v0.55.1, hdmf v3.6.1, pynwb v2.3.3, h5py v3.9.0
2023-06-30T17:19:50+0200 [INFO    ] dandi 3077715:140494520747840 sys.argv = ['/opt/bin/dandi', 'upload', '--allow-any-path', '--validation', 'ignore', 'sub-I59/ses-SPIM/micr/sub-I59_ses-SPIM_sample-BrocaAreaS01_stain-Calretinin_SPIM.ome.zarr']
2023-06-30T17:19:50+0200 [INFO    ] dandi 3077715:140494520747840 os.getcwd() = /mnt/NIH/DANDI/000026
2023-06-30T17:19:50+0200 [DEBUG   ] urllib3.connectionpool 3077715:140494520747840 Starting new HTTPS connection (1): rig.mit.edu:443
2023-06-30T17:19:51+0200 [DEBUG   ] urllib3.connectionpool 3077715:140494520747840 https://rig.mit.edu:443 "GET /et/projects/dandi/dandi-cli HTTP/1.1" 200 579
2023-06-30T17:19:51+0200 [DEBUG   ] dandi 3077715:140494520747840 No newer (than 0.55.1) version of dandi/dandi-cli found available
...
2023-06-30T17:19:53+0200 [DEBUG   ] urllib3.connectionpool 3077715:140494520747840 Starting new HTTPS connection (1): dandiarchive.org:443
2023-06-30T17:19:53+0200 [DEBUG   ] urllib3.connectionpool 3077715:140494520747840 https://dandiarchive.org:443 "GET /server-info HTTP/1.1" 200 354
2023-06-30T17:19:53+0200 [DEBUG   ] dandi 3077715:140494520747840 GET https://api.dandiarchive.org/api/info/
2023-06-30T17:19:53+0200 [DEBUG   ] urllib3.connectionpool 3077715:140494520747840 Starting new HTTPS connection (1): api.dandiarchive.org:443
2023-06-30T17:19:53+0200 [DEBUG   ] urllib3.connectionpool 3077715:140494520747840 https://api.dandiarchive.org:443 "GET /api/info/ HTTP/1.1" 200 354
2023-06-30T17:19:53+0200 [DEBUG   ] dandi 3077715:140494520747840 Response: 200
...
2023-06-30T17:19:58+0200 [DEBUG   ] dandi 3077715:140494520747840 Using API key from keyring
2023-06-30T17:19:58+0200 [DEBUG   ] dandi 3077715:140494520747840 GET https://api.dandiarchive.org/api/auth/token
2023-06-30T17:19:58+0200 [DEBUG   ] urllib3.connectionpool 3077715:140494520747840 https://api.dandiarchive.org:443 "GET /api/auth/token HTTP/1.1" 301 0
2023-06-30T17:19:58+0200 [DEBUG   ] urllib3.connectionpool 3077715:140494520747840 https://api.dandiarchive.org:443 "GET /api/auth/token/ HTTP/1.1" 200 42
2023-06-30T17:19:58+0200 [DEBUG   ] dandi 3077715:140494520747840 Response: 200
2023-06-30T17:19:58+0200 [DEBUG   ] dandi 3077715:140494520747840 Found identifier DANDI:000026 in top level 'identifier'
2023-06-30T17:20:17+0200 [INFO    ] dandi 3077715:140494520747840 Found 2 files to consider
2023-06-30T17:20:19+0200 [WARNING ] bids-schema 3077715:140493548414720 No BIDS reference root provided.
2023-06-30T17:20:19+0200 [INFO    ] bids-schema 3077715:140493548414720 No schema path specified, defaulting to the bundled schema, `/opt/lib/python3.8/site-packages/bidsschematools/data/schema`.
...
2023-06-30T17:21:58+0200 [DEBUG   ] dandi 3077715:140493548414720 GET https://api.dandiarchive.org/api/dandisets/000026/versions/draft/assets/
2023-06-30T17:21:58+0200 [DEBUG   ] urllib3.connectionpool 3077715:140493548414720 Resetting dropped connection: api.dandiarchive.org
2023-06-30T17:21:59+0200 [DEBUG   ] urllib3.connectionpool 3077715:140493548414720 https://api.dandiarchive.org:443 "GET /api/dandisets/000026/versions/draft/assets/?path=sub-I59%2Fses-SPIM%2Fmicr%2Fsub-I59_ses-SPIM_sample-BrocaAreaS01_stain-Calretinin_SPIM.ome.zarr HTTP/1.1" 200 52
2023-06-30T17:21:59+0200 [DEBUG   ] dandi 3077715:140493548414720 Response: 200
2023-06-30T17:22:01+0200 [DEBUG   ] fscacher.cache 3077715:140493548414720 Calling memoized version of <function get_dandietag at 0x7fc69e831ca0> for /mnt/NIH/DANDI/000026/sub-I59/ses-SPIM/micr/sub-I59_ses-SPIM_sample-BrocaAreaS01_stain-Calretinin_SPIM.ome.zarr
2023-06-30T17:22:01+0200 [DEBUG   ] fscacher.cache 3077715:140493548414720 Running original <function get_dandietag at 0x7fc69e831ca0> on PosixPath('/mnt/NIH/DANDI/000026/sub-I59/ses-SPIM/micr/sub-I59_ses-SPIM_sample-BrocaAreaS01_stain-Calretinin_SPIM.ome.zarr')
2023-06-30T17:22:01+0200 [ERROR   ] dandi 3077715:140493548414720 Error uploading /mnt/NIH/DANDI/000026/sub-I59/ses-SPIM/micr/sub-I59_ses-SPIM_sample-BrocaAreaS01_stain-Calretinin_SPIM.ome.zarr:
Traceback (most recent call last):
  File "/opt/lib/python3.8/site-packages/dandi/upload.py", line 263, in process_path
    for r in dfile.iter_upload(
  File "/opt/lib/python3.8/site-packages/dandi/files/bases.py", line 339, in iter_upload
    etagger = get_dandietag(self.filepath)
  File "/opt/lib/python3.8/site-packages/fscacher/cache.py", line 157, in fingerprinter
    ret = fingerprinted(*args, **kwargs_)
  File "/opt/lib/python3.8/site-packages/joblib/memory.py", line 655, in __call__
    return self._cached_call(args, kwargs)[0]
  File "/opt/lib/python3.8/site-packages/joblib/memory.py", line 598, in _cached_call
    out, metadata = self.call(*args, **kwargs)
  File "/opt/lib/python3.8/site-packages/joblib/memory.py", line 856, in call
    output = self.func(*args, **kwargs)
  File "/opt/lib/python3.8/site-packages/fscacher/cache.py", line 97, in fingerprinted
    return f(path, *args, **kwargs)
  File "/opt/lib/python3.8/site-packages/dandi/support/digests.py", line 100, in get_dandietag
    return DandiETag.from_file(filepath)
  File "/opt/lib/python3.8/site-packages/dandischema/digests/dandietag.py", line 147, in from_file
    with open(path, "rb") as f:
IsADirectoryError: [Errno 21] Is a directory: '/mnt/NIH/DANDI/000026/sub-I59/ses-SPIM/micr/sub-I59_ses-SPIM_sample-BrocaAreaS01_stain-Calretinin_SPIM.ome.zarr'
2023-06-30T17:22:01+0200 [DEBUG   ] urllib3.connectionpool 3077715:140494520747840 Starting new HTTPS connection (1): rig.mit.edu:443
2023-06-30T17:22:01+0200 [DEBUG   ] urllib3.connectionpool 3077715:140494520747840 https://rig.mit.edu:443 "GET /et/projects/dandi/dandi-cli HTTP/1.1" 200 579
2023-06-30T17:22:01+0200 [DEBUG   ] dandi 3077715:140494520747840 Caught exception [Errno 21] Is a directory: '/mnt/NIH/DANDI/000026/sub-I59/ses-SPIM/micr/sub-I59_ses-SPIM_sample-BrocaAreaS01_stain-Calretinin_SPIM.ome.zarr'

The text was updated successfully, but these errors were encountered:

jwodder · 2023-07-05T13:16:36Z

@gmazzamuto

Is the dataset on an NFS file system?
Is sub-I59_ses-SPIM_sample-BrocaAreaS01_stain-Calretinin_SPIM.ome.zarr a symlink or the target of a symlink?
Is sub-I59_ses-SPIM_sample-BrocaAreaS01_stain-Calretinin_SPIM.ome.zarr empty?

gmazzamuto · 2023-07-05T13:33:43Z

Yes, the dataset is on an NFS file system. The directory is not a symlink and is not empty.

gmazzamuto · 2023-09-28T15:17:20Z

Hi all, any news on this one? I have a bunch of new data to upload. Otherwise I'll have to upload in OME-TIFF rather than zarr, to meet the project deadlines

yarikoptic · 2023-09-28T16:42:01Z

I thought we provided guidance, sorry -- slipped through, probably mixed up with #1321. Could you please

reconfirm that issue persists and traceback is similarly implicating fscacher
try upload while specifying DANDI_CACHE=clear, and if that doesn't help:
try upload while specifying DANDI_CACHE=ignore
report back on either 👍 or 👎

edit note: I have added clear in addition to original ignore only

yarikoptic · 2023-09-28T16:52:44Z

1 more question: which dandiset it is? it is BIDS one right?

yarikoptic · 2023-09-28T17:00:29Z

one more... if you get a chance before trying above, sorry for the flood, it would help

DANDI_DEVEL=1 dandi --pdb upload  --devel-debug ...the rest of options like you did

then whenever it errors out and gets into debugger, could you please show me what is that object is . Do command up in pdb a number of times until you get to the level of File "/opt/lib/python3.8/site-packages/dandi/files/bases.py", line 339, in iter_upload. There just do p self and show what that self is ;-)

PS edit: above is for myself to see if it is due to the fact that ZarrBIDSAsset(BIDSAsset, ZarrAsset): and BIDSAsset(LocalFileAsset): so I think that iter_upload gets from LocalFileAsset not from ZarrAsset here. We would need to overload to ensure that correct one is used, or may be just place ZarrAsset first in multiple inheritance?... need to check if we have any test which uploads zarr from a bids dataset.

github-actions · 2023-09-29T03:06:38Z

🚀 Issue was released in 0.56.2 🚀

github-actions · 2023-09-29T03:06:41Z

🚀 Issue was released in 0.56.2 🚀

gmazzamuto · 2023-10-03T10:41:31Z

OK thanks, seems fixed now

yarikoptic assigned jwodder Jun 30, 2023

yarikoptic added the bug Something isn't working label Jun 30, 2023

CodyCBakerPhD mentioned this issue Jul 17, 2023

[Feature] Support upload of Zarr-backend NWB files #1310

Open

yarikoptic-gitmate assigned yarikoptic-gitmate and unassigned jwodder Aug 10, 2023

yarikoptic mentioned this issue Sep 28, 2023

Fix uploading Zarr within a BIDS dataset; typing-check guided fix for handling requests exception #1331

Merged

2 tasks

yarikoptic closed this as completed in #1331 Sep 29, 2023

github-actions bot added the released label Sep 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cannot upload zarr files #1307

cannot upload zarr files #1307

gmazzamuto commented Jun 30, 2023

jwodder commented Jul 5, 2023

gmazzamuto commented Jul 5, 2023

gmazzamuto commented Sep 28, 2023

yarikoptic commented Sep 28, 2023 •

edited

Loading

yarikoptic commented Sep 28, 2023

yarikoptic commented Sep 28, 2023 •

edited

Loading

github-actions bot commented Sep 29, 2023

github-actions bot commented Sep 29, 2023

gmazzamuto commented Oct 3, 2023

cannot upload zarr files #1307

cannot upload zarr files #1307

Comments

gmazzamuto commented Jun 30, 2023

jwodder commented Jul 5, 2023

gmazzamuto commented Jul 5, 2023

gmazzamuto commented Sep 28, 2023

yarikoptic commented Sep 28, 2023 • edited Loading

yarikoptic commented Sep 28, 2023

yarikoptic commented Sep 28, 2023 • edited Loading

github-actions bot commented Sep 29, 2023

github-actions bot commented Sep 29, 2023

gmazzamuto commented Oct 3, 2023

yarikoptic commented Sep 28, 2023 •

edited

Loading

yarikoptic commented Sep 28, 2023 •

edited

Loading