Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ensure storage_options are passed to data loader #508

Merged
merged 1 commit into from
Aug 22, 2022

Conversation

andersy005
Copy link
Member

@andersy005 andersy005 commented Aug 22, 2022

Change Summary

@mgrover1, this PR should fix the issue you ran into with storage_options

In [1]: import intake

In [2]: catalog = intake.open_esm_datastore(
   ...:     'https://raw.githubusercontent.com/NCAR/cesm2-le-aws/main/intake-catalogs/aws-cesm2-le.json'
   ...: )
   ...: catalog
Out[2]: <IPython.core.display.HTML object>

In [3]: catalog_subset = catalog.search(variable='TREFHT', frequency='daily')
   ...: catalog_subset
Out[3]: <IPython.core.display.HTML object>

In [4]: dsets = catalog_subset.to_dataset_dict(storage_options={'anon':True})

--> The keys in the returned dictionary of datasets are constructed as follows:
        'component.experiment.frequency.forcing_variant'
 |████████████████████████████████████████████████████████████████████████████████████████| 100.00% [4/4 00:01<00:00]
In [5]: 

Related issue number

Checklist

  • Unit tests for the changes exist
  • Tests pass on CI
  • Documentation reflects the changes where applicable

@andersy005 andersy005 enabled auto-merge (squash) August 22, 2022 19:44
@andersy005 andersy005 merged commit bb15984 into main Aug 22, 2022
@andersy005 andersy005 deleted the fix-storage-options branch August 22, 2022 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

To Dataset Dict failing with CESM2-LE Example
1 participant