-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update pooch downloader: use CESMDATAROOT
variable when available
#52
Conversation
Here's an example of this in action on casper: In [1]: from pop_tools.grid import grid_defs
In [2]: example = grid_defs['POP_gx1v7']
In [3]: example
Out[3]:
{'lateral_dims': [384, 320],
'vertical_dims': 60,
'vert_grid_file': 'gx1v7_vert_grid',
'horiz_grid_fname': 'inputdata/ocn/pop/gx1v7/grid/horiz_grid_20010402.ieeer8',
'topography_fname': 'inputdata/ocn/pop/gx1v7/grid/topography_20161215.ieeei4',
'region_mask_fname': 'inputdata/ocn/pop/gx1v7/grid/region_mask_20151008.ieeei4',
'type': 'dipole',
'region_mask_regions': {'Black Sea': -13,
'Baltic Sea': -12,
'Red Sea': -5,
'Southern Ocean': 1,
'Pacific Ocean': 2,
'Indian Ocean': 3,
'Persian Gulf': 4,
'Atlantic Ocean': 6,
'Mediterranean Sea': 7,
'Lab. Sea & Baffin Bay': 8,
'GIN Seas': 9,
'Arctic Ocean': 10,
'Hudson Bay': 11}}
In [4]: from pop_tools.grid import INPUTDATA
In [5]: from pop_tools.datasets import fetch
In [6]: path = fetch(INPUTDATA, fname=example['horiz_grid_fname'])
In [7]: path
Out[7]: '/glade/p/cesmdata/cseg/inputdata/ocn/pop/gx1v7/grid/horiz_grid_20010402.ieeer8' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@andersy005, thanks for your prompt response to this issue.
One thing I find annoying is the statement when importing pop_tools, that the intputdata
directory is not writable. Will this PR remove that message?
If inputdata
is not writable, we effectively have to assume that all files are present. I see two potential design responses to this:
- Explicitly switch-off
pooch
and makefetch
use direct access. - Have the option of multiple cache directories in order of preference, if a file is not found in 1, use pooch to download it to 2.
What do you think?
CESMDATAROOT
variable when available
Addresses permission issue when local storage path points to a location that we don't have permissions to write to.
Closes #51