WIP Let GMT detect if DCW version on server has been updated #6240
Conversation
For those who rely on the cloud to get the latest DCW, this PR will check if the version in the cloud is fresher than theirs, and if so we delete the local files in .gmt/geography/dcw so new ones can be downloaded.
|
What if you have custom entries in |
|
You should have those in ./gmt/dcw.conf. The downloaded DCW files are in .gmt/geography/dcw so no collisions. |
|
Could this be used to support the |
|
Yes, probably. |
|
We have to be careful not to over-complicate this. Consider those who install DCW from their Linux repo or our installers: We currently check that directory first for DCW and if found we do not go on and check if there might be a more recent version in the cloud. If we did, we certainly do not want to override the first directory which may be under system control w.r.t. permissions. So then what? Do we always check the cloud no matter what so we can inform the user (once a day) that they are behind, which also depends on the GMT version needed to handle the latest DCW). Or do we restrict the update to those who installed from cloud in the first place only. Also if we found the cloud data is more recent than what the repo/installer placed, and if we therefore downloaded into .gmt/geograpjhy/dcw, do we now have to always check all these places to find the most recent DCW before we open files? I.e., open all those dcw-gmt.nc files, get version, and go with the most recent. Who amongst you want to code all this up? I think there are legitimate reasons why many will want to install dcw-gmt from a repo or installer (permission issues, automation, local restrictions). These people may not want to have GMT auto-update anything. I suspect none of this is clearly documented, and not sure if this behavior is ideal anyway. Happy to take comments on this. |
|
I am bit lost. I have Ubuntu. I can't test this, right? |
|
You can, since you are build GMT on ubuntu, not installing gmt6.x.x form the ubuntu bistro. But you would have to do the same I did to test: Build the branch normally - this will then use your DCW installation. |
|
I don't have any strong opinions about this, but (mixed bag);
|
It doesn't. It would be only during runtime.
No. |
Yes, I think we can assume that anyone who choses a different path than having GMT auto-download to |
|
OK, this PR is starting to get old (Jan 18). I suggest we start with simply letting cloud-derived DCW be updated and anyone with other forms for dcw installations shall deal with those manually (or via their Linux repo). That is the extent of this PR. Enabling |
|
I try it on ubuntu and failed. |
|
I just tried installing gmt without supplying paths to gshhg and dcw. It looks like gmt only fetches the resolution it needs for the current job, not the whole thing (i.e. all resolutions). This applies both when getting gshhg and dcw. I think #4040 is a good idea - and get all the files. better be done getting everything before your internet is gone.. |
|
Just a question... is there an access to older version of DCW? What happen if someone nostalgic of good ol' times needs consistency? |
I think that should make a personal copy of the dataset or turnoff when the server data is update ( |
|
But we can't go back right? Once it's updated, we're done for |
|
Mmm I think that you could donwload a previous version of DCW from Github. |
For those who did not install DCW from a tarball or Linux distributions, GMT will download the latest from the gmt server. However, until this PR there was no mechanism to update files if we release a newer version, like we just did with 2.1.0.
This PR will check if the version in the cloud is fresher than the one in.gmt/geography/dcw , and if so we delete the local files so new ones can be downloaded. Note: This does not affect the installation of those that pre-install DCW from the tarball or via Linux distributions. They need to update via their distribution or get the latest tarball.
Here I have renamed my 2.1.0 so it could not be found and my ./gmt/geography/dcw contained old 2.0.1 files. I then run
gmt pscoast -RFR -JM15c -Baf -Gred -P -V > t.pswhich will need to access DCW. Here is what happens (I have removed unrelated messages:
Now my .gmt/geography/dcw contains the contents of the latest release, including dcw-collections.txt. Rerunning this command shows that the files are accepted and no new downloads take place:
Like for other remote files, we check the presence of and age of .gmt/geography/dcw/VERSION and if older than a day we refresh it. It fetches the 6 byte file from the server.
Note: I have labelled this WIP in case there are some unintended consequences - hence I hope you can give this a spin - you may need to hide or remove any DCW files installed from tarball or installers.