-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make IMSFV3 converter more robust #1497
Comments
We can add a line to find the basename ( |
The best way is IMS processing code to output time into nc4 file, the ioda converter read time directly from there. This also happened for SMAP ioda converter. We finally solve it by reading date and time from hd5 file directly. @jiaruidong2017, if you have experience to add a line to find the basename and then get the date, that is great. It should not be hard to add time into the repocessed nc4 file from ims processing code. Here I include @ClaraDraper-NOAA here. |
I agree @YoulongXia-NOAA it's dangerous to infer from filenames and if we can put it in the file itself that would be the best solution. |
@WalterKolczynski-NOAA suggested here: NOAA-EMC/global-workflow#2484 (comment) some regex that would be a quick fix. |
My solution there only works if the date is always delimited by periods. If that isn't the case (as in the example above), it won't work as-is. I agree with @YoulongXia-NOAA reading the date directly from the file is the best long-term solution. |
I am working on this issue. Example data has be produced and the PR (NOAA-PSL/land-SCF_proc#5) is under review. Offline modified imsfv3_scf2ioda.py which reads time directly from data file has been checked and it works. Now I am planning to work on iodacov within ioda-bundle to update imsfv3_scf2ioda.py. |
https://github.com/JCSDA-internal/ioda-converters/blame/0bc00802515fe5834fb70311bfcac3f4cf12d349/src/land/imsfv3_scf2ioda.py#L101
This tries to get the date from the filename, and works fine if:
/path/to/file/20240416_sample.nc4
but if one is doing something like:
/path/to/jenny/86753099/work/20240416_sample.nc4
it will find the wrong 8 digit integer.
First found by @aerorahul as part of global-workflow CI, as the hash happened to be an 8 digit integer.
The text was updated successfully, but these errors were encountered: