Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gdas-validation naming convention for diagnostic files #735

Closed
RussTreadon-NOAA opened this issue Nov 16, 2023 · 2 comments
Closed

gdas-validation naming convention for diagnostic files #735

RussTreadon-NOAA opened this issue Nov 16, 2023 · 2 comments

Comments

@RussTreadon-NOAA
Copy link
Contributor

The finalize method in g-w ush/python/pygfs/task/atm_analysis.py assumes fv3jedi_var.x netcdf diagnostic files end with the suffix .nc4.

         # get list of diag files to put in tarball
        diags = glob.glob(os.path.join(self.task_config.DATA, 'diags', 'diag*nc4'))

This is not the case in gdas-validation. Our yamls use .nc as the suffix for netcdf diagnostic files. For example,

satwind_goes-16.yaml:      obsfile: $(DATA)/diags/diag_satwind_abi_goes-16_{{ current_cycle | to_YMDH }}.nc
satwind_goes-17.yaml:      obsfile: $(DATA)/diags/diag_satwind_abi_goes-17_{{ current_cycle | to_YMDH }}.nc
scatwind_metop-a.yaml:      obsfile: $(DATA)/diags/diag_ascatw_ascat_metop-a_{{ current_cycle | to_YMDH }}.nc
scatwind_metop-b.yaml:      obsfile: $(DATA)/diags/diag_ascatw_ascat_metop-b_{{ current_cycle | to_YMDH }}.nc

As a result gdasatmanlfinal creates an empty atmstat file (diagnostic file tarball).

We need to either

  1. change the diagnostic file suffix to .nc4 in the observation yamls, or
  2. update g-w atm_analysis.py to look for diagnostic files with the .nc suffix

This gets to the larger question of what suffix to use for netcdf files as we transition to JEDI. Currently the GFS uses .nc even though GFS .nc files are netCDF-4.

@CoryMartin-NOAA
Copy link
Contributor

Building off of this discussion, the Unidata netCDF4 developers seem to suggest that .nc is their preferred suffix (see https://docs.unidata.ucar.edu/netcdf-c/current/faq.html#why-arent-different-extensions-used and https://docs.unidata.ucar.edu/netcdf-c/current/faq.html#What-convention-should-be-used-for-the-names-of-netCDF-files)

But we should discuss the convention of filenames in order to ensure that all the IODA observation file names are consistently named, we can iterate and comment on this Google doc (anyone with NOAA credentials can access): https://docs.google.com/document/d/1QLq5a_uVlTkcUkY8CfLOHCRzcLj5yvptigaiQ8B2LtY/edit?usp=sharing

@RussTreadon-NOAA
Copy link
Contributor Author

Partially addressed by PR #937

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants