Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature 1454 nccf grid definition #1688

Merged
merged 8 commits into from
Mar 1, 2021
Merged

Conversation

hsoh-u
Copy link
Collaborator

@hsoh-u hsoh-u commented Mar 1, 2021

Pull Request Testing

  • Describe testing already performed for these changes:

The input NetcDF was converted by wgrib2. The projection was converted from Lambert Conformal to Lat/Lon. The converted lat/lon is not evenly spaced, so MET should reject it.

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
/d1/personal/hsoh/git/features/feature_1454_nccf_grid_definition/MET/met/src/tools/other/plot_data_plane/plot_data_plane /d1/personal/hsoh/data/MET-1454/nam_prmsl.nc PRMSL_meansealevel.ps 'name="PRMSL_meansealevel"; level="(0,*,*)";' -v 4

DEBUG 4: NcCfFile::get_grid_from_lat_lon_vars() -> lat[290]=41.9683 lat[291]=42.0659  0.00516041 > 0.00513663
ERROR  :
ERROR  : NcCfFile::get_grid_from_lat_lon_vars() -> MET can only process Latitude/Longitude files where the latitudes are evenly spaced (dlat=0.102733, delta[291]=0.0975721)
ERROR  :
DEBUG 4: NcCfFile::get_grid_from_lat_lon_vars() -> lon[130]=240.565 lon[131]=240.676  0.0052606 > 0.00525546
ERROR  :
ERROR  : NcCfFile::get_grid_from_lat_lon_vars() -> MET can only process Latitude/Longitude files where the longitudes are evenly spaced (dlon=0.105109, delta[131]=0.11037)
ERROR  :
ERROR  :
ERROR  : NcCfFile::get_grid_from_lat_lon_vars() -> Please check the input data is the lat/lon projection
ERROR  :
  • Do these changes include sufficient documentation and testing updates? [No]

  • Will this PR result in changes to the test suite? [Yes]

    Since two unit tests were removed, the unit test output be will be different.
    The NCCF with not evenly spaced lat/lon won't be processed. So two unit tests were removed. If the test result is changed, we have to review (it would be expected).

Pull Request Checklist

See the METplus Workflow for details.

  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s), Project(s), and Milestone
  • After submitting the PR, select Linked Issues with the original issue number.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

Summary of changes

  • The percentage is used for tolerance of the delta for lat/lon (fixed number for Lambert Conformal)
  • Added log messages
  • Bug fix on getting lat/lon from 2D lat/lon variables
  • Two unit tests were removed (invalid lat/lon projection): plot_data_plane_CESM_SSMI_microwave & plot_data_plane_CESM_sea_ice_nc
  • The attribute names are moved to nc_utils.h

@@ -343,6 +343,7 @@
</output>
</test>

<!-- The delta of lat/lon is not con
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than just commenting out these tests, why don't we just remove them entirely?
Is there a reason they should be left commented out?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleted test cases which are commented out.

Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve of these changes. MET should be erroring out when the input NetCDF grid is not well-defined. Thanks Howard!

@hsoh-u hsoh-u merged commit 0f5366c into develop Mar 1, 2021
JohnHalleyGotway added a commit that referenced this pull request Mar 2, 2021
* Per #1429, enhance error message from DataLine::get_item(). (#1682)

* Feature 1429 tc_log second try (#1686)

* Per #1429, enhance error message from DataLine::get_item().

* Per #1429, I realize that the line number actually is readily available in the DataLine class... so include it in the error message.

* Feature 1588 ps_log (#1687)

* Per #1588, updated pair_data_point.h/.cc to add detailed Debug(4) log messages, as specified in the GitHub issue. Do still need to test each of these cases to confirm that the log messages look good.

* Per #1588, switch very detailed interpolation details from debug level 4 to 5.

* Per #1588, remove the Debug(4) log message about duplicate obs since it's been moved up to a higher level.

* Per #1588, add/update detailed log messages when processing point observations for bad data, off the grid, bad topo, big topo diffs, bad fcst value, and duplicate obs.

* #1454 Disabled plot_data_plane_CESM_SSMI_microwave and plot_data_plane_CESM_sea_ice_nc becaues of not evenly spaced

* #1454 Moved NC attribute name to nc_utils.h

* #1454 Corrected sanity checking for lat/lon projection based on the percentage of the delta instead of fixed tolerance

* #1454 Corrected sanity checking for lat/lon projection based on the percentage of the delta instead of fixed tolerance

* #1454 Corrected data.delta_lon

* #1454 Change bact to use diff instead of absolute value of diff

* 454 Deleted instea dof commenting out

* 454 Deleted instea dof commenting out

* Feature 1684 bss and 1685 single reference model (#1689)

* Per #1684, move an instance of the ClimoCDFInfo class into PairBase. Also define derive_climo_vals() and derive_climo_prob() utility functions.

* Add to VxPairDataPoint and VxPairDataEnsemble functions to set the ClimoCDFInfo class.

* Per #1684, update ensemble_stat and point_stat to set the ClimoCDFInfo object based on the contents of the config file.

* Per #1684, update the vx_statistics library and stat_analysis to make calls to the new derive_climo_vals() and derive_climo_prob() functions.

* Per #1684, since cdf_info is a member of PairBase class, need to handle it in the PairDataPoint and PairDataEnsemble assignment and subsetting logic.

* Per #1684, during development, I ran across and then updated this log message.

* Per #1684, working on log messages and figured that the regridding climo data should be moved from Debug(1) to at least Debug(2).

* Per #1684 and #1685, update the logic for the derive_climo_vals() utility function. If only a single climo bin is requested, just return the climo mean. Otherwise, sample the requested number of values.

* Per #1684, just fixing the format of this log message.

* Per #1684, add a STATLine::get_offset() member function.

* Per #1684, update parse_orank_line() logic. Rather than calling NumArray::clear() call NumArray::erase() to preserve allocated memory. Also, instead of parsing ensemble member values by column name, parse them by offset number.

* Per #1684, call EnsemblePairData::extend() when parsing ORANK data to allocate one block of memory instead of bunches of litte ones.

* Per #1684 and #1685, add another call to Ensemble-Stat to test computing the CRPSCL_EMP from a single climo mean instead of using the full climo distribution.

* Per #1684 and #1685, update ensemble-stat docs about computing CRPSS_EMP relative to a single reference model.

* Per #1684, need to update Grid-Stat to store the climo cdf info in the PairDataPoint objects.

* Per #1684, remove debug print statements.

* Per #1684, need to set cdf_info when aggregating MPR lines in Stat-Analysis.

* Per #1684 and #1685, update PairDataEnsemble::compute_pair_vals() to print a log message indicating the climo data being used as reference:

For a climo distribution defined by mean and stdev:
DEBUG 3: Computing ensemble statistics relative to a 9-member climatological ensemble.

For a single deterministic reference:
DEBUG 3: Computing ensemble statistics relative to the climatological mean.

Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu>
Co-authored-by: hsoh-u <hsoh@ucar.edu>
@hsoh-u hsoh-u deleted the feature_1454_nccf_grid_definition branch March 9, 2021 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enhance MET to error out when reading CF-compliant NetCDF data with incomplete grid definition.
2 participants