Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

address some comments; add duplication checker script #2

Merged
merged 2 commits into from
May 26, 2021
Merged

address some comments; add duplication checker script #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
708c2a8
address some comments; add duplication checker script
grantfirl May 24, 2021
1d7fe51
add duplicate removal capability to check_xml_unique.py
grantfirl May 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 11 additions & 23 deletions Metadata-standard-names.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Currently, the only dimension which supports all six dimension types is horizont
* `integer`: units = count
* `vertical_dimension`: number of vertical levels
* `integer`: units = count
* `vertical_dimension_plus_one`: number of vertical levels + 1
* `vertical_interface_dimension`: number of vertical interfaces
* `integer`: units = count
* `vertical_layer_index`: index of a particular vertical level
* `integer`: units = count
Expand All @@ -53,7 +53,7 @@ Currently, the only dimension which supports all six dimension types is horizont
## constants
* `avogadro_number`: Avogadro number
* `real(kind=kind_phys)`: units = molecules mole-1
* `reference_air_pressure_for_atmosphere_vertical_coordinate`: Reference air pressure for atmosphere vertical coordinate
* `reference_surface_air_pressure_for_atmosphere_vertical_coordinate`: Reference surface air pressure for atmosphere vertical coordinate
* `real(kind=kind_phys)`: units = Pa
* `boltzmann_constant`: Boltzmann constant
* `real(kind=kind_phys)`: units = J K-1
Expand Down Expand Up @@ -81,7 +81,7 @@ Currently, the only dimension which supports all six dimension types is horizont
* `gravitational_acceleration`: Gravitational acceleration
* `real(kind=kind_phys)`: units = m s-2
* `cell_area`: Cell area
* `real(kind=kind_phys)`: units = steradian
* `real(kind=kind_phys)`: units = m2
* `cell_weight`: Cell weight
* `real(kind=kind_phys)`: units = none
## state_variables
Expand All @@ -98,7 +98,7 @@ Note that appending '_on_previous_timestep' to standard_names in this section yi
* `real(kind=kind_phys)`: units = Pa
* `surface_air_pressure`: Surface air pressure
* `real(kind=kind_phys)`: units = Pa
* `surface_dry_air_pressure`: Surface dry air pressure
* `surface_pressure_of_dry_air`: Surface pressure of dry air
* `real(kind=kind_phys)`: units = Pa
* `surface_geopotential`: Surface geopotential
* `real(kind=kind_phys)`: units = m2 s-2
Expand All @@ -118,19 +118,19 @@ Note that appending '_on_previous_timestep' to standard_names in this section yi
* `real(kind=kind_phys)`: units = Pa s-1
* `air_pressure`: Midpoint air pressure
* `real(kind=kind_phys)`: units = Pa
* `dry_air_pressure`: Dry midpoint pressure
* `air_pressure_of_dry_air`: Dry midpoint pressure
* `real(kind=kind_phys)`: units = Pa
* `air_pressure_thickness`: Air pressure thickness
* `real(kind=kind_phys)`: units = Pa
* `dry_air_pressure_thickness`: Dry air pressure thickness
* `air_pressure_thickness_of_dry_air`: Air pressure thickness of dry air
* `real(kind=kind_phys)`: units = Pa
* `reciprocal_of_air_pressure_thickness`: Reciprocal of air pressure thickness
* `real(kind=kind_phys)`: units = Pa-1
* `reciprocal_of_dry_air_pressure_thickness`: Reciprocal of dry air pressure thickness
* `reciprocal_of_air_pressure_thickness_of_dry_air`: Reciprocal of air pressure thickness of dry air
* `real(kind=kind_phys)`: units = Pa-1
* `ln_air_pressure`: Ln air pressure
* `real(kind=kind_phys)`: units = 1
* `ln_dry_air_pressure`: Ln dry air pressure
* `ln_air_pressure_of_dry_air`: Ln air pressure of dry air
* `real(kind=kind_phys)`: units = 1
* `reciprocal_of_dimensionless_exner_function_wrt_surface_air_pressure`: inverse exner function w.r.t. surface pressure, (ps/p)^(R/cp)
* `real(kind=kind_phys)`: units = 1
Expand All @@ -140,11 +140,11 @@ Note that appending '_on_previous_timestep' to standard_names in this section yi
* `real(kind=kind_phys)`: units = kg/kg moist or dry air depending on type
* `air_pressure_at_interface`: Air pressure at interface
* `real(kind=kind_phys)`: units = Pa
* `dry_air_pressure_at_interface`: Dry air pressure at interface
* `air_pressure_of_dry_air_at_interface`: Air pressure of dry air at interface
* `real(kind=kind_phys)`: units = Pa
* `ln_air_pressure_at_interface`: Ln air pressure at interface
* `real(kind=kind_phys)`: units = 1
* `ln_dry_air_pressure_at_interface`: Ln dry air pressure at interface
* `ln_air_pressure_of_dry_air_at_interface`: Ln air pressure of dry air at interface
* `real(kind=kind_phys)`: units = 1
* `largest_model_top_pressure_that_allows_molecular_diffusion`: Largest model top pressure that allows molecular diffusion
* `real(kind=kind_phys)`: units = Pa
Expand Down Expand Up @@ -1093,8 +1093,6 @@ Standard / required CCPP variables
* `real`: units = flag
* `number_of_timesteps_for_concurrent_radiation_and_remainder_physics_calls_after_model_initialization`: Number of timesteps for concurrent radiation and remainder physics calls after model initialization
* `integer`: units = count
* `gravitational_acceleration`: Gravitational acceleration
* `real(kind=kind_phys)`: units = m s-2
* `gas_constant_dry_air`: Gas constant dry air
* `real(kind=kind_phys)`: units = J kg-1 K-1
* `gas_constant_water_vapor`: Gas constant water vapor
Expand All @@ -1107,12 +1105,6 @@ Standard / required CCPP variables
* `real(kind=kind_phys)`: units = none
* `ratio_of_dry_air_to_water_vapor_gas_constants_minus_one`: Ratio of dry air to water vapor gas constants minus one
* `real(kind=kind_phys)`: units = none
* `tendency_of_y_wind_due_to_model_physics`: Tendency of y wind due to model physics
* `real(kind=kind_phys)`: units = m s-2
* `tendency_of_x_wind_due_to_model_physics`: Tendency of x wind due to model physics
* `real(kind=kind_phys)`: units = m s-2
* `tendency_of_air_temperature_due_to_model_physics`: Tendency of air temperature due to model physics
* `real(kind=kind_phys)`: units = K s-1
* `tendency_of_vertically_diffused_tracer_concentration`: Tendency of vertically diffused tracer concentration
* `real(kind=kind_phys)`: units = kg kg-1 s-1
* `vertically_diffused_tracer_concentration`: Vertically diffused tracer concentration
Expand Down Expand Up @@ -1301,8 +1293,6 @@ Standard / required CCPP variables
* `real(kind=kind_phys)`: units = frac
* `surface_albedo_diffuse_NIR`: Surface albedo diffuse NIR
* `real(kind=kind_phys)`: units = frac
* `area_type`: Area type
* `real(kind=kind_phys)`: units = flag
* `surface_stochastic_weights_from_coupled_process`: Surface stochastic weights from coupled process
* `real(kind=kind_phys)`: units = none
* `air_pressure_at_layer_for_RRTMGP_in_hPa`: Air pressure at layer for RRTMGP in hPa
Expand Down Expand Up @@ -2193,7 +2183,7 @@ Standard / required CCPP variables
* `real(kind=kind_phys)`: units = um
* `sppt_weights_from_coupled_process`: Sppt weights from coupled process
* `real(kind=kind_phys)`: units = none
* `total_ampltiude_of_sppt_perturbation`: Total ampltiude of sppt perturbation
* `total_amplitude_of_sppt_perturbation`: Total amplitude of sppt perturbation
* `real(kind=kind_phys)`: units = none
* `convective_cloud_water_mixing_ratio_in_xyz_dimensioned_restart_array`: Convective cloud water mixing ratio in xyz dimensioned restart array
* `real(kind=kind_phys)`: units = kg kg-1
Expand Down Expand Up @@ -2843,8 +2833,6 @@ Standard / required CCPP variables
* `real(kind=kind_phys)`: units = none
* `tendency_of_air_temperature_due_to_shortwave_heating_assuming_clear_sky_on_radiation_time_step_and_radiation_levels`: Tendency of air temperature due to shortwave heating assuming clear sky on radiation time step and radiation levels
* `real(kind=kind_phys)`: units = K s-1
* `air_temperature_on_previous_timestep`: Air temperature on previous timestep
* `real(kind=kind_phys)`: units = K
* `specific_humidity_on_previous_timestep`: Specific humidity on previous timestep
* `real(kind=kind_phys)`: units = kg kg-1
* `lwe_surface_snow_from_coupled_process`: Lwe surface snow from coupled process
Expand Down
85 changes: 85 additions & 0 deletions check_xml_unique.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
#!/usr/bin/env python

"""
Remove duplicates from a metadata standard-name XML library file.
"""

import argparse
import sys
import os.path
import xml.etree.ElementTree as ET
from xml_tools import read_xml_file
import copy

###############################################################################
def parse_command_line(args, description):
###############################################################################
parser = argparse.ArgumentParser(description=description,
formatter_class=argparse.RawTextHelpFormatter)

parser.add_argument("standard_name_file",
metavar='<standard names filename>',
type=str, help="XML file with standard name library")
parser.add_argument("--overwrite", action='store_true',
help="flag to remove duplicates and overwrite the file")

pargs = parser.parse_args(args)
return pargs

###############################################################################
def main_func():
###############################################################################
"""Parse the standard names database file and notify of duplicates.
"""
# Parse command line arguments
args = parse_command_line(sys.argv[1:], __doc__)
stdname_file = os.path.abspath(args.standard_name_file)
_, root = read_xml_file(stdname_file)

#get list of all standard names
all_std_names = []
for name in root.findall('./section/standard_name'):
all_std_names.append(name.attrib['name'])

#get list of all unique and duplicate standard names, in source order
seen = set()
uniq_std_names = []
dup_std_names = []
for x in all_std_names:
if x not in seen:
uniq_std_names.append(x)
seen.add(x)
else:
dup_std_names.append(x)

if args.overwrite:
#delete all duplicate elements after the first
if len(dup_std_names)>0:
print('The following duplicate standard names were found:')
for dup in dup_std_names:
rm_elements = root.findall('./section/standard_name[@name="%s"]'%dup)[1:]
print("{0}, ({1} duplicate(s))".format(dup, len(rm_elements)))
print('Removing duplicates and overwriting {}'.format(stdname_file))
for dup in dup_std_names:
rm_parents = root.findall('./section/standard_name[@name="%s"]...'%dup)[1:]
for par in rm_parents:
rm_ele = par.findall('./standard_name[@name="%s"]'%dup)
for ele in rm_ele:
par.remove(ele)

_.write(stdname_file, "utf-8")
else:
print('No duplicate standard names were found.')
else:
#write out duplicate standard names
if len(dup_std_names)>0:
print('The following duplicate standard names were found:')
for dup in dup_std_names:
rm_elements = root.findall('./section/standard_name[@name="%s"]'%dup)[1:]
print("{0}, ({1} duplicate(s))".format(dup, len(rm_elements)))
else:
print('No duplicate standard names were found.')

###############################################################################
if __name__ == "__main__":
main_func()
42 changes: 12 additions & 30 deletions standard_names.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@
long_name="number of vertical levels">
<type units="count">integer</type>
</standard_name>
<standard_name name="vertical_dimension_plus_one"
long_name="number of vertical levels + 1">
<standard_name name="vertical_interface_dimension"
long_name="number of vertical interfaces">
<type units="count">integer</type>
</standard_name>
<standard_name name="vertical_layer_index"
Expand Down Expand Up @@ -77,7 +77,7 @@
<standard_name name="avogadro_number">
<type kind="kind_phys" units="molecules mole-1">real</type>
</standard_name>
<standard_name name="reference_air_pressure_for_atmosphere_vertical_coordinate">
<standard_name name="reference_surface_air_pressure_for_atmosphere_vertical_coordinate">
<type kind="kind_phys" units="Pa">real</type>
</standard_name>
<standard_name name="boltzmann_constant">
Expand Down Expand Up @@ -125,7 +125,7 @@
<type kind="kind_phys" units="m s-2">real</type>
</standard_name>
<standard_name name="cell_area">
<type kind="kind_phys" units="steradian">real</type>
<type kind="kind_phys" units="m2">real</type>
</standard_name>
<standard_name name="cell_weight">
<type kind="kind_phys" units="none">real</type>
Expand All @@ -151,7 +151,7 @@
<standard_name name="surface_air_pressure">
<type kind="kind_phys" units="Pa">real</type>
</standard_name>
<standard_name name="surface_dry_air_pressure">
<standard_name name="surface_pressure_of_dry_air">
<type kind="kind_phys" units="Pa">real</type>
</standard_name>
<standard_name name="surface_geopotential">
Expand Down Expand Up @@ -187,26 +187,26 @@
long_name="Midpoint air pressure">
<type kind="kind_phys" units="Pa">real</type>
</standard_name>
<standard_name name="dry_air_pressure"
<standard_name name="air_pressure_of_dry_air"
long_name="Dry midpoint pressure">
<type kind="kind_phys" units="Pa">real</type>
</standard_name>
<standard_name name="air_pressure_thickness">
<type kind="kind_phys" units="Pa">real</type>
</standard_name>
<standard_name name="dry_air_pressure_thickness">
<standard_name name="air_pressure_thickness_of_dry_air">
<type kind="kind_phys" units="Pa">real</type>
</standard_name>
<standard_name name="reciprocal_of_air_pressure_thickness">
<type kind="kind_phys" units="Pa-1">real</type>
</standard_name>
<standard_name name="reciprocal_of_dry_air_pressure_thickness">
<standard_name name="reciprocal_of_air_pressure_thickness_of_dry_air">
<type kind="kind_phys" units="Pa-1">real</type>
</standard_name>
<standard_name name="ln_air_pressure">
<type kind="kind_phys" units="1">real</type>
</standard_name>
<standard_name name="ln_dry_air_pressure">
<standard_name name="ln_air_pressure_of_dry_air">
<type kind="kind_phys" units="1">real</type>
</standard_name>
<standard_name name="reciprocal_of_dimensionless_exner_function_wrt_surface_air_pressure"
Expand All @@ -222,13 +222,13 @@
<standard_name name="air_pressure_at_interface">
<type kind="kind_phys" units="Pa">real</type>
</standard_name>
<standard_name name="dry_air_pressure_at_interface">
<standard_name name="air_pressure_of_dry_air_at_interface">
<type kind="kind_phys" units="Pa">real</type>
</standard_name>
<standard_name name="ln_air_pressure_at_interface">
<type kind="kind_phys" units="1">real</type>
</standard_name>
<standard_name name="ln_dry_air_pressure_at_interface">
<standard_name name="ln_air_pressure_of_dry_air_at_interface">
<type kind="kind_phys" units="1">real</type>
</standard_name>
<standard_name name="largest_model_top_pressure_that_allows_molecular_diffusion">
Expand Down Expand Up @@ -1684,9 +1684,6 @@
<standard_name name="number_of_timesteps_for_concurrent_radiation_and_remainder_physics_calls_after_model_initialization">
<type units="count">integer</type>
</standard_name>
<standard_name name="gravitational_acceleration">
<type kind="kind_phys" units="m s-2">real</type>
</standard_name>
<standard_name name="gas_constant_dry_air">
<type kind="kind_phys" units="J kg-1 K-1">real</type>
</standard_name>
Expand All @@ -1705,15 +1702,6 @@
<standard_name name="ratio_of_dry_air_to_water_vapor_gas_constants_minus_one">
<type kind="kind_phys" units="none">real</type>
</standard_name>
<standard_name name="tendency_of_y_wind_due_to_model_physics">
<type kind="kind_phys" units="m s-2">real</type>
</standard_name>
<standard_name name="tendency_of_x_wind_due_to_model_physics">
<type kind="kind_phys" units="m s-2">real</type>
</standard_name>
<standard_name name="tendency_of_air_temperature_due_to_model_physics">
<type kind="kind_phys" units="K s-1">real</type>
</standard_name>
<standard_name name="tendency_of_vertically_diffused_tracer_concentration">
<type kind="kind_phys" units="kg kg-1 s-1">real</type>
</standard_name>
Expand Down Expand Up @@ -1996,9 +1984,6 @@
<standard_name name="surface_albedo_diffuse_NIR">
<type kind="kind_phys" units="frac">real</type>
</standard_name>
<standard_name name="area_type">
<type kind="kind_phys" units="flag">real</type>
</standard_name>
<standard_name name="surface_stochastic_weights_from_coupled_process">
<type kind="kind_phys" units="none">real</type>
</standard_name>
Expand Down Expand Up @@ -3334,7 +3319,7 @@
<standard_name name="sppt_weights_from_coupled_process">
<type kind="kind_phys" units="none">real</type>
</standard_name>
<standard_name name="total_ampltiude_of_sppt_perturbation">
<standard_name name="total_amplitude_of_sppt_perturbation">
<type kind="kind_phys" units="none">real</type>
</standard_name>
<standard_name name="convective_cloud_water_mixing_ratio_in_xyz_dimensioned_restart_array">
Expand Down Expand Up @@ -4309,9 +4294,6 @@
<standard_name name="tendency_of_air_temperature_due_to_shortwave_heating_assuming_clear_sky_on_radiation_time_step_and_radiation_levels">
<type kind="kind_phys" units="K s-1">real</type>
</standard_name>
<standard_name name="air_temperature_on_previous_timestep">
<type kind="kind_phys" units="K">real</type>
</standard_name>
<standard_name name="specific_humidity_on_previous_timestep">
<type kind="kind_phys" units="kg kg-1">real</type>
</standard_name>
Expand Down