Skip to content

Commit

Permalink
Load performance improvement (ignoring UGRID) (SciTools#6088)
Browse files Browse the repository at this point in the history
* proof of concept load performance improvement

* remove print line

* adjust for merge

* adjust to tolerant load behaviour, make methods private

* fix tests

* simplify changes

* Update lib/iris/fileformats/cf.py

Co-authored-by: Bill Little <bill.little@metoffice.gov.uk>

* address review comment

* add whatsnew

---------

Co-authored-by: Bill Little <bill.little@metoffice.gov.uk>
  • Loading branch information
stephenworsley and bjlittle authored Jul 29, 2024
1 parent 83905e9 commit 48f1f4e
Show file tree
Hide file tree
Showing 3 changed files with 35 additions and 5 deletions.
4 changes: 4 additions & 0 deletions docs/src/whatsnew/latest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,10 @@ This document explains the changes made to Iris for this release
subclasses of a common generic :class:`~iris.mesh.components.Mesh` class.
(:issue:`6057`, :pull:`6061`, :pull:`6077`)

#. `@pp-mo`_ and `@stephenworsley`_ Turned on UGRID loading by default, effectively removing
the need for and deprecating the :func:`~iris.ugrid.experimental.PARSE_UGRID_ON_LOAD`
context manager. (:pull:`6054`, :pull:`6088`)


🚀 Performance Enhancements
===========================
Expand Down
24 changes: 24 additions & 0 deletions lib/iris/fileformats/cf.py
Original file line number Diff line number Diff line change
Expand Up @@ -1331,6 +1331,11 @@ def __init__(self, file_source, warn=False, monotonic=False):

self._check_monotonic = monotonic

self._with_ugrid = True
if not self._has_meshes():
self._trim_ugrid_variable_types()
self._with_ugrid = False

self._translate()
self._build_cf_groups()
self._reset()
Expand All @@ -1348,6 +1353,25 @@ def __exit__(self, exc_type, exc_value, traceback):
# When used as a context-manager, **always** close the file on exit.
self._close()

def _has_meshes(self):
result = False
for variable in self._dataset.variables.values():
if hasattr(variable, "mesh") or hasattr(variable, "node_coordinates"):
result = True
break
return result

def _trim_ugrid_variable_types(self):
self._variable_types = (
CFAncillaryDataVariable,
CFAuxiliaryCoordinateVariable,
CFBoundaryVariable,
CFClimatologyVariable,
CFGridMappingVariable,
CFLabelVariable,
CFMeasureVariable,
)

@property
def filename(self):
"""The file that the CFReader is reading."""
Expand Down
12 changes: 7 additions & 5 deletions lib/iris/fileformats/netcdf/ugrid_load.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,11 +56,13 @@ def _meshes_from_cf(cf_reader):
# Mesh instances are shared between file phenomena.
# TODO: more sophisticated Mesh sharing between files.
# TODO: access external Mesh cache?
mesh_vars = cf_reader.cf_group.meshes
meshes = {
name: _build_mesh(cf_reader, var, cf_reader.filename)
for name, var in mesh_vars.items()
}
meshes = {}
if cf_reader._with_ugrid:
mesh_vars = cf_reader.cf_group.meshes
meshes = {
name: _build_mesh(cf_reader, var, cf_reader.filename)
for name, var in mesh_vars.items()
}
return meshes


Expand Down

0 comments on commit 48f1f4e

Please sign in to comment.