New ModelComponent class

[hboisgon]

Discussed in #636

^{Originally posted by DirkEilander November 7, 2023}

Current implementation

Currently model components, such as maps, geoms, grid, mesh, etc. are defined as properties of the Model class or extensions such as the GridMixin or MeshMixin. All these properties have their own associated read_, write_, and set_ methods and in some case some additional generic setup_ methods. This is perhaps too restrictive and requires quite some duplication of code.

It is restrictive in the sense that all models must use this terminology and squeeze the model data into these objects. In practice however, models may have multiple of the same components (e.g. SFINCS has a computational grid and subgrid) or model specific components are more meaningful to users (e.g. exposure and vulnerability model component classes in FIAT rather than geoms and tables).

The current approach also requires duplication of code as some components, e.g. maps, forcing, and states are basically the same component (all dict of xarray objects) and share a lot of the code which is now partly circumvented by calling methods defined in the workflows.

Discussed implementation

• Create common ModelComponent template class with abstract read, write, and set methods (and perhaps more) and a dynamic data property (not abstract) which reads the data when called for the first time (similar to e.g. current GridModel.grid property). Additonally, we should make subclasses for Grid, Mesh, Geoms, Maps, Table, etc. The generic setup_* methods should also be implemented here.
• These model components will be implemented as instance attributes of subclasses of Model (e.g. in plugins) and should be registered in a separate class attribute model_components that can be used in the Model.read and Model.write classes to loop over and read/write all components. This would defy the need for the GridMixin etc.
• Where needed plugins can subclass a ModelComponent (sub)class to replace methods with their own methods and add more.
• Syncing data between Model and modelComponent is done using weakref library.
• This also potentially solves the naming discussion in Rename the generic setup method to distinguish better between the setup_schematisation and add data methods #475. Instead of GridModel.setup_grid_data_from_raster it would GridModel.grid.add_data_from_raster, etc.
• We should be able to call these model component methods from the yaml hydromt configuration files, see e.g. below. This requires a small change in the Model._run_log_method
• We should review the Model class and remove hidden/non clear behavior (eg Model._FOLDERS used in Model.set_root)
• Note that there is a separate issue discussing the ModelRoot class new ModelRoot class #728

grid.create:
  arg: value

Example code implementation

#%% test ModelComponent class 
import weakref
from pathlib import Path

class ModelComponent:
    def __init__(self, model):
        self._model_ref = weakref.ref(model)
        self._data = None

    @property
    def model(self):
        # Access the Model instance through the weak reference
        return self._model_ref()  

    @property
    def data(self):
        return self._data
    
    def set(self, value):
        self._data = value

class ModelRoot:
    def __init__(self, path, mode='r'):
        self.set(path, mode)

    def set(self, path, mode='r'):
        self.path = Path(path)
        self.mode = mode

    def __repr__(self):
        return f"ModelRoot(path={self.path}, mode={self.mode})"

class Model:
    components = {'grid': ModelComponent, 'forcing': ModelComponent}

    def __init__(self, root, mode='r', data_libs=[]):
        self.root = ModelRoot(root, mode)

        # initialize components with linked model instance
        for name, component in self.components.items():
            setattr(self, name, component(model=self))

    def __repr__(self):
        return f"Model(root={self.root})"

# access model property from component
mod = Model('test', mode='r')
mod.root.set('test2', mode='w')
assert mod.root == mod.grid.model.root
print(mod.grid.model.root)
# >>> ModelRoot(path=test2, mode=w)

## acces one component from another
mod.grid.model.forcing.set('test')
print('forcing data:', mod.forcing.data)
# >>> forcing data: test

# infinite instance.attribute.instance.attribute stack .. ?
print(mod.grid.model.forcing.model.grid.model.forcing.model)
# >>> Model(root=ModelRoot(path=test2, mode=w))

[savente93]

How will this impact #432? I wonder if they are related. If so, we should consider them jointly and solve the underlying issue.

[DirkEilander]

How will this impact #432? I wonder if they are related. If so, we should consider them jointly and solve the underlying issue.

I think this can be considered separately if the DataCatalog interface will remain largely unchanged, which I think should be the case.

[DirkEilander]

In preparation of a design session I made these diagrams to help our understanding of the current and potential (up for discussion) v1 design.

v0.X

v1.0 suggestion

[deltamarnix]

@DirkEilander What would be the advantage of having a class hierarchy for models? To me it sounds like a plugin developer wants to inherit from a specific component for their plugin. Are all the subclasses of Model searched for in HydroMT, or are there other reasons to inherit from the Model class?

[DirkEilander]

Some plugin specific methods are implemented at the Model rather than the Component. Also the PluginModel is in the current setup exposed the the core through entrypoints. We have a meeting on Thu afternoon to discuss this further. You're welcome to join