Restructure _config.py

[stefsmeets]

Is your feature request related to a problem? Please describe.
Hi all, I would like to start a dicussion about reorganizing _config.py.

Currently, it feels like _config.py has gathered a bunch of functions with no other place to go. However, importing from _config causes a bunch of globals / objects to be created.

We think this may be unwanted, in case we are just interested in importing the logger, DIAGNOSTICS_PATH, or some of the utils functions (i.e. in view of the API we are developing). For better isolation, I would like to suggest we create a new _utils.py submodule for those functions that are not config related (of course they can still import from _config if needed).

Keep the main concern of _config.py to two functions:
read_config_user_file / read_config_developer_file

For the rest I propose:
_config.get_institutes -> _utils.get_institutes
_config.get_activity -> _utils.get_activity
_config._load_tags -> _utils._load_tags
_config.get_tag_value -> _utils.get_tag_value
_config.replace_tags -> _utils.replace_tags
_config.configure_logging -> _logging.configure_logging
This is useful if we want to import the logger without initializing the config.
_config.find_diagnostics -> _diagnostics.find_diagnostics
Defines DIAGNOSTICS_PATH, so it deserves its own submodule.

Would you be able to help out?
👍

[valeriupredoi]

This is useful if we want to import the logger without initializing the config.
_config.find_diagnostics -> _diagnostics.find_diagnostics
Defines DIAGNOSTICS_PATH, so it deserves its own submodule.

Yeah I like this! good idea for better clarity 👍

I would prob not transfer the others from _config to an _utils module tho, since they really are run configuration settings

[bouweandela]

For better isolation, I would like to suggest we create a new _utils.py submodule for those functions that are not config related

But these functions are configuration in almost all cases:
_config.get_institutes -> reads institutes from config-developer.yml
_config._load_tags -> reads tags from config-references.yml
_config.get_tag_value -> reads tags from config-references.yml
_config.replace_tags -> reads tags from config-references.yml
_config.configure_logging -> set up logging configuration
_config.find_diagnostics -> sets up the path to the diagnostics, indeed not yet configurable, but I think it would be good to be able to set DIAGNOSTICS_PATH in config-user.yml in the future

Only _config.get_activity does not read it's information from config-developer.yml, but from the CMIP6 CMOR table.