Possibility to factorize configuration for multiple project

[pvnieo]

🚀 Feature Request

Hi,

I have the following use case. I am working on a large project with multiple subtasks. For example, we are performing classification on some datasets, segmentation on others, and translation on yet another set of datasets (see below).

Each task has its own configuration. However, there is a part of the configuration that is shared among all tasks (such as the architecture of some network or specific blocks). This shared configuration is evolving, and each time I make a small change to it, I need to copy and paste the updated configuration into all the relevant folders.

Is there a way to centralize this configuration so that it can be imported into all the subtasks?

my_project
| classification_task
|--| conf
|--|--| config.yaml
|--|--| models
|--|--| datasets
|--| train.py
|--| ...
| segmentation_task
|--| conf
|--|--| config.yaml
|--|--| models
|--|--| datasets
|--| train.py
|--| ...
| translation_task
|--| conf
|--|--| config.yaml
|--|--| models
|--|--| datasets
|--| train.py
|--| ...

[omry]

Different config roots should be addressed by adding them to the config search paths.
There are several ways to do it, see Config Search Path for more info.

Regarding reusing a config object in the way you describe:
See Extending Configs for context on how to achieve that with config files.
You can also achieve it with Structured Configs (check the tutorial).