Here I present the examples that I have tested so far.
- Override config parameters from command line(ex_01)
- Config files are
yaml
files with.yaml
extension(ex_01) - saves the modified config in
output
directory(ex_01) - @hydra.main(version_base=None, config_path=".", config_name="config")
4)
config
is the name of theconfig.yaml
. It must be inconfig_path
dir. - Using
++
to override and existing item(ex_01) - switch between config files(default_config)
- multi-run. Running multiple (default_config, config_store_example)
???
must field in the future object type(ex_03)- different access types to features within code is given in ex_03
- You can remove a default entry from the defaults list by prefixing it with
~
- Get the exact command to install the completion from --hydra-help. Currently, Bash, zsh and Fish are supported. We are relying on the community to implement tab completion plugins for additional shells.
- Structure Configs: 10) Union types are only partially supported (see OmegaConf docs on unions) 11)
cd example_00
python my_app.py +db.driver=mysql +db.user=omry +db.password=secret
Output:
db:
driver: mysql
user: omry
password: secret
Notes:
- The
+
indicates that the field is new. - DictConfig is an empty object!
cd example_01
python my_app.py
output:
beautiful yaml format:
db:
driver: mysql
user: pooya
pass: secret
dict form:
{'db': {'driver': 'mysql', 'user': 'pooya', 'pass': 'secret'}}
python my_app.py db.user=root db.pass=1234
output:
beautiful yaml format:
db:
driver: mysql
user: root
pass: 1234
dict form:
{'db': {'driver': 'mysql', 'user': 'root', 'pass': 1234}}
using ++
:
python my_app.py ++db.user=pooya_mohammadi
Output:
db:
driver: mysql
user: pooya_mohammadi
pass: secret
Note: No need for +
because the params are already defined in config.yaml
In this case, the config name must be specified, so that the code can read the default values!
The defaults are ordered:
If multiple configs define the same value, the last one wins.
If multiple configs contribute to the same dictionary, the result is the combined dictionary.
cd default_config
python my_app.py db=postgresql db.timeout=10
output:
beautiful yaml format:
db:
driver: postgresql
pass: drowssap
timeout: 10
user: postgres_user
dict form:
{'db': {'driver': 'postgresql', 'pass': 'drowssap', 'timeout': 10, 'user': 'postgres_user'}}
cd default_config
python my_app.py --multirun db=mysql,postgresql
output:
[2022-11-08 11:39:59,931][HYDRA] Launching 2 jobs locally
[2022-11-08 11:39:59,931][HYDRA] #0 : db=mysql
beautiful yaml format:
db:
driver: mysql
pass: drowssap
timeout: 10
user: mysql_user
dict form:
{'db': {'driver': 'mysql', 'pass': 'drowssap', 'timeout': 10, 'user': 'mysql_user'}}
[2022-11-08 11:39:59,973][HYDRA] #1 : db=postgresql
beautiful yaml format:
db:
driver: postgresql
pass: drowssap
timeout: 20
user: postgres_user
dict form:
{'db': {'driver': 'postgresql', 'pass': 'drowssap', 'timeout': 20, 'user': 'postgres_user'}}
cd example_03
python main.py
Output:
Traceback (most recent call last):
File "my_app.py", line 32, in my_app
cfg.node.waldo
omegaconf.errors.MissingMandatoryValue: Missing mandatory value: node.waldo
full_key: node.waldo
object_type=dict
cd config_group
python my_app.py
output:
{}
It's empty because config_name is not defined in the my_app
file.
python my_app.py +db=postgresql
output:
db:
driver: postgresql
user: postgres_user
password: drowssap
timeout: 10
Overwrite:
python my_app.py +db=postgresql db.timeout=20
Output:
$ python my_app.py +db=postgresql db.timeout=20
db:
driver: postgresql
pass: drowssap
timeout: 20
user: postgres_user
ConfigStore is a singleton storing configs in memory. Instead of creating yaml files, one can create a dataclass and pass it to a ConfigStore
It covers every aspect that is possible with yaml
files configurations.
class ConfigStore(metaclass=Singleton):
def store(
self,
name: str,
node: Any,
group: Optional[str] = None,
package: Optional[str] = "_group_",
provider: Optional[str] = None,
) -> None:
"""
Stores a config node into the repository
:param name: config name
:param node: config node, can be DictConfig, ListConfig,
Structured configs and even dict and list
:param group: config group, subgroup separator is '/',
for example hydra/launcher
:param package: Config node parent hierarchy.
Child separator is '.', for example foo.bar.baz
:param provider: the name of the module/app providing this config.
Helps debugging.
"""
...
tests:
python my_app.py +db=mysql db.user=pooya
output:
db:
driver: mysql
user: pooya
password: secret
Because we didn't define any default list for this example like the others. Therefore, the + will add this option to default list itself.
The +
indicates that the field is new!
Note: default list for other examples are in config.yaml
!
python my_app.py --multirun +db=mysql,postgresql ++db.user=Ali,Hassan
Output
[2022-11-08 17:18:44,168][HYDRA] Launching 4 jobs locally
[2022-11-08 17:18:44,168][HYDRA] #0 : +db=mysql ++db.user=Ali
db:
driver: mysql
user: Ali
password: secret
[2022-11-08 17:18:44,209][HYDRA] #1 : +db=mysql ++db.user=Hassan
db:
driver: mysql
user: Hassan
password: secret
[2022-11-08 17:18:44,253][HYDRA] #2 : +db=postgresql ++db.user=Ali
db:
driver: postgresql
user: Ali
password: '*****'
[2022-11-08 17:18:44,294][HYDRA] #3 : +db=postgresql ++db.user=Hassan
db:
driver: postgresql
user: Hassan
password: '*****'
At the registration time, the values of the config class can be modified:
from dataclasses import dataclass
from hydra.core.config_store import ConfigStore
@dataclass
class MySQLConfig:
host: str = "localhost"
port: int = 3306
cs = ConfigStore.instance()
# Using the type
cs.store(name="config1", node=MySQLConfig)
# Using an instance, overriding some default values
cs.store(name="config2", node=MySQLConfig(host="test.db", port=3307))
# Using a dictionary, forfeiting runtime type safety
cs.store(name="config3", node={"host": "localhost", "port": 3308})
Hierarchical configuration is possible with hydra
Examples:
python main.py db.port=10 # tab completion works till the features!
db:
host: localhost
port: 10
ui:
title: My app
width: 1024
height: 768
different group of configs can be created with in config store
@dataclass
class MySQLConfig:
...
@dataclass
class PostGreSQLConfig:
...
@dataclass
class Config:
# We will populate db using composition.
db: Any
# Create config group `db` with options 'mysql' and 'postgreqsl'
cs = ConfigStore.instance()
cs.store(name="config", node=Config)
cs.store(group="db", name="mysql", node=MySQLConfig)
cs.store(group="db", name="postgresql", node=PostGreSQLConfig)
@hydra.main(version_base=None, config_name="config")
def my_app(cfg: Config) -> None:
print(OmegaConf.to_yaml(cfg))
Here db
has two options which none of them is default!
cd config_group_configstore
python main.py
Output:
db: ???
Select a db:
python main.py +db=postgresql
Output:
python main.py +db=postgresql
db:
driver: postgresql
host: localhost
port: 5432
timeout: 10
Note: The +
is required because no default value for db is defined.
Using the following code, db
can be initialized with a value:
@dataclass
class DBConfig:
host: str = "localhost"
port: int = MISSING
driver: str = MISSING
@dataclass
class Config:
db: DBConfig
Note:
- Assign MISSING to a field to indicates that it does not have a default value. This is equivalent to the ??? literal we have seen in OmegaConf configs before.
- Omitting a default value is equivalent to assigning MISSING to it, although it is sometimes convenient to be able to assign MISSING it to a field.