Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] key collision when using option@_global_ in defaults list #1784

Closed
Jasha10 opened this issue Aug 21, 2021 · 3 comments · Fixed by #1791
Closed

[Bug] key collision when using option@_global_ in defaults list #1784

Jasha10 opened this issue Aug 21, 2021 · 3 comments · Fixed by #1791
Assignees
Labels
bug Something isn't working In progress Work in progress

Comments

@Jasha10
Copy link
Collaborator

Jasha10 commented Aug 21, 2021

When a defaults list contains entries for both option and option@_global_, there is a key collision resulting in ConfigCompositionException.

Consider the following setup:

$ tree
.
├── conf
│   ├── config.yaml
│   └── db
│       ├── mysql.yaml
│       └── sqlite.yaml
└── my_app.py

2 directories, 4 files
# my_app.py
from omegaconf import OmegaConf
import hydra

@hydra.main(config_path="conf", config_name="config")
def my_app(cfg) -> None:
    print(OmegaConf.to_yaml(cfg))

my_app()
# conf/db/mysql.yaml
name: mysql
# conf/db/sqlite.yaml
name: sqlite

If config.yaml has a defaults list with just one entry, db: mysql, the result is as expected:

$ cat conf/config.yaml
defaults:
  - db: mysql

$ python my_app.py
db:
  name: mysql

If config.yaml has a defaults list with just one entry, db@_global_: sqlite, the result is as expected:

$ cat conf/config.yaml
defaults:
  - db@_global_: sqlite

$ python my_app.py
name: sqlite

If config.yaml has a defaults list with both of these entries, there is an unexpected key collision:

$ cat conf/config.yaml
defaults:
  - db: mysql
  - db@_global_: sqlite

$ python my_app.py
Multiple values for db. To override a value use 'override db: sqlite'

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

The expected output in this scenario would be the following:

name: sqlite
db:
  name: mysql
@Jasha10 Jasha10 added the bug Something isn't working label Aug 21, 2021
@Jasha10
Copy link
Collaborator Author

Jasha10 commented Aug 21, 2021

This issue comes from the InputDefault.get_override_key method:

    def get_override_key(self) -> str:
        default_pkg = self.get_default_package()
        final_pkg = self.get_final_package(default_to_package_header=False)
        key = self.get_group_path()
        if default_pkg != final_pkg and final_pkg != "":
            key = f"{key}@{final_pkg}"
        return key

Key collision occurs if the defaults list has two entries, where one entry satisfies default_pkg == final_pkg and the other entry satisfies final_pkg == "".

@omry
Copy link
Collaborator

omry commented Aug 24, 2021

There is some ambiguity here. This could be tricky to fix.

@Jasha10 Jasha10 self-assigned this Aug 25, 2021
@Jasha10 Jasha10 added the In progress Work in progress label Aug 25, 2021
@Jasha10
Copy link
Collaborator Author

Jasha10 commented Aug 25, 2021

I'm having success with the following diff:

     def get_override_key(self) -> str:
         default_pkg = self.get_default_package()
         final_pkg = self.get_final_package(default_to_package_header=False)
         key = self.get_group_path()
-        if default_pkg != final_pkg and final_pkg != "":
+        if default_pkg != final_pkg:
+            if final_pkg == "":
+                final_pkg = "_global_"
+
             key = f"{key}@{final_pkg}"
         return key

Will submit a PR shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working In progress Work in progress
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants