Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-722] DuplicateYamlKeyException needs to be warning, not (broken) exception #5331

Closed
jtcohen6 opened this issue Jun 3, 2022 · 0 comments · Fixed by #5403
Closed

[CT-722] DuplicateYamlKeyException needs to be warning, not (broken) exception #5331

jtcohen6 opened this issue Jun 3, 2022 · 0 comments · Fixed by #5403
Labels
bug Something isn't working
Milestone

Comments

@jtcohen6
Copy link
Contributor

jtcohen6 commented Jun 3, 2022

Net-new bug in v1.2. See also #5268, which I believe is caused by the same root change (#5146).

def load_yaml_text(contents, path=None):
try:
return safe_load(contents)
except (yaml.scanner.ScannerError, yaml.YAMLError) as e:
if hasattr(e, "problem_mark"):
error = contextualized_yaml_error(contents, e)
else:
error = str(e)
raise dbt.exceptions.ValidationException(error)
except dbt.exceptions.DuplicateYamlKeyException as e:
# TODO: We may want to raise an exception instead of a warning in the future.
e.msg = f"{e} {path.searched_path}/{path.relative_path}."
dbt.exceptions.warn_or_raise(e, log_fmt=warning_tag("{}"))

If path=None, it has no searched_path / relative_path. If I change that to

Reproduction case

# dbt_project.yml
vars:
  one_var: foo
  one_var: bar
$ dbt parse
17:20:39  Encountered an error:
'NoneType' object has no attribute 'searched_path'

If I change the code linked above to:

     except dbt.exceptions.DuplicateYamlKeyException as e: 
         # TODO: We may want to raise an exception instead of a warning in the future. 
         if path:
             e.msg = f"{e} {path.searched_path}/{path.relative_path}." 
         dbt.exceptions.warn_or_raise(e, log_fmt=warning_tag("{}")) 
$ dbt parse
17:21:36  [WARNING]: Compilation Error
  Duplicate 'one_var' key found in yaml file
17:21:36  Encountered an error while reading the project:
17:21:36    ERROR: Runtime Error
  dbt_project.yml does not parse to a dictionary
17:21:36  Encountered an error:
Runtime Error
  Could not run dbt

This is still a problem. This parses successfully in v1.1. We MUST make this a warning, not an error.

We can do that by using SafeLoader instead of UniqueKeyLoader, and still returning the loaded result (if not --warn-error):

    except dbt.exceptions.DuplicateYamlKeyException as e:
        if path:
            # TODO: We may want to raise an exception instead of a warning in the future.
            e.msg = f"{e} {path.searched_path}/{path.relative_path}."
        dbt.exceptions.warn_or_raise(e, log_fmt=warning_tag("{}"))
        return yaml.load(contents, Loader=SafeLoader)
@jtcohen6 jtcohen6 added bug Something isn't working triage Team:Language labels Jun 3, 2022
@github-actions github-actions bot changed the title DuplicateYamlKeyException needs to be warning, not (broken) exception [CT-722] DuplicateYamlKeyException needs to be warning, not (broken) exception Jun 3, 2022
@leahwicz leahwicz added this to the v1.2 milestone Jun 3, 2022
@jtcohen6 jtcohen6 removed the triage label Jun 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants