[Bug] Disabled models are logged as "does not exist"

[randypitcherii]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

When performing a dbt run -s my_model when my_model is configured with enabled=False, dbt reports that the model doesn't exist rather than indicating it is disabled.

Expected Behavior

The expected behavior is for dbt to differentiate error messages between missing models and disabled models.

Steps To Reproduce

1. create a simple model
2. configure that model with enabled=False
3. run dbt run -s my_model
4. observe that dbt indicates that model does not exist

Relevant log output

No response

Environment

- dbt cloud
- dbt: 0.21.0

What database are you using dbt with?

snowflake

Additional Context

No response

[jtcohen6]

@randypitcherii Thanks for opening!

We do raise different error messages, during parsing, if you try to ref() a disabled vs. nonexistent model:

Compilation Error in model my_view (models/my_view.sql)
  Model 'model.testy.my_view' (models/my_view.sql) depends on a node named 'my_disabled_table' which is disabled

Compilation Error in model my_view (models/my_view.sql)
  Model 'model.testy.my_view' (models/my_view.sql) depends on a node named 'my_nonexistent_table' which was not found

This distinction would be much trickier to make during node selection. We've already constructed the DAG, and left disabled nodes out of it; we select nodes from that DAG; there's no good way to tell whether a node is missing from the DAG because it's been disabled, or because it never existed in the first place.

So, we try to raise a more helpful warning that indicates both possibilities. Here's the message I see when I select a model that is missing from the DAG, either because it's disabled or because it doesn't exist:

$ dbt run -s my_view
Running with dbt=0.21.0
Found 1 model, 0 tests, 0 snapshots, 0 analyses, 165 macros, 0 operations, 0 seed files, 0 sources, 0 exposures
The selection criterion 'my_view' does not match any nodes

WARNING: Nothing to do. Try checking your model configs and model specification args

The intent behind the inclusion of "try checking your model configs" is to indicate that a model might be configured with enabled: false. Perhaps we could make those messages clearer, by including the word enabled explicitly?

The selection criterion ... does not match any enabled nodes
WARNING: Nothing to do. Try checking your model configs (enabled) and model specification args

Note that this is a "warn"-level log event, meaning that it will raise an error in "warning-as-errors" mode:

$ dbt --warn-error run -s my_view
Running with dbt=0.21.1
Found 1 model, 0 tests, 0 snapshots, 0 analyses, 165 macros, 0 operations, 0 seed files, 0 sources, 0 exposures
Encountered an error:
Compilation Error
  The selection criterion 'my_view' does not match any nodes

[jeremyyeo]

@jtcohen6 is there any appetite for adding some sort of --error-if-selection-criterion-has-no-matches flag OR adding some config that allows a user to choose what specific warning to treat as errors instead of treating all warnings as errors via the --warn-error flag?

The use case was raised by a customer where they have multiple jobs that they orchestrate outside of dbt Cloud (lets say there's a job A that runs and then upon success, a job B is triggered). Now someone might accidentally have a typo of the model name in job A (e.g. dbt run --select this_doesnt_exist) and because an error is not tripped (warning results in a successful dbt Cloud job run), job B will be executed by the orchestration tool (which may error if it depends on a model in job A).

In the context of a single dbt run command, a selection criterion that doesn't match any nodes may be warn-level worthy but if we zoom out to multiple dbt run commands in various orchestration jobs - we may want to treat this as error-level worthy instead (makes it easier to catch / debug).

---

Here are some of the things I'm thinking about of how to achieve the above without requiring any change to dbt-core or dbt Cloud.

1. Use --warn-error flag.

   There's an undesirable side effect here that an error will be raised for all other warning level events too (data-paths is now source-paths, etc).

2. Try parsing some on-run-end results context.

   In my testing, if there are no nodes that are selected and thus nothing is "run", the on-run-end doesn't actually trigger. My hopes was that we could intercept the results object's msg variable during the on-run-end and perhaps raise_compiler_error if the msg contains the string "The selection criterion 'this_doesnt_exist' does not match any nodes".

3. Make use of run-operations and on-run-end hook.

   At this stage, I'm truly stretching it by having a couple of run-operation macros that run before and after a dbt run command. See this gist for some true shenanigans 😵‍💫.

[jtcohen6]

@jeremyyeo Thanks for sharing that context!

OR adding some config that allows a user to choose what specific warning to treat as errors instead of treating all warnings as errors via the --warn-error flag?

I think this is the thing. I don't think we should try adding separate flags for each individual warning. I do think the pattern here should look like:

• Users / dbt Cloud should parse structured logs produced by dbt-core, e.g. to return all warn-level messages in a run summary (even for successful runs)
• An external system / observability tool can check the event codes for each structured log line, and match them up against specific criteria. (This is much more reliable than checking the message strings for those events, which are intended for human, not machine, consumption.)

Such a system isn't exactly possible today (I'll explain why below) — but does that at least sound like the right direction, to achieve the sort of fine-grained control we're looking for?

---

Ok, here's the catch: Unfortunately, the --warn-error capability here is also our limitation today. For v1.0, we converted all log sites to structured logging, but we still haven't converted exceptions to use the new logging/eventing system. As it is, warn_or_error is implemented as an exception (if --warn-error) that otherwise calls out to a generic log event:

dbt-core/core/dbt/exceptions.py

Lines 1011 to 1015 in 8f50eee

	def warn_or_error(msg, node=None, log_fmt=None):
	if flags.WARN_ERROR:
	raise_compiler_error(scrub_secrets(msg, env_secrets()), node)
	else:
	fire_event(GeneralWarningMsg(msg=msg, log_fmt=log_fmt))

That is, all log events fed through warn_or_error return GeneralWarningMsg with code Z046. I think next steps here would look like:

• Rework exceptions to flow through the structured logging framework, wherein WarnOrError can be a new base type, and its conditional behavior powered by the centralized logger
• Create unique event types for each existing warn_or_error call site

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest; add a comment to notify the maintainers.