[Bug] Write run_results.json even if we encounter an exception on main thread

[aranke]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

run_results.json is not present if an exception is encountered on the main thread; this is a problem because we run hooks on the main thread

Exception is caught here:

dbt-core/core/dbt/cli/requires.py

Line 158 in 359a2c0

fire_event(MainEncounteredError(exc=str(e)))

Expected Behavior

run_results.json should be written out regardless of error on main thread.

Steps To Reproduce

❯ cat models/my_model.sql 
select 1 as id

❯ cat macros/rce.sql 
{% macro rce(relation) %}

{% if execute %}

{{ exceptions.raise_compiler_error("Always raise a compiler error in execute") }}

{% endif %}

{% endmacro %}

❯ cat dbt_project.yml
...
on-run-end:
  - "{{ rce() }}"
...

❯ dbt clean && dbt run
...
16:27:47  Finished running 3 table models, 3 view models in 0 hours 0 minutes and 0.25 seconds (0.25s).
16:27:47  Encountered an error:
Compilation Error in operation jaffle_shop-on-run-end-0 (./dbt_project.yml)
  Always raise a compiler error in execute
  
  > in macro rce (macros/rce.sql)
  > called by operation jaffle_shop-on-run-end-0 (./dbt_project.yml)

❯ ls -l target/
total 2000
drwxr-xr-x  3 karanke  staff      96 18 Nov 16:27 compiled/
-rw-r--r--  1 karanke  staff   27828 18 Nov 16:27 graph.gpickle
-rw-r--r--  1 karanke  staff    3079 18 Nov 16:27 graph_summary.json
-rw-r--r--  1 karanke  staff  493456 18 Nov 16:27 manifest.json
-rw-r--r--  1 karanke  staff  490411 18 Nov 16:27 partial_parse.msgpack
drwxr-xr-x  3 karanke  staff      96 18 Nov 16:27 run/
-rw-r--r--  1 karanke  staff     253 18 Nov 16:27 semantic_manifest.json

❯ cat target/run_results.json
cat: target/run_results.json: No such file or directory

Relevant log output

No response

Environment

- OS: MacOS Sequoia 15.1
- Python: 3.12
- dbt: 1.9.0-b4

Which database adapter are you using with dbt?

snowflake

Additional Context

We have similar code for handling exceptions on runner threads:

dbt-core/core/dbt/task/run.py

Lines 805 to 813 in 89caa33

	except (KeyboardInterrupt, SystemExit):
	run_result = self.get_result(
	results=self.node_results,
	elapsed_time=time.time() - self.started_at,
	generated_at=datetime.utcnow(),
	)
	if self.args.write_json and hasattr(run_result, "write"):
	run_result.write(self.result_path())

The scope of this issue is to port this work to the main thread for hooks.