[CT-676] Set default table type to parquet (instead of implicit default text)

[iknox-fa]

Per discussion: https://dbt-labs.slack.com/archives/C017GDLAF7D/p1652996315344739 it may make sense to set the default table type to delta.

[jtcohen6]

Probably parquet, given that not everyone uses dbt-spark with access to the delta file format. We actually have code already that's meant to do this, but I suspect that "adapter-specific configs" are not being resolved correctly in dbt-core:

dbt-spark/dbt/adapters/spark/impl.py

Line 38 in 8744cf1

file_format: str = "parquet"

In the meantime, we could set that default here instead:

dbt-spark/dbt/include/spark/macros/adapters.sql

Line 6 in 8744cf1

{%- set file_format = config.get('file_format', validator=validation.any[basestring]) -%}

The default file format in the dbt-databricks plugin is indeed set to delta, since users are guaranteed access to it in the Databricks Runtime, and it gets them some extra functionality.

[lostmygithubaccount]

@iknox-fa @jtcohen6 anything further here? should we keep this issue open? remove the triage label and move to the backlog? something else?

[jtcohen6]

Let's:

• Set the default to parquet, by changing the config.get logic in spark__file_format_clause
• Make sure we have a ticket to track the fact that AdapterSpecific configs don't really work as intended. Most recently captured in [CT-634] Node configs: tech debt dbt-core#5236

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.