Incremental updates of tables

[sugendran]

With the delta_scan extension. Is there a way to do an incremental update of the data in motherduck? Or is the only option to CREATE OR REPLACE the table?

My theory is to follow this approach https://www.databricks.com/blog/delivering-cost-effective-data-real-time-dbt-and-databricks

That would then mean the delta table history has a record of which rows changed. So if we have a checkpoint somewhere then we know where in the history data to start reading from, and select the rows for a merge statement. Ideally this is done by the delta driver/extension rather than me having to write custom code to do this.