[SNOW-172] Convert file_latest to dynamic table

[danlu1]

Problem:

Currently, the file_latest table is a table that is updated via upserting task and remove deleted records task that are based on filesnapshots stream. We want to convert it to dynamic table so it gets refreshed automatically.

Solution:
Following the instructions of this SOP:

• Add a script to create a backup of the original file_latest table which need to be removed manually after validation
• Add a script to delete the upserting task and deletion task for the original file_latest table
• Add a script to delete stream for filesnapshots table that is referred in file_latest upserting and deletion task
• Add a script to drop the original file_latest table so we can create a dynamic table with the same name
• Create file_latest dynamic table and filter out records that are non-preview or status as available.
• Add a script to add table and column comments.
• Explicitly SELECT the columns when creating dynamic table to avoid errors if upstream table changes. See [SNOW-222] Reintroduce node_latest dynamic table #119 for more info.

[danlu1]

Thanks guys for the feedback. @philerooski I keep the interval as 30 days here even though we don't filter out CHANGE_TYPE != DELETE. As what we discussed and archived here, for tables having CHANGE_TYPE with 'DELETE', we would use 30 days instead of 14 days.

[thomasyu888]

🔥 LGTM! tagging @philerooski for final review.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud