[ADAP-366] [Bug] Issue with table materialization #365
Assignees
Labels
type:bug
Something isn't working
Comments
github-actions
bot
changed the title
|
Thanks for reporting @dave-connors-3 ! I tried the following model and wasn't able to replicate:
I know you are able to replicate this in the CI for dbt-project-evaluator though :) Could you try tweaking the example above to see if you can get it to fail? |
|
@dbeatty10 I have done you a great disservice! these are view models!! |
|
Lol, no prob @dave-connors-3 😂 I was able to reproduce similar error to what you reported! 🎉 I used dbt-redshift==1.5.0b2 and dbt-core==1.5.0-b2.
{{
config(
materialized='table',
)
}}
select 1 as id
{{
config(
materialized='view',
)
}}
select * from {{ ref('base_model') }}Run it twice -- first time should work and second time should 💥 dbt run
dbt run |
|
@dbeatty10 To clarify on your reproduction, did you run the first |
|
@mikealfare ran both using the same virtual environment. Doing |
|
I am going to re-label the subject line from "Regression" to "Bug" instead since we don't consider it a regression unless it shows up in a final release. Maybe the most accurate title would be "Pre-regression"? But we'll stick with "Bug" for today. |
dbeatty10
changed the title
|
Thanks @dbeatty10. That's how I read it too, which is slightly different than @dave-connors-3's version. The @Fleid, I want to make sure you see this so that we can prioritize it. If I understand the issue correctly, we basically can't run |
|
A bit of a guess, but could it be due to the fact that this commit changed |
|
@b-per I'm guess you're right that they are related. Context: dbt Core 1.5 is switching from the |
|
@sathiish-kumar we will need your eyes on this one too! |
|
I troubleshot this a bit, based on the above mentioned examples, I was able to repro the issue but it's a bit puzzling as to how it was working in the first place in 1.4.0 (this may just be me not understanding things right) When doing the To understand how things work with 1.4.0, I would love to have access to the raw SQL that's being logged when executing |
|
What is the issue you are getting when going back to For this one I generated the debug output for both Output on 1.4 (working)Output on 1.5 (failing) |
|
@b-per you are prince 👑 for generating output on 1.4 and 1.5. For anyone looking for an easy way to use Python These commands are not optimized for readability. Rather, they are optimized for copy-paste + updating the template (regardless if a pre-release version or not). dbt-redshift 1.4.0python3 -m venv redshift_1.4
source redshift_1.4/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install --pre dbt-redshift~=1.4.0.dev0 dbt-core~=1.4.0.dev0
source redshift_1.4/bin/activate
dbt --version
deactivatedbt-redshift 1.5.0python3 -m venv redshift_1.5
source redshift_1.5/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install --pre dbt-redshift~=1.5.0.dev0 dbt-core~=1.5.0.dev0
source redshift_1.5/bin/activate
dbt --version
deactivateGeneral casepython3 -m venv {{adapter}}_{{version_number}}
source {{adapter}}_{{version_number}}/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install --pre dbt-{{adapter}}~={{version_number}}.dev0 dbt-core~={{version_number}}.dev0
source {{adapter}}_{{version_number}}/bin/activate
dbt --version
deactivate |
|
Thanks @b-per I'll try using the venv. In the 1.4.0 logs (relevant snippet below), we see that the 1.4.0In the 1.5.0-b3 logs (relevant snippet below), we see that the 1.5.0-b3Assuming the view
Assuming this is correct, I'm not sure why 1.4.0 doesn't do this but directly chooses to rename the tmp view to |
From a cursory glance, it seems like maybe we tried to implement a safety rollback feature and botched the view name.
There are several spots where |
|
I think I've stumbled upon the root-cause of the issue here, I'm still yet to figure out why this doesn't happen in 1.4.5. This is predicated on my understanding of the sequence of steps for materialization being right: Here is the sequence of SQL that would be executed on the second run with 1.5.0-b3, assuming the first run created a After the first Will try to figure out why things aren't working the same way in 1.4.5 but any pointers on changes in materialization between the two would be appreciated. Thanks! |
|
@sathiish-kumar is correct. I created a test case to reflect this issue and it fails. If I add Since this didn't seem to be an issue in |
|
Providing an update in case you're also looking into this @sathiish-kumar, @b-per, @dbeatty10, et. al. It looks like |
|
We dropped inheritance from |
2 tasks
mikealfare
added a commit
that referenced
this issue
Mar 16, 2023
mikealfare
added a commit
that referenced
this issue
Mar 16, 2023
* moved models out of test file (more than 20 lines); moved files to directory (more than 1 file) * created test case for #365 * added sslmode to `RedshiftCredentials._connection_keys` * moved code out of try block that would not trigger exception * added link relations in cache logic * pulled up abstract methods that were not implemented, but retained `NotImplementedError` * the macro `postgres_get_relations` only has one underscore in `dbt-core` instead of two, like `redshift__get_relations` * changie
abbywh
pushed a commit
to abbywh/dbt-redshift
that referenced
this issue
Oct 11, 2023
* moved models out of test file (more than 20 lines); moved files to directory (more than 1 file) * created test case for dbt-labs#365 * added sslmode to `RedshiftCredentials._connection_keys` * moved code out of try block that would not trigger exception * added link relations in cache logic * pulled up abstract methods that were not implemented, but retained `NotImplementedError` * the macro `postgres_get_relations` only has one underscore in `dbt-core` instead of two, like `redshift__get_relations` * changie
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is this a regression in a recent version of dbt-redshift?
Current Behavior
when running dbt on-redshift on 1.5.0.b2, the backup relation step of the materialization seems to fail.
Seems to be unable to find a relation in the
load_cached_relationstep, and throws aLocalRangeVarGetReliderror.Expected/Previous Behavior
In 1.4.4 this same project runs without error!
Steps To Reproduce
Relevant log output
No response
Environment
Additional Context
No response
The text was updated successfully, but these errors were encountered: