Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add option to create streams which reference EXTERNAL TABLEs #661

Closed
wants to merge 12 commits into from
Closed

feat: Add option to create streams which reference EXTERNAL TABLEs #661

wants to merge 12 commits into from

Conversation

momer
Copy link
Contributor

@momer momer commented Aug 24, 2021

Allow the creation of STREAMS on EXTERNAL TABLEs.

Test Plan

  • acceptance tests
    • Note: I haven't run these against an account, and there's some copy/pasta from the external_table which I modified a bit, so is a bit fragile.
  • unit tests

References

@momer momer requested a review from a team as a code owner August 24, 2021 23:08
@momer momer requested a review from edulop91 August 24, 2021 23:08
resource.TestCheckResourceAttr("snowflake_stream.test_stream", "name", accName),
resource.TestCheckResourceAttr("snowflake_stream.test_stream", "database", accName),
resource.TestCheckResourceAttr("snowflake_stream.test_stream", "schema", accName),
resource.TestCheckResourceAttr("snowflake_stream.test_stream", "on_table", fmt.Sprintf("%s.%s.%s", accName, accName, "STREAM_ON_TABLE")),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this supposed to be on_external_table?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I think the name is supposed be accName, as well... not STREAM_ON_TABLE

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @alldoami on_external_table is a bool, and creating a stream on an external table without ... EXTERNAL TABLE will raise an error; so, the acceptance test is simply ensuring that the table referenced by on_table (which is an external table created via externalTableStreamConfig) exists.

That said, I should definitely have added a check that on_external_table is true - see the most recent commits, which also resolve the STREAM_ON_TABLE reference - thanks! I'll have to set up an environment to run these tests locally which will reduce impact on you via these PRs, sorry.

@alldoami
Copy link
Contributor

/ok-to-test sha=eb335a8

@github-actions
Copy link

Integration tests failure for eb335a8

@momer
Copy link
Contributor Author

momer commented Aug 25, 2021

@alldoami this should be resolved - I believe this was due to an extra param in the Sprintf in externalTableStreamConfig.

@alldoami
Copy link
Contributor

/ok-to-test sha=c58be0c

@github-actions
Copy link

Integration tests failure for c58be0c

momer and others added 3 commits August 25, 2021 14:21
Co-authored-by: Allison Doami <allison.ox@gmail.com>
Co-authored-by: Allison Doami <allison.ox@gmail.com>
Co-authored-by: Allison Doami <allison.ox@gmail.com>
@momer
Copy link
Contributor Author

momer commented Aug 25, 2021

@alldoami Thank you!

@alldoami
Copy link
Contributor

/ok-to-test sha=a356e27

@github-actions
Copy link

Integration tests failure for a356e27

}
`

return fmt.Sprintf(s, name, name, name, name, locations, name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't want to reuse the config above, you'll have to create another random string to pass in so that the names are different. Just put
accNameExternalTable := strings.ToUpper(acctest.RandStringFromCharSet(10, acctest.CharSetAlpha)) below the definition ofaccName above

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And then replace the accNames with accNameExternalTable in your config above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see - thank you again, see the most recent commits!

Mo Omer added 2 commits August 26, 2021 09:22
…at referenced name, db, schema are different between previous and new external tests
…omer/terraform-provider-snowflake into feature/add-external-tables-to-streams
@alldoami
Copy link
Contributor

/ok-to-test sha=c95a50f

@github-actions
Copy link

Integration tests failure for c95a50f

column {
name = "column2"
type = "TIMESTAMP_NTZ(9)"
as = "($1:'CreatedDate'::timestamp)"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it's not happy with this... Is this written correctly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, should be; I grabbed it directly from the external_table_acceptance_test.go test. I can dig in a bit tonight hopefully, sorry for the many issues with this test

@michael-robbins
Copy link
Contributor

How's this going? Just stumbled across this issue myself!

@alldoami
Copy link
Contributor

@momer Please reopen when ready for another review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants