[SPARK-39800][SQL][WIP] DataSourceV2: View Support #44197
Conversation
|
Had to create a new pull request since #39796 was closed |
|
Current status:
Follow up in future pull requests:
|
| @@ -0,0 +1,104 @@ | |||
| /* | |||
There was a problem hiding this comment.
nit: the file is named VIewSubstitution. Should be ViewSubstitution
| if (catalog.viewExists(ident)) { | ||
| catalog.dropView(ident) | ||
| } | ||
| catalog.createView( |
There was a problem hiding this comment.
I think this should be using the new createOrReplace() API that was introduced by #43677:
catalog.createOrReplaceView(
ident,
sql,
currentCatalog,
currentNamespace,
viewSchema,
queryColumnNames,
columnAliases,
columnComments,
newProperties.asJava)
There was a problem hiding this comment.
Actually
catalog.replaceView(viewInfo, true)
| ident: Identifier, | ||
| ifExists: Boolean) extends LeafV2CommandExec { | ||
|
|
||
| override protected def run(): Seq[InternalRow] = { |
There was a problem hiding this comment.
I think this can be simplified to
override def run(): Seq[InternalRow] = {
if (catalog.viewExists(ident)) {
catalog.dropView(ident)
} else if (!ifExists) {
throw QueryCompilationErrors.noSuchViewError(ident)
}
Seq.empty
}
| catalogManager.currentCatalog.name, | ||
| catalogManager.currentNamespace, | ||
| c.comment, | ||
| c.schema, |
There was a problem hiding this comment.
| c.schema, | |
| c.viewSchema, |
There was a problem hiding this comment.
otherwise this will be passing the wrong schema here
There was a problem hiding this comment.
Thanks for catching this!
|
Looking to leverage the newly added |
|
Almost done converting to v2 command framework. Seeing great reduction in changes. |
|
Failed unit tests
|
|
Only 2 unit tests failed but they succeeded on my laptop
Maybe they are flaky? |
|
Done:
TODO:
Future pull requests:
|
|
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. |
What changes were proposed in this pull request?
This PR adds support to load, create, alter, and drop views in DataSource V2 catalogs.
Why are the changes needed?
Support views stored in DataSourceV2 catalogs. Details in SPIP.
Does this PR introduce any user-facing change?
How was this patch tested?