Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-36980][SQL] Insert support query with CTE #34252

Closed
wants to merge 4 commits into from
Closed

[SPARK-36980][SQL] Insert support query with CTE #34252

wants to merge 4 commits into from

Conversation

AngersZhuuuu
Copy link
Contributor

What changes were proposed in this pull request?

For current SQL grammar

INSERT INTO t_delta (WITH v1(c1) as (values (1)) select 1, 2,3 from v1);   OK
INSERT INTO t_delta WITH v1(c1) as (values (1)) select 1, 2,3 from v1;    FAILED

It's caused by

insertInto query queryOrganization

Here change to

insert query

can support CTE after insert

Why are the changes needed?

Support CTE after insert query

Does this PR introduce any user-facing change?

How was this patch tested?

Added UT

@github-actions github-actions bot added the SQL label Oct 12, 2021
@AngersZhuuuu
Copy link
Contributor Author

ping @cloud-fan

@SparkQA
Copy link

SparkQA commented Oct 12, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48599/

@SparkQA
Copy link

SparkQA commented Oct 12, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48599/

@cloud-fan
Copy link
Contributor

can you also update the sql reference doc?

@github-actions github-actions bot added the DOCS label Oct 12, 2021
@AngersZhuuuu
Copy link
Contributor Author

can you also update the sql reference doc?

How about current?

@SparkQA
Copy link

SparkQA commented Oct 12, 2021

Test build #144122 has finished for PR 34252 at commit 8a1b4e4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 12, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48617/

@SparkQA
Copy link

SparkQA commented Oct 12, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48617/

@SparkQA
Copy link

SparkQA commented Oct 12, 2021

Test build #144140 has finished for PR 34252 at commit 39307ba.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AngersZhuuuu
Copy link
Contributor Author

ping @cloud-fan

@SparkQA
Copy link

SparkQA commented Oct 14, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48744/

@SparkQA
Copy link

SparkQA commented Oct 14, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48744/

@SparkQA
Copy link

SparkQA commented Oct 14, 2021

Test build #144265 has finished for PR 34252 at commit 846f6c8.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • class HistogramPlotBase(NumericPlotBase):
  • class KdePlotBase(NumericPlotBase):
  • new_class = type(NameTypeHolder.short_name, (NameTypeHolder,),
  • class SparkUpgradeException(CapturedException):
  • public class ExpressionImplUtils
  • class NoSuchIndexException(message: String, cause: Option[Throwable] = None)
  • case class AesEncrypt(input: Expression, key: Expression, child: Expression)
  • case class AesDecrypt(input: Expression, key: Expression, child: Expression)
  • case class CreateView(
  • case class ShowCatalogsCommand(pattern: Option[String]) extends LeafRunnableCommand

@cloud-fan
Copy link
Contributor

thanks, merging to master!

@cloud-fan cloud-fan closed this in 9865c09 Oct 15, 2021
dongjoon-hyun pushed a commit that referenced this pull request Nov 4, 2023
@LuciferYang @dongjoon-hyun
[SPARK-45781][BUILD] Upgrade Arrow to 14.0.0
749c79e 
### What changes were proposed in this pull request?
This pr upgrade Apache Arrow from 13.0.0 to 14.0.0.

### Why are the changes needed?
The Apache Arrow 14.0.0 release brings a number of enhancements and bug fixes.
‎
In terms of bug fixes, the release addresses several critical issues that were causing failures in integration jobs with Spark([GH-36332](apache/arrow#36332)) and problems with importing empty data arrays([GH-37056](apache/arrow#37056)). It also optimizes the process of appending variable length vectors([GH-37829](apache/arrow#37829)) and includes C++ libraries for MacOS AARCH 64 in Java-Jars([GH-38076](apache/arrow#38076)).
‎
The new features and improvements focus on enhancing the handling and manipulation of data. This includes the introduction of DefaultVectorComparators for large types([GH-25659](apache/arrow#25659)), support for extended expressions in ScannerBuilder([GH-34252](apache/arrow#34252)), and the exposure of the VectorAppender class([GH-37246](apache/arrow#37246)).
‎
The release also brings enhancements to the development and testing process, with the CI environment now using JDK 21([GH-36994](apache/arrow#36994)). In addition, the release introduces vector validation consistent with C++, ensuring consistency across different languages([GH-37702](apache/arrow#37702)).
‎
Furthermore, the usability of VarChar writers and binary writers has been improved with the addition of extra input methods([GH-37705](apache/arrow#37705)), and VarCharWriter now supports writing from `Text` and `String`([GH-37706](apache/arrow#37706)). The release also adds typed getters for StructVector, improving the ease of accessing data([GH-37863](apache/arrow#37863)).

The full release notes as follows:
- https://arrow.apache.org/release/14.0.0.html

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #43650 from LuciferYang/arrow-14.

Lead-authored-by: yangjie01 <yangjie01@baidu.com>
Co-authored-by: YangJie <yangjie01@baidu.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cloud-fan cloud-fan cloud-fan approved these changes

Assignees
No one assigned
Labels
DOCS SQL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@AngersZhuuuu @SparkQA @cloud-fan