Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-39665][INFRA] Bump workflow versions in GitHub Actions #37056

Closed
wants to merge 9 commits into from
Closed

[SPARK-39665][INFRA] Bump workflow versions in GitHub Actions #37056

wants to merge 9 commits into from

Conversation

ArjunSharda
Copy link

@ArjunSharda ArjunSharda commented Jul 2, 2022 •
edited
Loading

This pull requests bumps some of the workflow versions as some are outdated.
*Note, this has not been tested with an IDE, I just did some basic changes I spotted.

@github-actions github-actions bot added the INFRA label Jul 2, 2022
@ArjunSharda ArjunSharda closed this Jul 2, 2022
@ArjunSharda ArjunSharda reopened this Jul 2, 2022
@AmplabJenkins
Copy link

Can one of the admins verify this patch?

HyukjinKwon
HyukjinKwon reviewed Jul 3, 2022
View reviewed changes
Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ArjunSharda mind taking a look at https://spark.apache.org/contributing.html? We should file a JIRA, fix the title format, feel the PR description format, etc.

@ArjunSharda
Copy link
Author

@ArjunSharda mind taking a look at https://spark.apache.org/contributing.html? We should file a JIRA, fix the title format, feel the PR description format, etc.

Hey, I thought a JIRA would not be needed as it's a minor change with versions.

@HyukjinKwon
Copy link
Member

Let's file a JIRA. I think we should track it with a JIRA since technically we're upgrading the major versions of the plugins.

@ArjunSharda
Copy link
Author

Let's file a JIRA. I think we should track it with a JIRA since technically we're upgrading the major versions of the plugins.

Alright, made. https://issues.apache.org/jira/browse/SPARK-39665

@HyukjinKwon HyukjinKwon changed the title (GitHub CI) Bump workflow versions [SPARK-39665][INFRA] Bump workflow versions in GitHub Actions Jul 4, 2022
dongjoon-hyun
dongjoon-hyun reviewed Jul 6, 2022
View reviewed changes
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I ask the reason why we need this, @ArjunSharda ?

@ArjunSharda
Copy link
Author

May I ask the reason why we need this, @ArjunSharda ?

Not needed, but the current versions used are outdated and could be bumped.

dongjoon-hyun
dongjoon-hyun requested changes Jul 10, 2022
View reviewed changes
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is no known issues affecting Apache Spark repository, let's avoid any risk by holding on this PR until v3 become stable. As we see in the release history, it seems to be still changing in a fast pace unlike v2.

@ArjunSharda
Copy link
Author

If there is no known issues affecting Apache Spark repository, let's avoid any risk by holding on this PR until v3 become stable. As we see in the release history, it seems to be still changing in a fast pace unlike v2.

I can revert the cache versions

@dongjoon-hyun
Copy link
Member

Do you think the other GitHub Actions are stabler? IMO, the situation is not different. There is no point to track one by one in case of all GitHub Action. I'd recommend to close this PR.

If there is no known issues affecting Apache Spark repository, let's avoid any risk by holding on this PR until v3 become stable. As we see in the release history, it seems to be still changing in a fast pace unlike v2.

I can revert the cache versions

@ArjunSharda
Copy link
Author

Do you think the other GitHub Actions are stabler? IMO, the situation is not different. There is no point to track one by one in case of all GitHub Action. I'd recommend to close this PR.

If there is no known issues affecting Apache Spark repository, let's avoid any risk by holding on this PR until v3 become stable. As we see in the release history, it seems to be still changing in a fast pace unlike v2.

I can revert the cache versions

I think as per what I saw except setup-java it should be okay.

@dongjoon-hyun
Copy link
Member

Yes, setup-java is especially aggressive. So, @ArjunSharda , your approach initially didn't consider much about the GitHub Action's upgrade cadence and the risk on Apache Spark community side. To prevent accidental merging, let me close this PR. We can still continue our discussion about the necessity of this PR content.

I think as per what I saw except setup-java it should be okay.

dongjoon-hyun pushed a commit that referenced this pull request Nov 4, 2023
@LuciferYang @dongjoon-hyun
[SPARK-45781][BUILD] Upgrade Arrow to 14.0.0
749c79e 
### What changes were proposed in this pull request?
This pr upgrade Apache Arrow from 13.0.0 to 14.0.0.

### Why are the changes needed?
The Apache Arrow 14.0.0 release brings a number of enhancements and bug fixes.
‎
In terms of bug fixes, the release addresses several critical issues that were causing failures in integration jobs with Spark([GH-36332](apache/arrow#36332)) and problems with importing empty data arrays([GH-37056](apache/arrow#37056)). It also optimizes the process of appending variable length vectors([GH-37829](apache/arrow#37829)) and includes C++ libraries for MacOS AARCH 64 in Java-Jars([GH-38076](apache/arrow#38076)).
‎
The new features and improvements focus on enhancing the handling and manipulation of data. This includes the introduction of DefaultVectorComparators for large types([GH-25659](apache/arrow#25659)), support for extended expressions in ScannerBuilder([GH-34252](apache/arrow#34252)), and the exposure of the VectorAppender class([GH-37246](apache/arrow#37246)).
‎
The release also brings enhancements to the development and testing process, with the CI environment now using JDK 21([GH-36994](apache/arrow#36994)). In addition, the release introduces vector validation consistent with C++, ensuring consistency across different languages([GH-37702](apache/arrow#37702)).
‎
Furthermore, the usability of VarChar writers and binary writers has been improved with the addition of extra input methods([GH-37705](apache/arrow#37705)), and VarCharWriter now supports writing from `Text` and `String`([GH-37706](apache/arrow#37706)). The release also adds typed getters for StructVector, improving the ease of accessing data([GH-37863](apache/arrow#37863)).

The full release notes as follows:
- https://arrow.apache.org/release/14.0.0.html

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #43650 from LuciferYang/arrow-14.

Lead-authored-by: yangjie01 <yangjie01@baidu.com>
Co-authored-by: YangJie <yangjie01@baidu.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HyukjinKwon HyukjinKwon HyukjinKwon left review comments

@dongjoon-hyun dongjoon-hyun dongjoon-hyun requested changes

Assignees
No one assigned
Labels
INFRA
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ArjunSharda @AmplabJenkins @HyukjinKwon @dongjoon-hyun