Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

[NSE-1135] Introduce shim layer for supporting spark 3.2.2 #1146

Merged
merged 5 commits into from
Oct 31, 2022

Conversation

PHILO-HE
Copy link
Collaborator

What changes were proposed in this pull request?

Refactored the shim layer code for spark 3.2.x. A common module for spark 3.2.x shims is introduced to maintain common code for spark321 and spark322. The spark322 shim layer module was added and some classes applicable for spark322 are also kept in this module, like what we did for other spark versions.

We have spark-3.2.1 and spark-3.2.2 profile added in the PR. To compile for spark321 or spark322, use -Pspark-3.2.1 and -Pspark-3.2.2 respectively in maven building. And -Pspark-3.2 is still kept, which uses spark321 as default.

How was this patch tested?

TPC-H/DS.

@github-actions
Copy link

#1135

@zhouyuan zhouyuan merged commit 192ecc3 into oap-project:main Oct 31, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants