[OPPRO-320] Get spark short version for loading corresponding spark shim

[PHILO-HE]

What changes were proposed in this pull request?

In spark, short version stands for major_version-minor_version-patch_version. For a spark release, there is no difference between SPARK_VERSION & SPARK_VERSION_SHORT. But according to some customers' feedback, they are using their internally maintained spark. For example, their SPARK_VERSION can be 3.2.2.xxx. By using SPARK_VERSION_SHORT, gluten can load the expected spark-3.2.2 shim by ignoring the last part.

Ref.: oap-project/gazelle_plugin#824

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[Gluten-${ISSUES_ID}] ${detailed message}

See also:

• Other pull requests

[jinchengchenghh]

@FelixYBW @zhouyuan as we last discussed, how about that?

[FelixYBW]

@FelixYBW @zhouyuan as we last discussed, how about that?

This PR make sense. What's our discussion?