Spark 3.* bundles transitively depend on Spark 2.4 #516
Replies: 2 comments 1 reply
-
When the bundles are created profiles are used. So to find out actual dependencies maven should be called with the profile.
From this you can see the dependency is provided and should not have any influence further down the line. But the bundle itself is not supposed to be used as maven dependency anyway. It is an artifact ready to use with the specific Spark and Scala version. So it doesn't really matter what it declares dependency on. https://github.com/AbsaOSS/spline-spark-agent#selecting-artifact |
Beta Was this translation helpful? Give feedback.
-
Thank you for the fast and detailed response! Maven packages that contain a spark version Looks like |
Beta Was this translation helpful? Give feedback.
-
The
agent-core
package version is agnostic to the Spark version and it depends on Spark 2.4. In result all Spark 3.* bundles have transitive dependencies on Spark 2.4. See the dependency tree forspark-3.2-spline-agent-bundle_2.12
below:Beta Was this translation helpful? Give feedback.
All reactions