- Expose
JvmException
to capture JVM error messages separately (#566)
- AssemblyLoader should use absolute assembly path when loading assemblies (570)
- None
- None
- Broadcast variables do not work with dotnet-interactive (#561)
The following table describes the oldest version of the worker that the current version is compatible with, along with new features that are incompatible with the worker.
Oldest compatible Microsoft.Spark.Worker version | Incompatible features |
---|---|
v0.9.0 | DataFrame with Grouped Map UDF (#277) |
DataFrame with Vector UDF (#277) | |
Support for Broadcast Variables (#414) | |
Support for TimestampType (#428) |
The following table describes the oldest version of .NET for Apache Spark release that the current worker is compatible with.
Oldest compatible .NET for Apache Spark release version |
---|
v0.9.0 |
The following table outlines the supported Spark versions along with the microsoft-spark JAR to use with:
Spark Version | microsoft-spark JAR |
---|---|
2.3.* | microsoft-spark-2.3.x-0.12.1.jar |
2.4.0 | microsoft-spark-2.4.x-0.12.1.jar |
2.4.1 | |
2.4.3 | |
2.4.4 | |
2.4.5 | |
2.4.6 | |
2.4.2 | Not supported |