[TOREE-555] Upgrade Coursier from 1.0.3 to 2.0.0

[pan3793]

We need to upgrade Coursier to 2.x to support Scala 2.13, unfortunately, the coursier's APIs we used are totally different ...

Why Coursier 2.0.0 is chosen? Coursier 2.0.1 onwards depends on
scala-collection-compat_2.12:2.2.0, but Spark 3.3.2 ships scala-collection-compat_2.12:2.1.1. By default, spark.driver.userClassPathFirst is false, which means scala-collection-compat_2.12:2.1.1 will be used, then a higher version of Coursier will fail with

24/09/03 05:23:03 WARN MagicManager: Magic AddDeps failed to execute with error: 
scala.collection.compat.package$.canBuildFromIterableViewMapLike()Lscala/collection/generic/CanBuildFrom;
java.lang.NoSuchMethodError: scala.collection.compat.package$.canBuildFromIterableViewMapLike()Lscala/collection/generic/CanBuildFrom;
        at coursier.ivy.IvyXml$.$anonfun$project$4(IvyXml.scala:186)
        at scala.util.Either.map(Either.scala:353)
        at coursier.ivy.IvyXml$.$anonfun$project$3(IvyXml.scala:166)
        at scala.util.Either.flatMap(Either.scala:341)
        at coursier.ivy.IvyXml$.project(IvyXml.scala:162)
        at org.apache.toree.dependencies.CoursierDependencyDownloader.$anonfun$getBaseDependencies$8(CoursierDependencyDownloader.scala:255)

Due to API change, the log for %AddDeps has a little different

%AddDeps org.joda joda-money 0.11 --transitive --trace --verbose

Before:

Marking org.joda:joda-money:0.11 for download
Preparing to fetch from:
-> file:/tmp/toree_add_deps5662724810625125387/
-> https://repo1.maven.org/maven2
=> 1 (): Downloading https://repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.pom.sha1
=> 1 (): Downloading https://repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.pom
===> 1 (joda-money-0.11.pom.sha1): Is 40 total bytes
===> 1 (joda-money-0.11.pom.sha1): Downloaded 40 bytes (100.00%)
=> 1 (joda-money-0.11.pom.sha1): Finished downloading
===> 1 (joda-money-0.11.pom): Is 22792 total bytes
===> 1 (joda-money-0.11.pom): Downloaded 15727 bytes (69.00%)
===> 1 (joda-money-0.11.pom): Downloaded 22792 bytes (100.00%)
=> 1 (joda-money-0.11.pom): Finished downloading
=> 1 (): Downloading https://repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.jar.sha1
=> 2 (): Downloading https://repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.jar
===> 1 (joda-money-0.11.jar.sha1): Is 40 total bytes
===> 1 (joda-money-0.11.jar.sha1): Downloaded 40 bytes (100.00%)
=> 1 (joda-money-0.11.jar.sha1): Finished downloading
===> 2 (joda-money-0.11.jar): Is 63725 total bytes
===> 2 (joda-money-0.11.jar): Downloaded 15713 bytes (24.66%)
===> 2 (joda-money-0.11.jar): Downloaded 32097 bytes (50.37%)
===> 2 (joda-money-0.11.jar): Downloaded 48481 bytes (76.08%)
===> 2 (joda-money-0.11.jar): Downloaded 63725 bytes (100.00%)
=> 2 (joda-money-0.11.jar): Finished downloading
-> New file at /tmp/toree_add_deps5662724810625125387/https/repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.jar

Now

Marking org.joda:joda-money:0.11 for download
Preparing to fetch from:
-> file:/tmp/toree-tmp-dir954705995229959887/toree_add_deps/
-> https://repo1.maven.org/maven2
=> 0 (joda-money-0.11.pom): Downloading https://repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.pom
===> 0 (joda-money-0.11.pom): Is 22792 total bytes
===> 0 (joda-money-0.11.pom): Downloaded 1371 bytes (6.02%)
===> 0 (joda-money-0.11.pom): Downloaded 2742 bytes (12.03%)
===> 0 (joda-money-0.11.pom): Downloaded 4113 bytes (18.05%)
===> 0 (joda-money-0.11.pom): Downloaded 5484 bytes (24.06%)
===> 0 (joda-money-0.11.pom): Downloaded 6855 bytes (30.08%)
===> 0 (joda-money-0.11.pom): Downloaded 8226 bytes (36.09%)
===> 0 (joda-money-0.11.pom): Downloaded 9597 bytes (42.11%)
===> 0 (joda-money-0.11.pom): Downloaded 10968 bytes (48.12%)
===> 0 (joda-money-0.11.pom): Downloaded 12339 bytes (54.14%)
===> 0 (joda-money-0.11.pom): Downloaded 13710 bytes (60.15%)
===> 0 (joda-money-0.11.pom): Downloaded 15081 bytes (66.17%)
===> 0 (joda-money-0.11.pom): Downloaded 16452 bytes (72.18%)
===> 0 (joda-money-0.11.pom): Downloaded 17823 bytes (78.20%)
===> 0 (joda-money-0.11.pom): Downloaded 19194 bytes (84.21%)
===> 0 (joda-money-0.11.pom): Downloaded 20565 bytes (90.23%)
===> 0 (joda-money-0.11.pom): Downloaded 21936 bytes (96.24%)
===> 0 (joda-money-0.11.pom): Downloaded 22792 bytes (100.00%)
=> 0 (joda-money-0.11.pom): Finished downloading
=> 1 (joda-money-0.11.jar): Downloading https://repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.jar
===> 1 (joda-money-0.11.jar): Is 63725 total bytes
===> 1 (joda-money-0.11.jar): Downloaded 1371 bytes (2.15%)
===> 1 (joda-money-0.11.jar): Downloaded 2742 bytes (4.30%)
===> 1 (joda-money-0.11.jar): Downloaded 4113 bytes (6.45%)
===> 1 (joda-money-0.11.jar): Downloaded 5484 bytes (8.61%)
===> 1 (joda-money-0.11.jar): Downloaded 6855 bytes (10.76%)
===> 1 (joda-money-0.11.jar): Downloaded 8226 bytes (12.91%)
===> 1 (joda-money-0.11.jar): Downloaded 9597 bytes (15.06%)
===> 1 (joda-money-0.11.jar): Downloaded 10968 bytes (17.21%)
===> 1 (joda-money-0.11.jar): Downloaded 12339 bytes (19.36%)
===> 1 (joda-money-0.11.jar): Downloaded 13710 bytes (21.51%)
===> 1 (joda-money-0.11.jar): Downloaded 15081 bytes (23.67%)
===> 1 (joda-money-0.11.jar): Downloaded 16384 bytes (25.71%)
===> 1 (joda-money-0.11.jar): Downloaded 17755 bytes (27.86%)
===> 1 (joda-money-0.11.jar): Downloaded 19126 bytes (30.01%)
===> 1 (joda-money-0.11.jar): Downloaded 20497 bytes (32.16%)
===> 1 (joda-money-0.11.jar): Downloaded 21868 bytes (34.32%)
===> 1 (joda-money-0.11.jar): Downloaded 23239 bytes (36.47%)
===> 1 (joda-money-0.11.jar): Downloaded 24610 bytes (38.62%)
===> 1 (joda-money-0.11.jar): Downloaded 25981 bytes (40.77%)
===> 1 (joda-money-0.11.jar): Downloaded 27352 bytes (42.92%)
===> 1 (joda-money-0.11.jar): Downloaded 28723 bytes (45.07%)
===> 1 (joda-money-0.11.jar): Downloaded 30094 bytes (47.22%)
===> 1 (joda-money-0.11.jar): Downloaded 31465 bytes (49.38%)
===> 1 (joda-money-0.11.jar): Downloaded 32768 bytes (51.42%)
===> 1 (joda-money-0.11.jar): Downloaded 49152 bytes (77.13%)
===> 1 (joda-money-0.11.jar): Downloaded 63725 bytes (100.00%)
=> 1 (joda-money-0.11.jar): Finished downloading
-> New file at /tmp/toree-tmp-dir954705995229959887/toree_add_deps/https/repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.jar
Obtained 1 files

Also tested other functionalities

[pan3793]

The LICENSE_extras is out-of-sync, I updated it too.

[pan3793]

@lresende I think this is the last major change to prepare support Scala 2.13, please take a look when you have time.