Cross-build commands for plugins do not work if plugins are subprojects

[netvl]

steps

1. Use this build definition:

lazy val common = Vector(
  organization := "org.example",
  crossSbtVersions := Vector("0.13.16", "1.0.0")
)

lazy val root = (project in file("."))
  .settings(
    common
  )
  .aggregate(plugin1, plugin2)

lazy val plugin1 = project
  .settings(
    common,
    sbtPlugin := true
  )

lazy val plugin2 = project
  .settings(
    common,
    sbtPlugin := true
  )

2. Run ^^0.13.16 in the SBT console:

sbt:root> ^^0.13.16
[info] Setting `sbtVersion in pluginCrossBuild` to 0.13.16
[info] Set current project to root (in build file:/private/tmp/sbtest/)

3. Observe that Scala version has not changed (although pluginCrossBuild::sbtVersion is correct):

sbt:root> show scalaVersion
[info] plugin1/*:scalaVersion
[info]  2.12.3
[info] plugin2/*:scalaVersion
[info]  2.12.3
[info] root/*:scalaVersion
[info]  2.12.3
sbt:root> show pluginCrossBuild::sbtVersion
[info] plugin1/*:pluginCrossBuild::sbtVersion
[info]  0.13.16
[info] plugin2/*:pluginCrossBuild::sbtVersion
[info]  0.13.16
[info] root/*:pluginCrossBuild::sbtVersion
[info]  0.13.16

Not sure if sbtVersion being not set to 0.13.16 is a problem, but it is not updated as well:

sbt:root> show sbtVersion
[info] plugin1/*:sbtVersion
[info]  1.0.0
[info] plugin2/*:sbtVersion
[info]  1.0.0
[info] root/*:sbtVersion
[info]  1.0.0

problem

The Scala version is not changed for child projects. The compilation fails with various errors about macros being compiled for invalid version of Scala.

expectation

Scala version is changed correctly, and I'm able to cross-build all of the plugins in the build.

notes

If you go into a particular project, then it works:

sbt:root> project plugin1
[info] Set current project to plugin1 (in build file:/private/tmp/sbtest/)
sbt:plugin1> ^^0.13.16
[info] Setting `sbtVersion in pluginCrossBuild` to 0.13.16
[info] Set current project to plugin1 (in build file:/private/tmp/sbtest/)
sbt:plugin1> show scalaVersion
[info] 2.10.6

sbt version: 1.0.0

[eed3si9n]

Here's an interesting workaround by @jroper - sbt/sbt-pgp@97180d3:

     // Because we're both a library and an sbt plugin, we use crossScalaVersions rather than crossSbtVersions for
     // cross building. So you can use commands like +scripted.
     crossScalaVersions := Seq("2.10.6", "2.12.3"), 
     sbtVersion in pluginCrossBuild := {
       scalaBinaryVersion.value match {
         case "2.10" => "0.13.16"
         case "2.12" => "1.0.0"
       }
     },

[jroper]

The way I see it, the sbt cross building support is a convenience for when you just have one project that's an sbt plugin. I don't think there's a straight forward fix for making it work when you have a mixture of sbt plugins and regular libraries - what does it mean to cross build something that isn't an sbt plugin against multiple versions of sbt? In many situations you may want to cross build the library against Scala 2.10, 2.11 and 2.12. How do you cross build part of the project against different Scala versions, and another part against different sbt versions? How would sbt know to line up which scala versions with which sbt versions in order to make sure everything resolves correctly? I don't think there's any straight forward way.

At the end of the day, both sbt and Scala cross building is a bit of a hack, and even with the improvements in Scala cross building in sbt 1.0, there are still scenarios that don't work well. So I'd be inclined to say that the resolution to this is to document the limitation, and say it's by design. The above work around will work as long as sbt versions can be differentiated by Scala versions. That assumption might not always hold, but it does hold now and there's not going to be a new major sbt version for a long time so I don't think it's something we need to worry too much about, we can tackle that problem if and when it happens in the future.

[eed3si9n]

At the end of the day, both sbt and Scala cross building is a bit of a hack, and even with the improvements in Scala cross building in sbt 1.0, there are still scenarios that don't work well. So I'd be inclined to say that the resolution to this is to document the limitation, and say it's by design.

I agree.

The above work around will work as long as sbt versions can be differentiated by Scala versions. That assumption might not always hold, but it does hold now and there's not going to be a new major sbt version for a long time so I don't think it's something we need to worry too much about, we can tackle that problem if and when it happens in the future.

Right. We have the "extra" _0.13 because we don't assume scalaBinaryVersion => sbtBinaryVersion being injective, but thus far it's been injective. The closest we came is sbt 0.11.0 using Scala 2.9.1, and sbt 0.12 using Scala 2.9.2.

[netvl]

I don't think there's a straight forward fix for making it work when you have a mixture of sbt plugins and regular libraries

I'm not sure why are you talking about regular libraries. My project (and the example above) consists only of SBT plugins; there are no libraries in it. The root project is used solely for aggregation and its build results, if any, are completely ignored. It's just it is really unfeasible to split our set of plugins into multiple builds, because they are intended to work together (with some being optional), as well as to simplify the way they are built and published.

[jroper]

Have you tried setting sbtPlugin := true in the root project?

[netvl]

No, I didn't; because it is not really a plugin, it didn't even occur to me. I'll try it and report asap.

[netvl]

Still does not work. ^^ works for the root project after I add sbtPlugin := true in it, but not for the child projects.

[jroper]

I guess then that this can probably be fixed fairly easily (note, I didn't write the code and I've barely looked at it, I've been far more involved in the Scala cross building support, but both features are very similar in nature), but in the mean time I would recommend using the word around above.

[olafurpg]

The workaround in #3473 (comment) is preferable to crossSbtVersions IMO since it works with multi-module builds (which are the norm).

It would be great to update the documentation on cross-building sbt plugins. The top hit on Google leads to the outdated document https://www.scala-sbt.org/1.0/docs/Cross-Build-Plugins.html recommending to set scalaCompilerBridgeSource.

[eed3si9n]

https://www.scala-sbt.org/1.0/docs/Cross-Build-Plugins.html#Mixing+libraries+and+sbt+plugins+in+a+build is updated.