Mill fails to recompile on version change with "interp.load.ivy"/"@" in build.sc

[hilcode]

The attached tarball contains a tiny project that shows the problem.

Steps to reproduce:

1. Run ./mill clean + __.compile
2. In build.sc, change the dependency version from 2.0.0 to 2.0.0-RC6
3. Run ./mill __.compile (NOTE: no clean!)

Observe that no error occurs: Main.scala is not recompiled. If you run ./mill clean + __.compile instead, you will see the error.

As soon as you remove the "@" in build.sc, everything works as expected again.

mill-fails-to-recompile.tar.gz

[hilcode]

I should add that this is on Linux and using Lefou's Mill wrapper (https://github.com/lefou/millw).

[lefou]

Thanks for reporting. Can you please check, whether a change to the first part (before the @) of your build.sc triggers a re-compilation? My prediction is, that change detection works for the first part of a multi-part script, but not for the remaining ones.

My suspicion is, that we only catch changes to the first multi-part script part. Whether this is because of wrong usage of the Ammonite API or this is a bug in Ammonite or Mill, I can't say ATM.

[hilcode]

No, that doesn't seem to be the case. Changes to subsequent import statements or renaming the trait are noticed and correctly fail the build. So "structural" build changes appear to get noticed. I don't seem to be able to put the interp.load.ivy between (say) my module and trait.

I also tried adding a Dep val for the dependency before the interp.load.ivy. That seems to behave even worse: now not even build.sc is recompiled.

It does not seem to be all that simple. I see different behaviour in different scenarios but none of it seems right.

[lefou]

Your build.sc starts with the following lines:

import mill.scalalib.DepSyntax

interp.load.ivy("com.goyeau" %% "mill-scalafix_mill0.10" % "0.2.8")
@

Is there some reason, you don't just use the following line instead?

import $ivy.`com.goyeau::mill-scalafix::0.2.8`

This would avoid the use of Ammonite multi-stage scripting and hence work around the issues.

[hilcode]

Certainly, I have simplified the problem as much as possible for this bug report.

In fact, I do as you suggest in my actual build.sc (as a workaround) but that is not a real solution as it means I have to hard code the version. What I really want is import $ivy.'com.goyeau::mill-scalafix::$my_version' to work. That does not work so I had to use the interp.load.ivy approach ... which has (or leads to) this bug.

[lefou]

Makes sense, yet I wonder where this variable is defined.

What I want to state here, sooner or later, we might deprecate use of multi-stage scripts, as it's currently unclear whether we keep the full Ammonite feature set as our foundation. (Related: #1675)

[hilcode]

I have a separate Versions.scala file (with a versions.sc symlink to it) that simply contains a Versions object with all the version_* vals. This is imported at the top of build.sc.

It works beautifully ... except with $ivy.`aaa::bbb::$my_version`. :-(

I really don't care for the multi-stage part, that was just a way to achieve my goal.

[lefou]

Here's how I do this in some larger project.

I do all plugin imports is a file build_plugins.sc

// build_plugins.sc

// calculate the version from git version
import $ivy.`de.tototec::de.tobiasroeser.mill.vcs.version::0.1.2`
// generate OSGi manifests
import $ivy.`de.tototec::de.tobiasroeser.mill.osgi::0.3.2-30-1900b5`

Then I import this file in all other *.sc files, e.g. my build.sc starts like this:

// build.sc

import $file.build_plugins
import $file.build_deps
import build_deps.Deps
...

That way, I shared my Mill classpath in a single place.

All dependencies are collected in a file build_deps.sc which I also include whereever needed.

// build_deps.sc

import mill._
import mill.scalalib._

object Deps {
  val scalaVersion = "2.13.16"
  val aIvyDep = ivy"a.ivy::dep:1.2"
  //  ... 
}

Dependencies in the project look like this

def ivyDeps = Agg(
  Deps.slf4j,
  Deps.aIvyDep
)

Whenever I need some of these information in the project itself, I generate files from Mill.

Example:

mill/build.sc

Lines 440 to 461 in 62cf005

	override def generatedSources = T {
	val dest = T.ctx.dest
	val artifacts = T.traverse(dev.moduleDeps)(_.publishSelfDependency)()
	os.write(
	dest / "Versions.scala",
	s"""package mill.scalalib
	|
	|/**
	| * Dependency versions.
	| * Generated from mill in build.sc.
	| */
	|object Versions {
	| /** Version of Ammonite. */
	| val ammonite = "${Deps.ammonite.dep.version}"
	| /** Version of Zinc. */
	| val zinc = "${Deps.zinc.dep.version}"
	|}
	|
	|""".stripMargin
	)
	super.generatedSources() ++ Seq(PathRef(dest))
	}

[hilcode]

That's an interesting approach.

I am in a hybrid situation. The main build is still SBT but I have moved some of the code to Mill. SBT calls Mill as its first step.

That also means I really need just the versions, not actual imports, as I need to share them between SBT and Mill.

[lolgab]

Another possible workaround is to parse the imports you have in the build_plugins.sc file from sbt.
For example:

// build_plugins.sc
import $ivy.`com.goyeau::mill-scalafix::0.2.8`
import $ivy.`de.tototec::de.tobiasroeser.mill.vcs.version::0.1.2`

// project/plugins.sbt
val millPlugins = {
  val dep = """import \$ivy\.`.+::(.+)::(.+)`""".r
  val lines = IO.readLines(file("build_plugins.sc"))
  lines.collect {
    case dep(name, version) => (name, version)
  }.toMap
}
val scalafixVersion = millPlugins("mill-scalafix") // will contain "0.2.8"

[lefou]

I'm going to close this, as it is related to Ammonite multi-stage scripts, which are no longer supported, since we moved away from Ammonite as our script runner foundation. (See #2377)