FileNotFoundException when using a library published with scoverage enabled

[chuwy]

I built and published library X, using sbt-scoverage 1.6.1 and following settings:

coverageEnabled := true,
coverageMinimum := 50,
coverageFailOnMinimum := true,
coverageHighlighting := false,
(test in Test) := {
  coverageReport.dependsOn(test in Test).value
}

Now, I'm trying to build library Y without scoverage enabled, but with dependency on X. Travis gives me the following exception:

Caused by: java.io.FileNotFoundException: /home/travis/build/mycompany/X/submodule-of-X/target/scala-2.12/scoverage-data/scoverage.measurements.97 (No such file or directory)

I think the solution is to disable scoverage during X's publish (at least what I see in simlar #84 and #228), but I just wanted to raise this as a bug, because it feels very confusing that someone can enable a minor enhancement in their lib and crash other's people builds.

Thanks!

[omerlh]

Same problem here unfortunately :( Going to disable scoverage now...

[alexklibisz]

Same problem. Even if I don't run tests, having coverageEnabled := true modifies the Jar contents. To repro: make a project with coverageEnabled := true, publishLocal, hash the jar, set coverageEnabled := false, publishLocal again, hash the jar again. The jars are very slightly different and I can't quite tell how.

[ckipp01]

Same problem. Even if I don't run tests, having coverageEnabled := true modifies the Jar contents

This is actually expected. You shouldn't ever publish a jar that was created with coverageEnabled := true. When you do this you're actually instrumenting your code, as scoverage is a compiler plugin that traverses your AST and actually wraps certain things in a method call used to track "hits". So you need to ensure to turn coverage off and re-compile before publishing.

[alexklibisz]

Hey Chris, thanks for clarifying that. Do you know if this is intentional? If not, is there any hope to prevent it? It seems like the scope of coverage should strictly be testing. Having it affect published code is definitely an unexpected, undesireable side-effect for me.

[ckipp01]

Do you know if this is intentional?

Yea, it's by design. In order to tell what parts of your code are hit, this sort of has to happen. Apart from putting the instrumented code in a place where it might not be packaged, I'm not sure there is a good way around this. That would require work in both the scalac plugin and also the sbt plugin.

Having it affect published code is definitely an unexpected, undesireable side-effect for me.

So again, it sort of depends on what you mean by "published code". It doesn't affect published code, but rather affects your compiled code by compiling it with instrumentation. If you decide to publish that code, then sure it has unexpected results, but this shouldn't be done. Maybe we can try to make this clearer.

[alexklibisz]

Thanks Chris. I'll expand on what I'm trying to do to make this more concrete. The situation is workable, but maybe it'll give someone some ideas for improvement, or maybe someone will point out something obvious that I can improve in my workflow :).

I have a custom command called validate that calls a few other commands: scalafmtCheckAll, scalastyleCheckAll, evicted, compile, coverageOn, test, coverageReport. I don't have the code in front of me right now, so maybe a couple of those are slightly mis-named, but that's roughly what it does and I got all that working fine.

The part I hit issues is after this validate command is done, when I want to publish an artifact. If it's a server-side app, I assemble a fat jar w/ sbt-assembly and build a docker image. If it's a library, I just call publish.

That leaves room for two problems.

The first problems is that if I have coverageEnabled := true in the project, even if I haven't run the validate step, i.e., I've run zero tests, the artifact ends up instrumented. For this reason, I don't actually see why anyone would have coverageEnabled := true as the default -- you would have to remember to always disable it when publishing, which is very easy to forget. It's also surprising behavior that coverageEnabled affects the source code, even if tests are never invoked. The workaround here is easy enough: just don't set coverageEnabled := true and remember to call coverageOn in the validate command.

The second problem is that I also need to remember to clean before I assemble or publish the jar, since it might have been instrumented by the validate task. This clean has also proven easy to forget. As another workaround, I can add a clean as the last step of the validate task. But that's also not ideal, as it can unnecessarily slow down the turn around time on that command, which can waste time on re-compiling a large project.

[ckipp01]

Thanks for the explanation @alexklibisz. I get what you mean. I guess still the most important thing for people to know is that the coverage command is sticky, so once you use it, it's still on until you turn it off. At work we have a very similiar flow to you where we have a command that runs a bunch of checks, runs coverage, and then aggregates the coverage. What we did to work around what you outlined is to include a coverageOff at the end of the alias. This will then remove the scalac options that were added when you turned coverage off, which changes the build and force a recompilation if you run another task like publish. Maybe something like this would also help in your situation?

[mcanlas]

Can this be closed?

[alexklibisz]

IMO, Yes. I think the conclusion is that we should avoid enabling coverage when building and publishing a JAR.

[clee704]

I think I'm being affected by the same problem, but in my case it's not publishing coverage-enabled jars, but reusing binaries with sbt/zinc remote caching. To run tests in multiple instances, I have a setup where the compilation happens in one instance and the results are published using pushRemoteCache, then in other instances, pullRemoveCache is used to reuse the compilation results. You can even reproduce the issue in a single machine, by running the following commands one after another:

$ sbt clean coverage Test/compile pushRemoteCache
$ sbt clean pullRemoteCache coverage Test/compile test
...
[info] io.delta.storage.OracleCloudLogStoreSuite *** ABORTED *** (43 milliseconds)
[info]   java.io.FileNotFoundException: /ssd/chungmin/repos/delta/core/target/scala-2.12/scoverage-data/scoverage.measurements.95419b61-8e05-49b8-b42d-fff335c61d73.25 (No such file or directory)

I tried copying every target directory from one instance to another before running the sbt command, but it didn't help. It seems copying target directories help.