JDK 11 - Class sun.reflect.CallerSensitive not found - continuing with a stub. #11679

Sciss · 2019-08-13T21:34:54Z

I am building a project with "-release", "8", compiling with sbt under OpenJDK 11. For both cross Scala versions 2.12.9 and 2.13.0, this fails at several places with errors like

[error] /home/hhrutz/Documents/devel/SoundProcesses/core/src/main/scala/de/sciss/synth/proc/SynthGraphObj.scala:166:27: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error]           val companion = Class.forName(s"$className").getField("MODULE$").get(null)
[error]                           ^

or

[error] /home/hhrutz/Documents/devel/SoundProcesses/core/src/main/scala/de/sciss/synth/proc/impl/MemoryClassLoader.scala:20:39: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error] final class MemoryClassLoader extends ClassLoader {
[error]                                       ^

Unfortunately it's not easy to isolate the code that breaks (I tried simple reductions but they don't crash the compiler). It's on a project that runs on multiple snapshot dependencies, so the heavy way to reproduce is

java -version
openjdk version "11.0.4" 2019-07-16
OpenJDK Runtime Environment (build 11.0.4+11-post-Debian-1deb10u1)
OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Debian-1deb10u1, mixed mode, sharing)

(i.e. use JDK 11)

git clone https://github.com/Sciss/Lucre.git
cd Lucre
git checkout -b debug 53c779f0edfbf6bf0fc8f276d946290c145d85e3
sbt update
sbt publishLocal
cd ..
git clone https://github.com/Sciss/LucreSwing.git
cd LucreSwing
git checkout -b debug e6b50892553e765a3e840ea68c4bf5e5e0879fd9
sbt update
sbt publishLocal
cd ..
git clone https://github.com/Sciss/SoundProcesses.git
cd SoundProcesses
git checkout -b debug f25d534f5f74b80c3cc52abbec5fcbbed3958e9d
sbt update
sbt compile

This should be reproducible and yield

[error] /data/temp/bla/SoundProcesses/core/src/main/scala/de/sciss/synth/proc/SynthGraphObj.scala:166:27: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error]           val companion = Class.forName(className).getField("MODULE$").get(null)
[error]                           ^

Perhaps related to #10817 . The only place that CallerSensitive appears in seems to be https://github.com/scala/scala/blob/3027652cab884c88f5cc20f710e351a0730c5ba2/src/compiler/scala/tools/nsc/backend/jvm/opt/BytecodeUtils.scala#L132

The text was updated successfully, but these errors were encountered:

SethTisue · 2019-08-21T00:02:15Z

@lrytz this one might interest you

lrytz · 2019-08-23T07:52:50Z

git clone https://github.com/Sciss/Lucre.git
cd Lucre
git checkout -b debug 53c779f0edfbf6bf0fc8f276d946290c145d85e3
sbt update

failed already for me

[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	::          UNRESOLVED DEPENDENCIES         ::
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	:: com.sleepycat#je;6.4.25: not found
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] 	Note: Unresolved dependencies path:
[warn] 		com.sleepycat:je:6.4.25 (/Users/luc/Downloads/tmp/Lucre/build.sbt#L171)
[warn] 		  +- de.sciss:lucre-bdb6_2.12:3.14.0-SNAPSHOT
[error] sbt.librarymanagement.ResolveException: unresolved dependency: com.sleepycat#je;7.4.5: not found

I don't think it's related to the code in the backend that you refer to, as we're not loading any symbols at this point.

It's probably some other entity that carries the @CallerSensitive annotation causing the symbol to be loaded.

It's quite certainly related to the use of -release. Since CallerSensitive is part of sun.reflect, does it actually exists in the ct.sym file? cc @retronym.

Sciss · 2019-08-23T09:57:00Z

failed already for me

Oh great, Oracle is doing its best to tear down all their Java related resources (https://download.oracle.com/maven - 404). It will take me a moment to see where that repository went.

Sciss · 2019-08-23T10:11:04Z

Hmmm, no, download works fine for me (and Travis CI also builds without complaining); perhaps you are behind a strange firewall?

...
[info] downloading http://download.oracle.com/maven/com/sleepycat/je/6.4.25/je-6.4.25.jar ...
[info] 	[SUCCESSFUL ] com.sleepycat#je;6.4.25!je.jar (4266ms)
[info] Done updating.
[info] downloading https://repo1.maven.org/maven2/de/sciss/fileutil_2.12/1.1.3/fileutil_2.12-1.1.3.jar ...
[info] downloading https://repo1.maven.org/maven2/de/sciss/model_2.12/0.3.4/model_2.12-0.3.4.jar ...
[info] downloading https://repo1.maven.org/maven2/de/sciss/span_2.12/1.4.2/span_2.12-1.4.2.jar ...
[info] 	[SUCCESSFUL ] de.sciss#fileutil_2.12;1.1.3!fileutil_2.12.jar (286ms)
[info] 	[SUCCESSFUL ] de.sciss#model_2.12;0.3.4!model_2.12.jar (286ms)
[info] 	[SUCCESSFUL ] de.sciss#span_2.12;1.4.2!span_2.12.jar (463ms)
[info] Done updating.
[info] downloading http://download.oracle.com/maven/com/sleepycat/je/5.0.104/je-5.0.104.jar ...
[info] 	[SUCCESSFUL ] com.sleepycat#je;5.0.104!je.jar (3239ms)
[info] Done updating.
[info] downloading http://download.oracle.com/maven/com/sleepycat/je/7.4.5/je-7.4.5.jar ...
[info] 	[SUCCESSFUL ] com.sleepycat#je;7.4.5!je.jar (4582ms)
[info] Done updating.
[info] downloading https://repo1.maven.org/maven2/de/sciss/fingertree_2.12/1.5.4/fingertree_2.12-1.5.4.jar ...
[info] 	[SUCCESSFUL ] de.sciss#fingertree_2.12;1.5.4!fingertree_2.12.jar (386ms)
[info] Done updating.
[info] Updating ...
[info] Done updating.

retronym · 2019-08-27T06:36:11Z

Yes, the sun.* classes are absent from the ct.sym based archive of historical Java standard library APIs which {javac,scalac} -release use.

APIPhase, a custom compiler phase in SBT/Zinc that serializes the API of each file to perform change detection in incremental compilation, is attempting to serialize the API of the member @CallerSensitive ClassLoader getParent that de.sciss.synth.proc.impl.MemoryClassLoader inherits from j.l.ClassLoader. Until that point, scalac was doing okay without having a classfile for CallerSensitve -- it just used a StubSymbol in its place.

When APIPhase filters the list of annotations for those that extend scala.StaticAnnotation, the stub symbol gives up the ruse and aborts compilation. Static annotations are part of the API because the are could affect how client code is compiled.

I see two ways to make this work.

We could update StubSymbol.isStatic to return false, which should exclude that annotation from Zinc's API serialization.
We could update Zinc with effectively the same change.

/cc @jvican @eed3si9n

retronym · 2019-08-27T06:49:14Z

An immediate workaround would be to declare an annotation sun.reflect.CallerSensitive in your project. You could structure the build in a way to avoid shipping this in your library.

retronym · 2019-08-27T07:06:42Z

I tried changing AnnotationInfo.isStatic. That makes the test project compile as I hoped.

$ sbt '++2.12.10-bin-b7c2be1-SNAPSHOT!' soundprocesses-core/compile
...
[info] Done compiling.
[success] Total time: 23 s, completed 27 Aug. 2019, 4:58:39 pm

commit b7c2be1df4fdc3d0664caaa88c82252d86531367 (HEAD -> ticket/11679, retronym/ticket/11679)
Author: Jason Zaugg <jzaugg@gmail.com>
Date:   Tue Aug 27 16:52:40 2019 +1000

    Assume StubSymbols are not StaticAnnotations

diff --git a/src/reflect/scala/reflect/internal/AnnotationInfos.scala b/src/reflect/scala/reflect/internal/AnnotationInfos.scala
index e438b2d677..619d9ac23e 100644
--- a/src/reflect/scala/reflect/internal/AnnotationInfos.scala
+++ b/src/reflect/scala/reflect/internal/AnnotationInfos.scala
@@ -323,7 +323,14 @@ trait AnnotationInfos extends api.Annotations { self: SymbolTable =>
     /** Check whether the type or any of the arguments are erroneous */
     def isErroneous = atp.isErroneous || args.exists(_.isErroneous)

-    def isStatic = symbol isNonBottomSubClass StaticAnnotationClass
+    def isStatic = {
+      symbol match {
+        case _: StubSymbol =>
+          false // See scala/bug#11679
+        case _ =>
+          symbol isNonBottomSubClass StaticAnnotationClass
+      }
+    }

     /** Check whether any of the arguments mention a symbol */
     def refsSymbol(sym: Symbol) = hasArgWhich(_.symbol == sym)

But I fear that this could make scalac too lenient -- when annotations based on stub symbols make it into signatures serialized by the pickler phase, we should emit a hard error.

retronym · 2019-08-27T08:09:06Z

Extended patch to scalac that would retain the desired fail-fast behaviour in the pickler: retronym/scala#67

In the reproduction in scala/bug#11679, an SBT build that uses `--release 8` and as such does not have `sun._` on the classpath, `APIPhase` (a custom compiler phase in SBT/Zinc that serializes the API of each file to perform change detection in incremental compilation) is attempting to serialize the API of the member `@CallerSensitive ClassLoader getParent` that `de.sciss.synth.proc.impl.MemoryClassLoader` inherits from `j.l.ClassLoader`. Until that point, scalac was doing okay without having a classfile for `CallerSensitve` -- it just used a `StubSymbol` in its place. Scala's pickle phase only serializes `.decls`, not `.members`. When `APIPhase` filters the list of annotations: https://github.com/sbt/zinc/blob/4b414b6677/internal/compiler-bridge/src/main/scala/xsbt/ExtractAPI.scala#L789-L800 for those that extend `scala.StaticAnnotation`, the stub symbol fails and aborts compilation. Static annotations are part of the API because the are could affect how client code is compiled. This commit changes `AnnotationInfo.isStatic` to return false for annotations to absent classfiles. It also makes sure that we still have a hard-error if annotations based on absent classfiles are processed by the pickle phase. I have manually tested this commit with the SBT project in the bug. Fixes scala/bug#11679

Sciss · 2019-09-23T10:26:21Z

Hello. Is there any workaround, I'm running into this again (cross building with Scala 2.12.10 and 2.13.1)

[error] /home/hhrutz/Documents/devel/dotterweide/core/src/main/scala/dotterweide/compiler/BytecodeInvoker.scala:49:44: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error]   private class DynamicClassLoader extends ClassLoader {
[error]                                            ^
[error] one error found

Is there a trick I can use? Unfortunately here I really have to compile under JDK 11 and I really need -release, 8, because one project module makes use of JDK 11 while all others should be compatible with JDK 8.

lrytz · 2019-09-23T10:45:05Z

Maybe you could put the JDK 8 rt.jar on the Scala compiler's classpath while compiling that module (and then not use -release)?

Fixes scala/bug#11679

retronym · 2019-10-11T03:40:33Z

I have proposed a fix in SBT/Zinc: sbt/zinc#701

Fixes scala/bug#11679

Fixes scala/bug#11679 Rewritten from sbt/zinc@2097464

Fixes scala/bug#11679

SethTisue added the jdk11 label Aug 21, 2019

SethTisue added this to the Backlog milestone Aug 21, 2019

retronym mentioned this issue Aug 28, 2019

Assume StubSymbols are not StaticAnnotations scala/scala#8379

Closed

SethTisue modified the milestones: Backlog, 2.12.11 Sep 23, 2019

retronym added a commit to retronym/zinc that referenced this issue Oct 11, 2019

Ignore stub symbols when API hashing annotations

2097464

Fixes scala/bug#11679

retronym mentioned this issue Oct 11, 2019

Ignore stub symbols when API hashing annotations sbt/zinc#701

Merged

eed3si9n closed this as completed in sbt/zinc#701 Oct 11, 2019

eed3si9n pushed a commit to eed3si9n/zinc that referenced this issue Oct 12, 2019

Ignore stub symbols when API hashing annotations

c2c5a14

Fixes scala/bug#11679

lrytz pushed a commit to lrytz/scala that referenced this issue Nov 5, 2019

Ignore stub symbols when API hashing annotations

b318675

Fixes scala/bug#11679 Rewritten from sbt/zinc@2097464

This was referenced Nov 28, 2019

With -release 8 in JDK 11 some classes are not found #11814

Closed

Don't set the (java)bootclasspath explicitly akka/akka#27079

Open

lrytz removed this from the 2.12.11 milestone Feb 27, 2020

jvican pushed a commit to scalacenter/zinc that referenced this issue Apr 27, 2020

Ignore stub symbols when API hashing annotations

b530c61

Fixes scala/bug#11679

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JDK 11 - Class sun.reflect.CallerSensitive not found - continuing with a stub. #11679

JDK 11 - Class sun.reflect.CallerSensitive not found - continuing with a stub. #11679

Sciss commented Aug 13, 2019

SethTisue commented Aug 21, 2019

lrytz commented Aug 23, 2019

Sciss commented Aug 23, 2019

Sciss commented Aug 23, 2019 •

edited

Loading

retronym commented Aug 27, 2019 •

edited

Loading

retronym commented Aug 27, 2019

retronym commented Aug 27, 2019 •

edited

Loading

retronym commented Aug 27, 2019

Sciss commented Sep 23, 2019

lrytz commented Sep 23, 2019

retronym commented Oct 11, 2019

JDK 11 - Class sun.reflect.CallerSensitive not found - continuing with a stub. #11679

JDK 11 - Class sun.reflect.CallerSensitive not found - continuing with a stub. #11679

Comments

Sciss commented Aug 13, 2019

SethTisue commented Aug 21, 2019

lrytz commented Aug 23, 2019

Sciss commented Aug 23, 2019

Sciss commented Aug 23, 2019 • edited Loading

retronym commented Aug 27, 2019 • edited Loading

retronym commented Aug 27, 2019

retronym commented Aug 27, 2019 • edited Loading

retronym commented Aug 27, 2019

Sciss commented Sep 23, 2019

lrytz commented Sep 23, 2019

retronym commented Oct 11, 2019

Sciss commented Aug 23, 2019 •

edited

Loading

retronym commented Aug 27, 2019 •

edited

Loading

retronym commented Aug 27, 2019 •

edited

Loading