Duplicate symbol on bridge generation

[odersky]

We get a duplicate generated symbol for fromIterableWithSameElemType in class ArrOps when compiling iarray.scala:

import scala.reflect.ClassTag

class Arr[T](private val underlying: scala.Array[T]) extends AnyVal

trait SeqMonoTransforms[+A, +Repr] extends Any {
  protected[this] def fromIterableWithSameElemType(): Repr
}

class ArrOps[A](val xs: Arr[A]) extends AnyRef with SeqMonoTransforms[A, Arr[A]] {
  def fromIterableWithSameElemType(): Arr[A] = xs
}

object Test {
  def main(args: Array[String]) =
    println(new ArrOps(new Arr(Array(1, 2, 3))).fromIterableWithSameElemType)
}

If we compile this with -Ycheck:erasure, we get:

exception occurred while compiling iarray.scala
Exception in thread "main" class dotty.tools.dotc.reporting.diagnostic.messages$Error at iarray.scala:9: method fromIterableWithSameElemType is already defined as method fromIterableWithSameElemType: ()ErasedValueType(Arr, Object)
(the definitions have matching type signatures)
at dotty.tools.dotc.reporting.Reporting$class.error(Reporter.scala:85)
at dotty.tools.dotc.core.Contexts$Context.error(Contexts.scala:57)
at dotty.tools.dotc.typer.Checking$$anonfun$checkDecl$1$1.doubleDefError$1(Checking.scala:548)
at dotty.tools.dotc.typer.Checking$$anonfun$checkDecl$1$1.apply(Checking.scala:551)
at dotty.tools.dotc.typer.Checking$$anonfun$checkDecl$1$1.apply(Checking.scala:540)
at scala.collection.immutable.List.foreach(List.scala:381)

[odersky]

BTW: Here is scalac's behavior, which is also wrong (see SI-10150)

iarray.scala:10: error: bridge generated for member method fromIterableWithSameElemType: ()Arr[A] in class ArrOps
which overrides method fromIterableWithSameElemType: ()Repr in trait SeqMonoTransforms
clashes with definition of the member itself;
both have erased type ()Object
def fromIterableWithSameElemType(): Arr[A] = xs
^
one error found

[DarkDimius]

In this example there indeed should be two methods with same signature with our current compilation scheme:

• one inherited from SeqMonoTransforms that returns Repr,
• another from ArrOps that returns erazed version of Arr[A] which is A.

Both those methods have same names and their entire JVM signature is the very same.

This can be fixed by making value-classes method be name-mangled, ie method that returns Arr[A] will have a name fromIterableWithSameElemType$vc.

[DarkDimius]

Note that you cant skip generation of the bridge, as callsites that observe the instance of the class as SeqMonoTransforms would expect fromIterableWithSameElemType:()Object to return Repr, while callsites that observe ArrOps would expect fromIterableWithSameElemType:()Object to return A.
As A and ArrOps are unrelated the same method can't satisfy both contracts.

[odersky]

@DarkDimius I don't understand. According to Scala's typing rules, these methods are in an overriding relationship (both scalac and dotc agree on this). Furthermore, they have the same erasure, so overriding also holds on the JVM level. So, no bridge should be generated.

[odersky]

another from ArrOps that returns erazed version of Arr[A] which is A.

No, the erasure of Arr[A] is Object, not A.

[DarkDimius]

@odersky,

No, the erasure of Arr[A} is Object, not A.

What I've ment here was observable semantics, not erasure:

• SeqMonoTransforms defines a method fromIterableWithSameElemType that semantically should return Repr
• ArrOps defines a methodfromIterableWithSameElemType that should semantically return an scala.Array[A](this method is value-class unboxed version).

Those two methods have different semantics, but the same erazed form, which means that in runtime they will be dispatch to same location that should follow both semantics, and this isn't possible as semantics contradict.

[odersky]

ArrOps defines a methodfromIterableWithSameElemType that should semantically return an A(this method is value-class unboxed version).

No, please have a look at the signature of the value class again. This is simply wrong.

[DarkDimius]

No, please have a look at the signature of the value class again. This is simply wrong.

Fixed a typo, but the message stays the same
It returns the underlying of Arr[T] which is scala.Array[T] with unbounded T, that is erazed to Object.
the same method fromIterableWithSameElemType()Object should both return Repr which is bridge returning Arr[A] and scala.Array[A], which is a value-class underlying

[odersky]

But Repr is scala.Array[T]! I still don't understand. Are you claiming that the two fromIterableWithSameElementType methods are not in an overriding relationship?

[DarkDimius]

Consider this, simpler example:

class A(a: Any) extends AnyVal
abstract class Super[T] {
 def id(arg: T): T
}

class Child extends Super[A] {
  def id(a: A): A = a
}

how many methods should there be defined and implemented in Child(including bridges)?
Given our semantics, there should be 3:

• the one defined in source A => A
• the unboxed version of the previous one Underlying(A) => Underlying(A), where Underlying is a function that given a type returns underlying type of a value class, Underlying(A) == Object
• the method that bridges to Super, which is T => T, which, given T is unbounded is erazed to Object => Object.

Note that two last methods clash on erasure.

[DarkDimius]

But Repr is scala.Array[T]!

Repr isn't scala.Array[T], Repr is Arr[A], whose unboxed version would indeed have been scala.Array[T] in your example, but as super class is a generic location you can't let unboxed version escape there without breaking semantics.

[odersky]

OK, can you come up with a counter example that breaks semantics? Just take #1906 and construct a crashing scenario. I think that would help me understand what you mean.

[DarkDimius]

class Arr[T](val underlying: scala.Array[T]) extends AnyVal {}

abstract class SeqMonoTransforms[+A, +Repr] {
  protected[this] def fromIterableWithSameElemType(): Repr
  def getFIWSET: Repr = fromIterableWithSameElemType
}

class ArrOps[A](val xs: Arr[A]) extends SeqMonoTransforms[A, Arr[A]] {
  def fromIterableWithSameElemType(): Arr[A] = xs
}

object Test {
  def main(args: Array[String]) = {
    val t = new ArrOps(new Arr(Array(1, 2, 3)))
    val t2 = t.getFIWSET
  }
}

Exception in thread "main" java.lang.ClassCastException: [I cannot be cast to Arr
	at Test$.main(arr-erasure.scala:15)
	at Test.main(arr-erasure.scala)

[DarkDimius]

@odersky this is after compiling with #1906 applied. Whithout it, wouldn't compile. Neither does it compile in scalac.

[odersky]

@DarkDimius Ah, that helps! What do you get when compiling with master?

[odersky]

Thanks for the counter-example. I think I get it now. The problem is that erasure assumes that if a value x is of type Object and it needs to be cast to a value class such as Arr[T], then x must have the boxed representation of Arr[T] at runtime. But without adding the bridge we return directly the unboxed representation and that leads to the classcast error.

But we still need to detect the problem (it seems that right now dotc does not).

Also, would it be possible to have a more refined test at runtime? I see we currently generate this:

          val ev$1: Object = t.getFIWSET()
          if ev$1.eq(null) then null else 
            ev$1.asInstanceOf[Arr].underlying()

Could we instead generate this?

          val ev$1: Object = t.getFIWSET()
          if ev$1.isInstanceOf[Arr] then ev$1.asInstanceOf[Arr].underlying()
          else ev$1 

And, would that solve the problem?

[DarkDimius]

Could we instead generate this?

          val ev$1: Object = t.getFIWSET()
          if ev$1.isInstanceOf[Arr] then ev$1.asInstanceOf[Arr].underlying()
          else ev$1 

Unforturnatelly you cant(what if one class is a sublass of another)?

There's a small change to the compilation scheme of value classes that would permit the code you want to compie: we need to name-mangle the name of fromIterableWithSameElemType method that returns the underlying value of value class

[odersky]

Unforturnatelly you cant(what if one class is a sublass of another?

You mean, the value class is a subclass of its underlying class? Do we have usecases for this? Could we simply forbid it?

[DarkDimius]

@odersky, similarly, you cant forbid it statically(without a closed world), it can be not be observed locally. Consider my example(with Super and Child). The underlying: Object of A can be a boxed A itself.

class A(a: Any) extends AnyVal
abstract class Super[T] {
 def id(arg: T): T
}

class Child extends Super[A] {
  def id(a: A): A = a
}

new Child.id(new A(new A(null))

[odersky]

I retract the idea. I believe value classes over Any are indeed rare, so maybe it would be OK to have

class A(a: Any) extends AnyVal

not compile. But parametric value classes such as

 implicit final class Ensuring[A](private val self: A) extends AnyVal 

are common, and they pose the same problem.

Can you explain in more detail the name mangling idea?

[DarkDimius]

Can you explain in more detail the name mangling idea?

The problem arises from the fact that two methods with different semantics have same signature. Signature includes name an argument&result type. We can make them be unique if we change the name.

The proposed scheme would be:
given a method with name <name> in case it:

• takes an unboxed value class as argument in position i we would append $va${i} to the name.
• returns a unboxed value class, we would append $vr to the name.

This means that a fromIterableWithSameElemType: Arr[A] will be compiled into three methods that don't clash:

• fromIterableWithSameElemType()Object defined by a superclass
• fromIterableWithSameElemType()Arr[A] defined by a class itself
• fromIterableWithSameElemType$vr()Underlying[Arr[A]] which is a unboxed shortcut method.

Similarly, in my example id(a: A): A will be compiled into three methods:

• id(Object)Object defined by a superclass
• id(A)A defined by a class itself
• id$va1$vr(Underlying[A])Underlying[A] which is a unboxed shortcut method.

This can either be implemented in Erasure or by a phase preceding erasure(I'd prefer this). he phase would also change callsites that know that they call a method that takes an unboxed value class to dispatch to freshly-created method.
Note that this can also be done under separate compilation. T

[odersky]

What happens if we have several unboxed arguments and an unboxed return type? Would we have to generate several methods? If not, why not simply append a $vr to the method name, if it has at least one unboxed argument or an unboxed return type?

[retronym]

I prefer @DarkDimius's proposal (to treat optimized calls to methods with value classes and rewrite calls sites, ala specialization) to the current approach. I think we discussed before the release of Scala 2.10, but it was seen as to hard to implement in scalac.

One thing to keep in mind is how to make these rewrites for value classes, specialization, and implicit function types composable. Ideally they could be unified somehow.

[odersky]

Yes, I am also coming around to think that name mangling is a good solution, if we can keep it simple & composable.

One thing to keep in mind is how to make these rewrites for value classes, specialization, and implicit function types composable. Ideally they could be unified somehow.

Which aspect of implicit function types did you have in mind here?

[odersky]

What happens if we have several unboxed arguments and an unboxed return type? Would we have to generate several methods? If not, why not simply append a $vr to the method name, if it has at least one unboxed argument or an unboxed return type?

I think I can answer my own question: Because several subclasses might have overriding methods with different value class parameters or results. We need a more fine-grained naming scheme to prevent accidental clashes between them.

[DarkDimius]

We need a more fine-grained naming scheme to prevent accidental clashes between them.

As value classes are final, every argument may become value class only once(and will then stay a value class).
Scheme that I've proposed has a good property that it is monotonic: on every level of hierarchy we will now precisely how many of arguments&return type are value classes and we'll be able to dispatch to the best version.

This ensures absence of clashes as well as some sort of optimality.

[retronym]

Which aspect of implicit function types did you have in mind here?

I recall part of the proposal that rewrites foo(...).apply(...) to foo$xxx(...)(...) ?

[odersky]

@retronym Ah, res, foo$direct. Yes, we need to explain in what order these are applied.

[odersky]

As a first measure, we still need to detect the duplicate bridge issue (scalac does it but dotty does not). @DarkDimius, can you look into this?

[odersky]

The missing error message is fixed by aa7ee91. The possible name mangling to avoid the error is left for future work.