Incorrect bytecode generated for StringBuilder.max #4214
Comments
|
Imported From: https://issues.scala-lang.org/browse/SI-4214?orig=1 |
|
@paulp said: (See also #4067 which I was hoping this would be a duplicate of, but is not.) |
|
@axel22 said: |
|
@paulp said: |
|
@SethTisue said: |
|
@odersky said: |
|
@odersky said: |
|
@SethTisue said: |
|
@paulp said: trait T[A] {
def f(): A
}
// the generic signature spec doesn't allow for parameterizing
// on primitive types, so this cannot remain Char. However
// translating it to Character, as was done, also has issues.
class C extends T[Char] {
def f(): Char = 'a'
}
// Note that neither of the signatures for f, the implementation
// or the bridge method, matches the type parameter.
Generic interfaces in class:
T<java.lang.Character>
Generic signatures:
public char C.f()
public java.lang.Object C.f()After this commit, primitive type parameters are translated into |
|
Ramnivas Laddad (ramnivas) said: |
|
@paulp said:
Yeah, I don't know what's going on with the maven deploy failing but I think I woke up the appropriate parties today. You can wait until that's fixed or download a nightly from http://www.scala-lang.org/node/212/distributions . |
|
@paulp said:
FYI now there are up to date snapshots, and any followup report positive or negative would be useful. |
|
Ramnivas Laddad (ramnivas) said: |
|
@paulp said:
A couple days back is probably too far, as I was still wrestling this until fairly recently. Here is what the signature of minBy for instance looks like in the current nightly; I think you will only see the error above if you have an older build. scala> classOf[StringBuilder].getMethods.filter(_.getName == "minBy").head.toGenericString
res0: java.lang.String = public <B> char scala.collection.mutable.StringBuilder.minBy(scala.Function1<java.lang.Object, B>,scala.math.Ordering<B>) |
|
Ramnivas Laddad (ramnivas) said: I tried just now (scala-library-2.9.0-20110309.025033-152.jar and scala-compiler-2.9.0-20110309.025033-152.jar) and still see the same problem. I can share the project if you want to try reproduce it (it is very small project). Just let me know. |
|
@paulp said:
Well that's a disappointment.
I certainly would like to fix the problem, but I am doubtful that anything involving eclipse and aspectj (neither of which I use and both of which I have terrible luck with) is amenable to straightforward reproduction. But please send me the project anyway. I'm [email protected] if you don't want to attach something here. |
|
@paulp said: |
|
Ramnivas Laddad (ramnivas) said: I sent you the project. I have additional input from the AspectJ lead. Problem still looks like StringBuilder. The signature of minBy (and similarly maxBy) in the bytecode is: public Object minBy(scala.Function1 f, scala.math.Ordering cmp) but the generic signature attribute for it is: <B:Ljava/lang/Object;>(Lscala/Function1<Ljava/lang/Object;TB;>;Lscala/math/Ordering<TB;>;)C The trailing 'C' indicating the return type is of type primitive char
Here is a crude reflection test program - it discovers the bytecode import java.lang.reflect.Method;
import java.lang.reflect.Type;
import scala.collection.mutable.StringBuilder;
public class Main {
public static void main(String[] args) {
StringBuilder sb = new StringBuilder();
Method[] ms = StringBuilder.class.getDeclaredMethods();
for (Method m : ms) {
if (m.getName().equals("minBy")) {
Class<?> c = m.getReturnType();
System.out.println("Bytecode return type: " + c);
Type t = m.getGenericReturnType();
System.out.println("Generic return type : " + t);
if (t instanceof Class) {
if (((Class) t).isPrimitive() && !c.isPrimitive()) {
System.out.println("Incompatible");
}
}
}
}
}
} |
|
@paulp said: |
|
@odersky said: |
|
@odersky said: |
|
@paulp said: import scala.collection.mutable
object Test {
def f[A, B] = new mutable.HashMap[A, B] { }
def g[A] = new mutable.HashMap[A, Int] { }
def h[A] = new mutable.HashMap[A, Int] { override def default(k: A) = 0 }
}Methods f and g are marked bridge as expected. public B scala.collection.mutable.HashMap.default(A) (bridge) in Test$$$$anon$$2
public B scala.collection.mutable.HashMap.default(A) (bridge) in Test$$$$anon$$3The third class gets these: bridge=false public int Test$$$$anon$$3.default(A)
bridge=true public java.lang.Object Test$$$$anon$$3.default(java.lang.Object)But Int is not a valid return type for the non-bridge method, because all code based on HashMap will be looking for an object on the stack. As far as I can see, there isn't any valid way to implement a generically defined method and have primitives appearing in the signature in type parameter positions. My only idea (this is what I was talking about at villars) is to implement additional bridges, like: public java.lang.Integer Test$$$$anon$$3.default(java.lang.Object)But that may be infeasible. So the signature bombs are still in there. I found this one in SeqLike. private def occCounts[B](sq: Seq[B]): mutable.Map[B, Int] = {
val occ = new mutable.HashMap[B, Int] { override def default(k: B) = 0 }
for (y <- sq.seq) occ(y) += 1
occ
} |
|
@odersky said: "But Int is not a valid return type for the non-bridge method, because all code based on HashMap? will be looking for an object on the stack." I thought, that's what the bridge method is for? -- Martin |
|
Ramnivas Laddad (ramnivas) said: Thank you Paul and Martin. Paul, I emailed you separately around the AspectJ issue that had been taken care of. |
|
@odersky said: |
|
I came across this bug today and was trying to reconstruct how we ended up in this point in the design space. I'll try to recap (below) and allow myself a @paulp question for the effort.
This seems like a good idea to me
Are you referring to the increase in bytecode, or something else? In terms of added bytecode, it doesn't seem too bad to add another bridge method (one constant pool entry and one 5-byte method). EDIT: I could imagine that there's another issue with this, when you complicate your example a little bit, wrapping the type parameter in a type constructor: This would require bridge methods whose signatures erase to the same types (one for Ignoring that snag, let me recap my understanding of the issue in the straightforward case: Ideally, a Scala "auto-boxing primitive type" such as scalac currently (2.12.3) compiles this to: It would be nice if we could instead have: I believe this is allowed at the bytecode level (overloading on result type), but, as I said above, it breaks down when you consider return types that include the type parameter in an erased position. |
|
I have no idea, I am afraid. I imagine "infeasible" means there's a lot of unknowns consequences of changing bridge generation. |
|
Thanks, here's one show-stopper, I think (included as a sneaky EDIT above): I could imagine that there's another issue with this, when you complicate your example a little bit, wrapping the type parameter in a type constructor: This would require bridge methods whose signatures erase to the same types (one for |
|
this matches my recollection of past discussions of what the show-stopper was. |
|
That doesn't matter. In that case only emit the bridge for |
|
Ah, good point, thanks! |
|
Hmm, but wouldn't that only be valid for covariant type constructors? |
|
Nope. The JVM doesn't know about variance. It won't care. |
|
Yes, but isn't it unsound in general to put something in an invariant box at, say, EDIT: I guess you're shielded from the unsoundness because you only get access to these bridges indirectly from well-typed code, they don't provide more surface area. |
|
The JVM doesn't know about variance, but java compilers do. This difficulty all originated with people trying to use ecj and javac to compile java source against scala bytecode. My guess is that everyone who cares about interop with java has learned they have to write the java-facing part in java if they want it to work. |
I probably meant the increase in bytecode, but I might have been picturing 2^N bridge methods for N > 1 when you have e.g. |
|
Yes, but the combinatorial explosion doesn't happen because all combinations erase to the same type -- ignoring Arrays and specialization. (Specialization is an orthogonal concerns, at least for the purpose of reasoning about the added bytecode due to more precise generic signatures). Assuming Sébastien is right (and he usually is ;-)), maybe we can improve the situation after all! |
=== Background ===
While trying to weave a simple AspectJ aspect is a Scala compiler jar,
we got a puzzling error that fails the AspectJ weaver if we use
Scala version 2.8.1 or 2.9.0-SNAPSHOT, but not 2.8.0.
Upon closer inspection (see below) there appears to be a problem
due to inconsistent bytecode generated by the Scala compiler.
This not only fails with AspectJ, but also
crashes the Eclipse compiler in a plain (non-AspectJ) Java project.
Here is the analysis based on input from AspectJ project lead Andy Clement.
The issue is with StringBuilder.max (with 2.8.1)
=== What steps will reproduce the problem ? ===
Here is the bytecode generated by the Scala compiler:
So it calls another method then finishes with an ARETURN (i.e.
returning an object, not a primitive). Now it is generic so we can
lookup the original signature using that attribute, the generic
signature is:
and that says that the method returns a char (the trailing C), so it
should have been declared 'char max(Ordering)' and finish with an
IRETURN.
Now, the compiler may be being clever and looking for the boxing to a
Character, but it seems odd that the generic signature disagrees
with the bytecode signature in that way.
In Scala 2.8.0 it was not a generic method so there is no mismatch in
signatures.
=== Seeing the problem without involving AspectJ ===
If you want to see the Eclipse compiler crashing, create a pure java project,
give it a dependency on the 2.8.1 scala library then try to compile
this:
Eclipse throws up an error message (an Internal compiler error)
I can attach the project if you wish, but as you can see,
there isn't much needed to get the Eclipse compiler to crash.
=== What versions of the following are you using? ===
The text was updated successfully, but these errors were encountered: