Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IllegalAccessError regression in internal testing introduced by #11708 #12285

Open
pshipton opened this issue Mar 25, 2021 · 29 comments
Open

IllegalAccessError regression in internal testing introduced by #11708 #12285

pshipton opened this issue Mar 25, 2021 · 29 comments
Labels
comp:vm project:MH Used to track Method Handles related work test failure
Milestone

Comments

@pshipton
Copy link
Member

pshipton commented Mar 25, 2021

There is an IllegalAccessError regression in internal testing, caused by #11708
Internal reference 145127.

@pshipton pshipton changed the title Regression in internal testing introduced by https://github.com/eclipse/openj9/pull/11708 Regression in internal testing introduced by #11708 Mar 25, 2021
@pshipton pshipton changed the title Regression in internal testing introduced by #11708 IllegalAccessError regression in internal testing introduced by #11708 Mar 25, 2021
@tajila
Copy link
Contributor

tajila commented Mar 30, 2021

@DanHeidinga Is there a reason we need the lookup.accessCheckArgRetTypes(type); in cases 5, 6, 7, 8, 9 in https://github.com/eclipse/openj9/blob/master/jcl/src/java.base/share/classes/java/lang/invoke/MethodHandleResolver.java#L443-L468 ?

The spec states that:

Resolution of an unresolved symbolic reference to a method handle is more complicated. Each method handle resolved by
 the Java Virtual Machine has an equivalent instruction sequence called its bytecode behavior, indicated by the method
 handle's kind. The integer values and descriptions of the nine kinds of method handle are given in Table 5.4.3.5-A. 

and

R (the symbolic reference) is resolved. This occurs as if by field resolution (§5.4.3.2) when MH's bytecode behavior is kind 1, 
2, 3, or 4, and as if by method resolution (§5.4.3.3) when MH's bytecode behavior is kind 5, 6, 7, or 8, and as if by interface
 method resolution (§5.4.3.4) when MH's bytecode behavior is kind 9.

see https://docs.oracle.com/javase/specs/jvms/se14/html/jvms-5.html#jvms-5.4.3.5

When resolving a method, we never perform access checks on the param/ret types themselves, we only check that the method is visible to the caller. This means our current MH behaviour differs from our bytecode behaviour

@DanHeidinga
Copy link
Member

There is an IllegalAccessError regression in internal testing, caused by #11708
Internal reference 145127.

Is the IAE thrown by Hotspot as well? It would be good to confirm that the original test is "correct".

The change was introduced here: be44482 as part of #2675 to make us compatible with the RI.

As to Tobi's question, the access check is equivalent to the class loader constraint checks that happen when looking up a java method - see for example the code in lookupmethod.c that handles clconstriant calls

@tajila
Copy link
Contributor

tajila commented Mar 30, 2021

Here is a small example that demonstrates the difference between hotspot and j9. Given the following:

package p;

public class A {
	protected static enum TEST { ONE }
}

and

package q;

public class B extends A {
	public static void main(String[] args) throws WrongMethodTypeException, ClassCastException, Throwable {
		TEST[] arr = new TEST[1];
		Class<?> arrClass = arr.getClass();
		
		//1) is TEST[] visible
		try
		{
			Lookup l = MethodHandles.lookup();
			Class<?> accessClass = l.accessClass(arrClass);
		}catch(Exception e)
		{
			System.out.println(e);
		}
		
		//2) Byte code behaviour with TEST[]
		meth(null);
		
		//3) MH lookup with TEST[]
		MethodHandle mh = MethodHandles.lookup().findStatic(B.class, "meth", MethodType.methodType(void.class, TEST[].class));
		mh.invoke(null);
		
		//4) Indy behvaiour with TEST[]
		Consumer<TEST[]> f = B::meth;
		f.accept(null);
	
	}
	
	public static void meth(TEST[] arr) {
		System.out.println("called method");
	}
}

With hotspot the result is:

1)
java.lang.IllegalAccessException: access violation: class [Lp.A$TEST;, from q.B (unnamed module @60215eee)
2)
called method
3)
called method
4)
called method

with J9 its:

1)
java.lang.IllegalAccessException: Class 'q.B' no access to: class '[Lp.A$TEST;'
2)
called method
3)
called method
4)
Exception in thread "main" java.lang.IllegalAccessError: Class 'q.B' no access to: class '[Lp.A$TEST;'
	at java.base/java.lang.invoke.MethodHandleResolver.sendResolveMethodHandle(MethodHandleResolver.java:338)
	at java.base/java.lang.invoke.MethodHandleResolver.getCPMethodHandleAt(Native Method)
	at java.base/java.lang.invoke.MethodHandleResolver.getAdditionalBsmArg(MethodHandleResolver.java:465)
	at java.base/java.lang.invoke.MethodHandleResolver.resolveInvokeDynamic(MethodHandleResolver.java:229)
	at q.B.main(B.java:34)
Caused by: java.lang.IllegalAccessException: Class 'q.B' no access to: class '[Lp.A$TEST;'
	at java.base/java.lang.invoke.MethodHandles$Lookup.checkClassAccess(MethodHandles.java:543)
	at java.base/java.lang.invoke.MethodHandles$Lookup.accessCheckArgRetTypes(MethodHandles.java:832)
	at java.base/java.lang.invoke.MethodHandleResolver.sendResolveMethodHandle(MethodHandleResolver.java:311)
	... 4 more

@tajila
Copy link
Contributor

tajila commented Mar 30, 2021

So you see that the even though hotspot doesnt allow access to TEST[] it does permit one to call a method with TEST[] as a parameter, and this behaviour is consistent with the MH behaviour.

In our case, the only place we differ is in 4) because we perform access checks on all method ret/params. We do not do this in 3).

@tajila
Copy link
Contributor

tajila commented Mar 30, 2021

I think there are two potential problems.

A) should 1) be failing at all? The reason we fail it is because if its an array, we look at the member access flags for the element type which will have protected, rather than the access flags which will have public

B) There is clearly a difference between bytecode behaviour and MH behaviour with both J9 and hotspot, but maybe in case its fine. See alternate example below:

public class B extends A {
	public static void main(String[] args) throws WrongMethodTypeException, ClassCastException, Throwable {
		//Unsafe is not visible
		
		//1) 
		try
		{
			System.out.println("1)");
			Lookup l = MethodHandles.lookup();
			Class<?> accessClass = l.accessClass(Unsafe.class);
		}catch(Throwable e)
		{
			System.out.println(e);
		}
		
		//2) Byte code behaviour with Unsafe
		System.out.println("2)");
		meth(null);
		
		try
		{
			//3) MH lookup with Unsafe
			System.out.println("3)");
			MethodHandle mh = MethodHandles.lookup().findStatic(B.class, "meth", MethodType.methodType(void.class, Unsafe.class));
			mh.invoke(null);
		}catch(Throwable e)
		{
			System.out.println(e);
		}

		try
		{
			//4) Indy behvaiour with Unsafe
			System.out.println("4)");
			Consumer<Unsafe> f = B::meth;
			f.accept(null);
		}catch(Throwable e)
		{
			System.out.println(e);
		}

	
	}
	
	public static void meth(Unsafe unsafe) {
		System.out.println("called method");
	}
}

With hotspot:

1)
java.lang.IllegalAccessError: class q.B (in unnamed module @0x4fccd51b) cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @0x4fccd51b
2)
called method
3)
java.lang.IllegalAccessError: class q.B (in unnamed module @0x4fccd51b) cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @0x4fccd51b
4)
java.lang.IllegalAccessError: class q.B (in unnamed module @0x4fccd51b) cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @0x4fccd51b

with j9

1)
java.lang.IllegalAccessError: Class q/B(unnamed module 0x00000007FFF192B0) can not access class jdk/internal/misc/Unsafe(module java.base) because module module java.base does not export package jdk/internal/misc to module unnamed module 0x00000007FFF192B0
2)
called method
3)
java.lang.IllegalAccessError: Class q/B(unnamed module 0x00000007FFF192B0) can not access class jdk/internal/misc/Unsafe(module java.base) because module module java.base does not export package jdk/internal/misc to module unnamed module 0x00000007FFF192B0
4)
java.lang.IllegalAccessError: Module 'unnamed module @7c1a9b8c' no access to: package 'jdk.internal.misc' which is not exported by module 'java.base' to module 'unnamed module @7c1a9b8c'

So 2) passes but 3) and 4) fail.

@tajila
Copy link
Contributor

tajila commented Mar 30, 2021

I think the correct behaviour in the first example is that 1, 2, 3 & 4 should pass, this implies that the hotspot behaviour is wrong (Note that in the second example there is consistency between 1, 3 & 4). But if the hotspot behaviour is correct then J9 shouldn't treat 3) and 4) differently.

In the second example, I may just be misunderstanding the spec. My understanding is that 2, 3 & 4 should behave the same according to the "as bytecode behaviour" rules.

@DanHeidinga
Copy link
Member

@tajila are you comparing equivalent JDK11 releases in these tests? Or some other release?

@tajila
Copy link
Contributor

tajila commented Mar 30, 2021

J9

openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.11+7-202103241355)
Eclipse OpenJ9 VM AdoptOpenJDK (build master-8cd58be62, JRE 11 Linux amd64-64-Bit Compressed References 20210324_958 (JIT enabled, AOT enabled)
OpenJ9   - 8cd58be62
OMR      - 165a4a817
JCL      - 322b40f539 based on jdk-11.0.11+7)

hotspot

openjdk version "11-internal" 2018-09-25
OpenJDK Runtime Environment (build 11-internal+0-adhoc.tobi.openjdk-jdk11)
OpenJDK 64-Bit Server VM (build 11-internal+0-adhoc.tobi.openjdk-jdk11, mixed mode)

@tajila
Copy link
Contributor

tajila commented Mar 30, 2021

Ill try a more recent hotspot

@tajila
Copy link
Contributor

tajila commented Mar 30, 2021

The behaviour is the same with

openjdk version "11.0.10" 2021-01-19
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.10+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.10+9, mixed mode)

@DanHeidinga
Copy link
Member

@tajila what does a JDK16 hotspot do with the Consumer<TEST[]> f = B::meth; test? Trying to triangulate whether this behaviour eventually matches or continues to be different in the future

@DanHeidinga
Copy link
Member

If we assume the RI is correct, then I'd look at why we're failing example 1, case 4. My guess would be that our checks are using the wrong access bits - ie: protected rather than public for the array component class.

@tajila
Copy link
Contributor

tajila commented Mar 30, 2021

what does a JDK16 hotspot do with the Consumer<TEST[]> f = B::meth; test?

JDK16 does the same as jdk11

if we assume the RI is correct, then I'd look at why we're failing example 1, case 4

We are failing it because of the way we've implemented getClassModifiers

		private static int getClassModifiers(Class<?> cls) {
			int modifiers = 0;
			/* Use Reflection.getClassAccessFlags to get the actual ROM Class modifiers
			 * instead of the attribute flags for innerclasses
			 */
			if (cls.isPrimitive() || cls.isArray()) {
				modifiers = cls.getModifiers();
			} else {
				modifiers = Reflection.getClassAccessFlags(cls);
			}

			return modifiers;
		}

For arrays cls.getModifiers(); is going to get the member access flags from the element class. Reflection.getClassAccessFlags(cls); just returns the access flags.

The thing is if we change the behaviour here, it is also going to change the behaviour of Lookup::accessClass() since we are performing a similar check there. So thats why I'm wondering whether the hotspot is correct to fail example 1 case 1.

@fengxue-IS
Copy link
Contributor

fengxue-IS commented Mar 30, 2021

@tajila @DanHeidinga
I wrote the following testcase to check if the MH arguments are being validated during resolution:
c/C.java

  1 package c;
  2 import java.lang.invoke.*;
  3 import java.lang.invoke.MethodHandles.*;
  4 
  5 public class C {
  6     private enum pType { X }
  7 
  8     public static MethodType mt() {
  9         return MethodType.methodType(void.class, pType.class);
 10     }
 11 
 12     public static MethodType mt2() {
 13             return MethodType.methodType(pType.class, Object.class);
 14     }
 15 
 16     public static pType test2(Object o) {
 17         return pType.X;
 18     }
 19 
 20     public static Object arg() {
 21         pType a = pType.X;
 22         return (Object)a;
 23     }
 24 
 25     public static void test(pType p) {}
 26 
 27 }

c2/Caller.java

  1 package c2;
  2 
  3 import java.lang.invoke.*;
  4 import java.lang.invoke.MethodHandles.*;
  5 import c.C;
  6 import java.util.function.*;
  7 
  8 public  class Caller {
  9     public static void testCall() throws Throwable {
 10         Lookup l = MethodHandles.lookup();
 11         MethodHandle mh = l.findStatic(C.class, "test2", C.mt2());
 12         mh.invoke(C.arg());
 13     }
 14 
 15     public static void main(String[] args) throws Throwable {
 16         testCall();
 17         C.test2(C.arg());
 18         Consumer<Object> f = C::test2;
 19         f.accept(C.arg());
 20     }
 21 }

result:

../OpenJDK11/bin/java c2.Caller 
Exception in thread "main" java.lang.IllegalAccessError: failed to access class c.C$pType from class c2.Caller (c.C$pType and c2.Caller are in unnamed module of loader 'app')
	at c2.Caller.main(Caller.java:18)

RI's behavior does match the requirement for arg check, I reviewed the VM specification and found:

5.4.3.5 (under point 3)
Resolution occurs as if of unresolved symbolic references to classes and interfaces whose names correspond to each type in A*, and to the type T, in that order.

I believe the reason that example 1, 3/4 are inconsistent is because RI have 2 different version of the access check (one in Java with is used by the MH APIs: ie example 1, and an native impl that is used during vm resolution: ie example 4) where as OpenJ9 shares the same access check API between Java and native code.

Given the spec/bytecode, my understanding is that the native access check impl from RI correctly matches the spec and is what we should be following.

as to example 1 the Java API doc:

accessClass
public Class accessClass​(Class targetClass) throws IllegalAccessException
Determines if a class can be accessed from the lookup context defined by this Lookup object. The static initializer of the class is not run.
The lookup context here is determined by the lookup class and the lookup modes.
Parameters:
targetClass - the class to be access-checked
Returns:
the class that has been access-checked
Throws:
IllegalAccessException - if the class is not accessible from the lookup class, using the allowed access modes.
SecurityException - if a security manager is present and it refuses access

since TEST[] can be accessed in the test method (array access is based on component type access), I think this is a bug on RI's behavior?

@fengxue-IS
Copy link
Contributor

what does a JDK16 hotspot do with the Consumer<TEST[]> f = B::meth; test?

JDK16 does the same as jdk11

if we assume the RI is correct, then I'd look at why we're failing example 1, case 4

We are failing it because of the way we've implemented getClassModifiers

		private static int getClassModifiers(Class<?> cls) {
			int modifiers = 0;
			/* Use Reflection.getClassAccessFlags to get the actual ROM Class modifiers
			 * instead of the attribute flags for innerclasses
			 */
			if (cls.isPrimitive() || cls.isArray()) {
				modifiers = cls.getModifiers();
			} else {
				modifiers = Reflection.getClassAccessFlags(cls);
			}

			return modifiers;
		}

For arrays cls.getModifiers(); is going to get the member access flags from the element class. Reflection.getClassAccessFlags(cls); just returns the access flags.

The thing is if we change the behaviour here, it is also going to change the behaviour of Lookup::accessClass() since we are performing a similar check there. So thats why I'm wondering whether the hotspot is correct to fail example 1 case 1.

Based on the javap output, this is similar to RI's impl and should have same result

@fengxue-IS
Copy link
Contributor

fengxue-IS commented Mar 30, 2021

I've created #12331 which changes the current getClassModifiers to match RI's native impl.

@DanHeidinga
Copy link
Member

@fengxue-IS that leaves us inconsistent with the RI for the accessClass case. I think we need to split J9's behaviour so accessClass does what it does today and the findX calls match the native impl

@DanHeidinga
Copy link
Member

We should also put together a straightfoward bug report and work with the RI to have accessClass made consistent

@babsingh
Copy link
Contributor

babsingh commented Apr 6, 2021

(6 Apr 21) OpenJ9 community call update: An OpenJDK bug report will be opened soon. We will need to match the RI for the 0.26 release in case the bug report does not get addressed in our favor before the release. To match the RI, the behaviour needs to be split between the MethodHandles API and MethodHandleResolver paths. fyi @fengxue-IS.

@babsingh
Copy link
Contributor

OpenJDK bug report: https://bugs.openjdk.java.net/browse/JDK-8266269

@fengxue-IS
Copy link
Contributor

Update: https://bugs.openjdk.java.net/browse/JDK-8266269 have been resolved in OpenJDK for Java 17+.

@DanHeidinga should we revert #12371 and continue with the fix in #12331?

@DanHeidinga
Copy link
Member

Yes, but as far as I can tell JDK-8266269 is only merged into JDK17+ so we should wait to merge to change our behaviour until we're sure that the 0.27 branches won't be refreshed from master.

fyi @pshipton for awareness to ensure we don't refresh the release branch once this is addressed

@fengxue-IS
Copy link
Contributor

Also, Java 17+ will use OJDK MH by default so JDK-8266269 should be picked up by 17+ builds automatically, do we need to backport JDK-8266269 to the 8 & 11 extensions repo in the future when we decide to switch to OJDK MH on 8/11?

@pshipton
Copy link
Member Author

Is there an extensions change, or just #12371? We won't be refreshing the 0.27 branch from master in the openj9 repo. We may pull other releases (IBM) from master before OJDK MH is backported to jdk8.

@fengxue-IS
Copy link
Contributor

JDK-8266269 is a OJDK MH change that is only available for Java 17+ in the extensions repo
#12331 is the equivalent changes for OpenJ9 MH impl
#12371 is a workaround that matches the "incorrect" openjdk behavior for OpenJ9

I guess the questions are:

  1. Do we want to update the OpenJ9 MH behavior for pre-Java 17 to match the correct spec behavior that is only fixed in Java17+ by RI
  2. Do we want to update the OpenJ9 w/ OJDK MH behavior for pre-Java 17 to match the correct spec behavior which I don't think is fixed by RI
  3. Do we want to fix 0.27 to match the correct spec behavior (I don't think that is fixed in RI)

@pshipton
Copy link
Member Author

My vote is to keep it as-is for the 0.27 release, and match the RI for 8, 11, 16.

Perhaps JDK-8266269 will be backported for the OpenJDK release in Oct. We can decide if we want to make any updates for this release. I'd prefer not to put anything in the extensions we don't want in 0.27, as usually I do refresh the 0.27 extensions branches with the latest, as long as that is feasible/possible.

@DanHeidinga
Copy link
Member

My vote is to keep it as-is for the 0.27 release, and match the RI for 8, 11, 16.

I agree with Peter, assuming there aren't any tests in 8/11 that will fail due to this.

Perhaps JDK-8266269 will be backported for the OpenJDK release in Oct.

We should propose the OJDK backport now if we want it for Oct.

@tajila tajila added the project:MH Used to track Method Handles related work label Jul 27, 2021
@fengxue-IS
Copy link
Contributor

I don't see this being backported anytime soon, we should move this to 0.29?
@JasonFengJ9 FYI

@JasonFengJ9
Copy link
Member

JDK-8266269 is available for JDK17+. OJDK MH has been enabled at openj9-staging branch which is expected to be promoted into openj9 branch for JDK 17 GA. No action required for JDK17.
Agreed to move this out of 0.28 (Java 17).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:vm project:MH Used to track Method Handles related work test failure
Projects
None yet
Development

No branches or pull requests

6 participants