AttachAPISanity_SE90 failed intermittently on Windows #1454

terryzuoty · 2018-03-15T19:22:47Z

java -version:

"C:/temp/bld_381273/jdk/bin\\java" -version
openjdk version "9-internal"
OpenJDK Runtime Environment (build 9-internal+0-adhoc.j9build.openjdk)
Eclipse OpenJ9 VM (build cbuild-37b9b7b2, JRE 9 Windows 7 amd64-64 Compressed References 20180315_381273 (JIT enabled, AOT enabled)
OpenJ9   - 37b9b7b2
OMR      - 3dfc56d5
JCL      - 6faae28dcd based on jdk-9+181)

Error Message:

===============================================
Running test AttachAPISanity_SE90_0 ...
===============================================
test with NoOptions
{ "C:/temp/bld_381273/jdk/bin\\java"  -Xcompressedrefs  \
--add-exports=java.base/com.ibm.tools.attach.target=ALL-UNNAMED \
-cp "C:/temp/bld_381273/openj9/test///..//jvmtest\\TestConfig\\resources;C:/temp/bld_381273/openj9/test///..//jvmtest\\TestConfig\\lib\\testng.jar;C:/temp/bld_381273/openj9/test///..//jvmtest\\TestConfig\\lib\\jcommander.jar;C:/temp/bld_381273/openj9/test///..//jvmtest\\Java8andUp\\GeneralTest.jar" \
-Dcom.ibm.tools.attach.enable=yes \
-Djdk.attach.allowAttachSelf=true \
-Dcom.ibm.tools.attach.timeout=15000 \
-Djava.sidecar="--add-exports=java.base/com.ibm.tools.attach.target=ALL-UNNAMED" \
org.testng.TestNG -d "C:/temp/bld_381273/openj9/test/\\TestConfig\\test_output_15211345307487\\AttachAPISanity_SE90_0" "C:/temp/bld_381273/openj9/test///..//jvmtest\\Java8andUp\\testng.xml" -testnames AttachAPISanity \
-groups level.sanity \
-excludegroups d.*.win_x86-64_cmprssptrs,d.*.arch.x86,d.*.os.win,d.*.bits.64,d.*.generic-all; \
if [ $? -eq 0 ] ; then echo "AttachAPISanity_SE90_0""_PASSED"; else echo -e "\nAttachAPISanity_SE90_0""_FAILED\n"; fi; } 2>&1 | tee -a "C:/temp/bld_381273/openj9/test/\\TestConfig\\test_output_15211345307487\\TestTargetResult"
[IncludeExcludeTestAnnotationTransformer] [INFO] exclude file is C:/temp/bld_381273/openj9/test///..//jvmtest\\TestConfig\\resources\\excludes\\latest_exclude_SE90.txt
...
... TestNG 6.14.2 by C�dric Beust (cedric@beust.com)
...

FAILED CONFIGURATION: @BeforeMethod setUp
java.lang.AssertionError: main process did not initialize attach API
	at org.testng.Assert.fail(Assert.java:96)
	at org.openj9.test.attachAPI.AttachAPISanity.setUp(AttachAPISanity.java:105)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-internal/Native Method)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-internal/NativeMethodAccessorImpl.java:62)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-internal/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java.base@9-internal/Method.java:564)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:59)
	at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:455)
	at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:222)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:520)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:716)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:988)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
	at org.testng.TestRunner.privateRun(TestRunner.java:648)
	at org.testng.TestRunner.run(TestRunner.java:505)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
	at org.testng.SuiteRunner.run(SuiteRunner.java:364)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)
	at org.testng.TestNG.runSuites(TestNG.java:1049)
	at org.testng.TestNG.run(TestNG.java:1017)
	at org.testng.TestNG.privateMain(TestNG.java:1354)
	at org.testng.TestNG.main(TestNG.java:1323)

SKIPPED CONFIGURATION: @BeforeMethod setUp
SKIPPED CONFIGURATION: @BeforeMethod setUp
SKIPPED: testAttachToOtherProcess
java.lang.AssertionError: main process did not initialize attach API
	at org.testng.Assert.fail(Assert.java:96)
	at org.openj9.test.attachAPI.AttachAPISanity.setUp(AttachAPISanity.java:105)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-internal/Native Method)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-internal/NativeMethodAccessorImpl.java:62)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-internal/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java.base@9-internal/Method.java:564)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:59)
	at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:455)
	at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:222)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:520)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:716)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:988)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
	at org.testng.TestRunner.privateRun(TestRunner.java:648)
	at org.testng.TestRunner.run(TestRunner.java:505)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
	at org.testng.SuiteRunner.run(SuiteRunner.java:364)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)
	at org.testng.TestNG.runSuites(TestNG.java:1049)
	at org.testng.TestNG.run(TestNG.java:1017)
	at org.testng.TestNG.privateMain(TestNG.java:1354)
	at org.testng.TestNG.main(TestNG.java:1323)

SKIPPED: testAttachToSelf
java.lang.AssertionError: main process did not initialize attach API
	at org.testng.Assert.fail(Assert.java:96)
	at org.openj9.test.attachAPI.AttachAPISanity.setUp(AttachAPISanity.java:105)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-internal/Native Method)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-internal/NativeMethodAccessorImpl.java:62)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-internal/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java.base@9-internal/Method.java:564)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:59)
	at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:455)
	at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:222)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:520)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:716)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:988)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
	at org.testng.TestRunner.privateRun(TestRunner.java:648)
	at org.testng.TestRunner.run(TestRunner.java:505)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
	at org.testng.SuiteRunner.run(SuiteRunner.java:364)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)
	at org.testng.TestNG.runSuites(TestNG.java:1049)
	at org.testng.TestNG.run(TestNG.java:1017)
	at org.testng.TestNG.privateMain(TestNG.java:1354)
	at org.testng.TestNG.main(TestNG.java:1323)

SKIPPED: testAttachToSelfAndOthers
java.lang.AssertionError: main process did not initialize attach API
	at org.testng.Assert.fail(Assert.java:96)
	at org.openj9.test.attachAPI.AttachAPISanity.setUp(AttachAPISanity.java:105)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-internal/Native Method)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-internal/NativeMethodAccessorImpl.java:62)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-internal/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java.base@9-internal/Method.java:564)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:59)
	at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:455)
	at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:222)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:520)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:716)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:988)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
	at org.testng.TestRunner.privateRun(TestRunner.java:648)
	at org.testng.TestRunner.run(TestRunner.java:505)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
	at org.testng.SuiteRunner.run(SuiteRunner.java:364)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)
	at org.testng.TestNG.runSuites(TestNG.java:1049)
	at org.testng.TestNG.run(TestNG.java:1017)
	at org.testng.TestNG.privateMain(TestNG.java:1354)
	at org.testng.TestNG.main(TestNG.java:1323)


===============================================
    AttachAPISanity
    Tests run: 3, Failures: 0, Skips: 3
    Configuration Failures: 1, Skips: 2
===============================================

The text was updated successfully, but these errors were encountered:

DanHeidinga · 2018-03-15T19:24:42Z

@pdbain-ibm Any ideas on why this failing?

terryzuoty · 2018-03-15T19:26:17Z

did a search and found this may be related to #1255 , but they failed at two different tests targets and two different Java files.

* TestNG before 6.14 treats configuration failure as pass, this has been fixed since TestNG 6.14, update TestNG version to 6.14.2 to fix this bug in product build * separate JLM test targets to avoid OOM failure * re-enable tests which are fixed and passing * update exclude file which failed on TestNG 6.14.2 * exclude reattachAfterExit from win since it is not supported on win * disable testSoftMxRemote tests since it failed at setup stage [ci skip] fixed:eclipse-openj9#1367 fixed:eclipse-openj9#1112 Issue:eclipse-openj9#1454 Signed-off-by: Tianyu Zuo <tianyu@ca.ibm.com>

pdbain-ibm · 2018-03-15T20:03:28Z

I'll take a look. One possible problem is the temporary directory may not be accessible.

pdbain-ibm · 2018-03-15T20:26:58Z

@TianyuZuo Could you post a link to the job? Thanks

pdbain-ibm · 2018-03-19T19:34:58Z

I ran the sanity test manually > 10X on a machine where it failed. What failure rate do you see?
Also, are you seeing failures in Java 8?

pdbain-ibm · 2018-03-20T11:53:02Z

Seeing the failure frequently on 9.28.161.140

pdbain-ibm · 2018-03-20T13:29:22Z

Fails consistently on that machine. Setting -Dcom.ibm.tools.attach.logging=yes, we get:
1521551840457 7560: 12 [Attach API initializer]: AttachHandler initialize
1521551840460 7560: 12 [Attach API initializer]: IPC Directory=C:\cygwin64\tmp.com_ibm_tools_attach
1521551840461 7560: 12 [Attach API initializer]: createDirectoryAndSemaphore C:\cygwin64\tmp.com_ibm_tools_attach
1521551840463 7560: 12 [Attach API initializer]: non-blocking locking file C:\cygwin64\tmp.com_ibm_tools_attach_master
1521551840464 7560: 12 [Attach API initializer]: locking file C:\cygwin64\tmp.com_ibm_tools_attach_master
1521551840466 7560: 12 [Attach API initializer]: lock failed, trying blocking lock
1521551840480 7560: 12 [Attach API initializer]: AttachHandler obtained master lock
1521551840483 7560: 12 [Attach API initializer]: unlocking file C:\cygwin64\tmp.com_ibm_tools_attach_attachlock
1521551840483 7560: 12 [Attach API initializer]: unlocking file C:\cygwin64\tmp.com_ibm_tools_attach_master
1521551840483 7560: 12 [Attach API initializer]: AttachHandler IOException while creating files: C:\cygwin64\tmp.com_ibm_tools_attach\7560
1521551840483 7560: 12 [Attach API initializer]: AttachHandler terminate: Attach API is being shut down
1521551840484 7560: 12 [Attach API initializer]: AttachHandler terminate removing directory : C:\cygwin64\tmp.com_ibm_tools_attach
1521551840484 7560: 12 [Attach API initializer]: deleting my directory
1521551840484 7560: 12 [Attach API initializer]: deleting my files
1521551840493 7560: 12 [Attach API initializer]: non-blocking locking file C:\cygwin64\tmp.com_ibm_tools_attach_master
1521551840494 7560: 12 [Attach API initializer]: VM already notified for termination, abandoning master lock
1521551840494 7560: 12 [Attach API initializer]: AttachHandler tryObtainMasterLock failed
1521551840494 7560: 12 [Attach API initializer]: ERROR: failed to initialize

Rebooted the machine, still failing. Cannot delete /tmp/.com_ibm_tools_attach

pdbain-ibm · 2018-03-20T13:52:20Z

This is a machine issue.
Tried specifying a different attach director using -Dcom.ibm.tools.attach.directory= and the attach API initialized properly.
Opened a ticket to delete C:\cygwin64\tmp.com_ibm_tools_attach.

DanHeidinga · 2018-03-20T14:18:23Z

Is there a change we can make to the test so its more robust and cleans up after itself?

pdbain-ibm · 2018-03-20T15:34:58Z

I doubt it. Note that the Attach API already cleans up after itself in the common directory (and tries to clean artifacts left by defunct processes).

The issue here is that the malformed files were in j9build's temp directory but were owned by Administrator and had what looked like the wrong permissions.

pdbain-ibm · 2018-03-20T19:12:05Z

temp directory cleared. I re-ran the test and problem looks fixed.
Please close this issue.

The failures were caused by machine problems. Fixes eclipse-openj9#1454 Signed-off-by: Peter Bain <peter_bain@ca.ibm.com>

DanHeidinga · 2018-03-20T19:26:42Z

Thanks Peter

* TestNG before 6.14 treats configuration failure as pass, this has been fixed since TestNG 6.14, update TestNG version to 6.14.2 to fix this bug in product build * separate JLM test targets to avoid OOM failure * re-enable tests which are fixed and passing * update exclude file which failed on TestNG 6.14.2 * exclude reattachAfterExit from win since it is not supported on win * disable testSoftMxRemote tests since it failed at setup stage [ci skip] fixed:eclipse-openj9#1367 fixed:eclipse-openj9#1112 Issue:eclipse-openj9#1454 Signed-off-by: Tianyu Zuo <tianyu@ca.ibm.com>

terryzuoty mentioned this issue Mar 16, 2018

update TestNG to 6.14.2 version to fix false positive test result #1398

Merged

pdbain-ibm added a commit to pdbain-ibm/openj9 that referenced this issue Mar 20, 2018

Re-enable AttachAPISanity_SE90

7477ba8

The failures were caused by machine problems. Fixes eclipse-openj9#1454 Signed-off-by: Peter Bain <peter_bain@ca.ibm.com>

pdbain-ibm mentioned this issue Mar 20, 2018

Re-enable AttachAPISanity_SE90 #1487

Merged

DanHeidinga added the comp:test label Mar 20, 2018

DanHeidinga closed this as completed Mar 20, 2018

adamfarley mentioned this issue Feb 2, 2021

functional.extended target TestJstack_SE80_0 hangs on 32bit Windows adoptium/aqa-tests#2220

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AttachAPISanity_SE90 failed intermittently on Windows #1454

AttachAPISanity_SE90 failed intermittently on Windows #1454

terryzuoty commented Mar 15, 2018

DanHeidinga commented Mar 15, 2018

terryzuoty commented Mar 15, 2018 •

edited

Loading

pdbain-ibm commented Mar 15, 2018

pdbain-ibm commented Mar 15, 2018

pdbain-ibm commented Mar 19, 2018

pdbain-ibm commented Mar 20, 2018

pdbain-ibm commented Mar 20, 2018

pdbain-ibm commented Mar 20, 2018

DanHeidinga commented Mar 20, 2018

pdbain-ibm commented Mar 20, 2018

pdbain-ibm commented Mar 20, 2018

DanHeidinga commented Mar 20, 2018

AttachAPISanity_SE90 failed intermittently on Windows #1454

AttachAPISanity_SE90 failed intermittently on Windows #1454

Comments

terryzuoty commented Mar 15, 2018

DanHeidinga commented Mar 15, 2018

terryzuoty commented Mar 15, 2018 • edited Loading

pdbain-ibm commented Mar 15, 2018

pdbain-ibm commented Mar 15, 2018

pdbain-ibm commented Mar 19, 2018

pdbain-ibm commented Mar 20, 2018

pdbain-ibm commented Mar 20, 2018

pdbain-ibm commented Mar 20, 2018

DanHeidinga commented Mar 20, 2018

pdbain-ibm commented Mar 20, 2018

pdbain-ibm commented Mar 20, 2018

DanHeidinga commented Mar 20, 2018

terryzuoty commented Mar 15, 2018 •

edited

Loading