Implement jps: java process status #4650

pdbain-ibm · 2019-02-07T18:01:07Z

Add Java code to implement the main function. This uses Attach API to acquire the list of processes
and query their information.

Update Attach API to return JVM arguments

Launchers will be added using coordinated changes in the extension repos.

Requires (mutually):

Add launcher for jps ibmruntimes/openj9-openjdk-jdk8#267 for Java 8 launcher
Add launcher definition for jps ibmruntimes/openj9-openjdk-jdk11#130 for Java 11 launcher
Add launcher definition for jps in Java 12 ibmruntimes/openj9-openjdk-jdk12#12 for Java 12 launcher
Add launcher definition for jps ibmruntimes/openj9-openjdk-jdk#74 for future release launcher

These required test utility changes should be merged before this is merged:

~~Create generic test process launcher #4672 for testing~~ Not required
~~Add test utility methods to get attach API information #4639 for testing.~~ Merged

Doc issues:

Fixes #2377

Signed-off-by: Peter Bain peter_bain@ca.ibm.com

pdbain-ibm · 2019-02-07T18:16:44Z

To build a launcher, add closed/make/closed_make/launcher/Launcher-openj9.diagnostics.gmk in the extensions repository containing the following:

include LauncherCommon.gmk

$(eval $(call SetupBuildLauncher, jps, \
    MAIN_MODULE := openj9.diagnostics, \
    MAIN_CLASS := openj9.tools.attach.diagnostics.Jps, \
    CFLAGS := -DEXPAND_CLASSPATH_WILDCARDS, \
    JAVA_ARGS := -Djdk.attach.allowAttachSelf=true, \
))

pdbain-ibm · 2019-02-07T18:43:59Z

Related issues:
#2259,
#2621,
#2378,
#2377.

pshipton · 2019-02-07T20:26:06Z

Please open a https://github.com/eclipse/openj9-docs issue

pshipton · 2019-02-07T20:31:28Z

I prefer to have the launchers and tests before this is merged, so they can be run to confirm everything is working properly.

jcl/src/openj9.diagnostics/share/classes/openj9/tools/attach/diagnostics/Jps.java

pshipton · 2019-02-07T21:03:51Z

Likely been asked before, but for the record here, does the sun.tools.jps.Jps implementation work on OpenJ9? Is there any need to write another one?

If we do, I'm thinking it should go into the jdk.jcmd module for compatibility, instead of openj9.diagnostics.

pshipton · 2019-02-07T21:05:32Z

@keithc-ca

pshipton · 2019-02-07T21:06:10Z

I'm thinking any new module we use needs to be added to the j9modules.txt to ensure everything is built in the correct order.

jcl/.classpath

jcl/src/java.base/share/classes/com/ibm/tools/attach/target/Attachment.java

jcl/src/openj9.diagnostics/share/classes/openj9/tools/attach/diagnostics/Jps.java

pdbain-ibm · 2019-02-08T13:12:21Z

Does the sun.tools.jps.Jps implementation work on OpenJ9? Is there any need to write another one?

No, the existing jps silently produces no results when compiled with OpenJ9. This utility predates,and does not use, attach API; it appears to use an undocumented proprietary mechanism. It is also not available on Windows.

If we do, I'm thinking it should go into the jdk.jcmd module for compatibility, instead of openj9.diagnostics.

Will do.

pdbain-ibm · 2019-02-08T13:36:06Z

I'm thinking any new module we use needs to be added to the j9modules.txt to ensure everything is built in the correct order.

Done

keithc-ca · 2019-02-08T13:43:11Z

I don't see the changes you've claimed to have made - will you be pushing to your branch shortly?

pdbain-ibm · 2019-02-08T14:04:39Z

I don't see the changes you've claimed to have made - will you be pushing to your branch shortly?

I am testing the changes now and will advise when the updates are pushed.

pshipton · 2019-02-08T14:18:00Z

Note that Keith advised me the j9modules.txt was only needed for IBM builds and is now obsolete.

Requires eclipse-openj9/openj9#4650 Signed-off-by: Peter Bain <peter_bain@ca.ibm.com>

Depends on eclipse-openj9/openj9#4650 Signed-off-by: Peter Bain <peter_bain@ca.ibm.com>

Requires eclipse-openj9/openj9#4650 Signed-off-by: Peter Bain <peter_bain@ca.ibm.com>

pdbain-ibm · 2019-02-20T19:42:09Z

Recent changes seem to have broken Java 8. Investigating...

pshipton · 2019-02-20T22:51:26Z

@pdbain-ibm please indicate when this is ready to go, and remove the WIP.

pdbain · 2019-02-20T23:07:45Z

Working on it. The first problem was I mucked up jppd_configuration.xml. I fixed that and it passes manual tests, but the unit tests are failing. Will fix that ASAP, expect late Thursday.

pdbain-ibm · 2019-02-22T12:51:52Z

Updated and re-tested on Linux X86 Java 8 & 11. Changes since last review:

jpp_configuration.xml fixed
TestJps.java setupSuite() updated to create an absolute path to the jps executable, searching both bin and jre/bin
playlist.xml added a Java 8-specific variant of the test with tools.jar in the classpath. Also add an export to the Java 11 variant to suppress error messages.

pshipton · 2019-02-22T15:48:43Z

I ran some testing on plinux, which passed, although the jps tests took 20min to run.
I see another change was pushed since this testing to fix the WARNING about illegal reflective access.

08:27:34  ...
08:27:34  ... TestNG 6.14.2 by Cédric Beust (cedric@beust.com)
08:27:34  ...
08:27:34  
08:27:34  WARNING: An illegal reflective access operation has occurred
08:27:34  WARNING: Illegal reflective access by org.openj9.test.attachAPI.TargetManager (file:/home/jenkins/jenkins-agent/workspace/Test-Grinder/jvmtest/functional/Java8andUp/GeneralTest.jar) to method com.ibm.tools.attach.target.AttachHandler.waitForAttachApiInitialization()
08:27:34  WARNING: Please consider reporting this to the maintainers of org.openj9.test.attachAPI.TargetManager
08:27:34  WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
08:27:34  WARNING: All illegal access operations will be denied in a future release
08:47:40  PASSED: testApplicationArguments
08:47:40  PASSED: testJpsIdOnly
08:47:40  PASSED: testJpsPackageName
08:47:40  PASSED: testJpsSanity
08:47:40  PASSED: testJvmArguments

jdk8: https://ci.eclipse.org/openj9/view/Test/job/Test-Grinder/308/ - ub16-ppcle-1
jdk11: https://ci.eclipse.org/openj9/view/Test/job/Test-Grinder/306/ - ub16-ppcle-1

pdbain-ibm · 2019-02-22T16:17:34Z

I ran some testing on plinux, which passed, although the jps tests took 20min to run.

The test runs in well under a minute on my Mac and Linux box, and on the internal Linux X86 build farm. I am testing on a PPCLE machine. Note that the AttachAPISanity exercises much of the same code. Does it run slowly in a grinder too? Could you try TestJps on xLinux?

pdbain-ibm · 2019-02-22T16:44:44Z

I ran TestJps on a pLinux box locally; it ran in ~15 seconds.

Manually tested on Windows.

pshipton · 2019-02-22T18:45:58Z

xlinux
jdk8: https://ci.eclipse.org/openj9/view/Test/job/Test-Grinder/310/
jdk11: https://ci.eclipse.org/openj9/view/Test/job/Test-Grinder/311/

pshipton · 2019-02-22T18:51:35Z

Checked AttachAPISanity on plinux from the nightly build last night, but it isn't taking much time to run.
https://ci.eclipse.org/openj9/job/Test-sanity.functional-JDK11-linux_ppc-64_cmprssptrs_le/542
ub16-ppcle-1

02:22:34  ... TestNG 6.14.2 by Cédric Beust (cedric@beust.com)
02:22:34  ...
02:22:34  
02:22:42  PASSED: testAttachToOtherProcess
02:22:42  PASSED: testAttachToSelf
02:22:42  PASSED: testAttachToSelfAndOthers

pshipton · 2019-02-22T19:05:14Z

~10s on xlinux.

Please request access to the plinux machine ub16-ppcle-1 and check why it is slow to run TestJps.

pshipton · 2019-02-15T19:42:05Z

test/functional/Java8andUp/src/org/openj9/test/attachAPI/TestJps.java

+		assertTrue(CHILD_PROCESS_DID_NOT_LAUNCH, tgtMgr.syncWithTarget());
+		List<String> jpsOutput = runCommand(Collections.singletonList("-l")); //$NON-NLS-1$
+		Optional<String> searchResult = search(tgtMgr.targetId, jpsOutput);
+		assertTrue(CHILD_IS_MISSING, search(tgtMgr.targetId, jpsOutput).isPresent());


searchResult can be used here.

pshipton · 2019-02-22T20:09:20Z

test/functional/Java8andUp/src/org/openj9/test/attachAPI/TestJps.java

+		commandName = JPS_COMMAND;
+		assertTrue("Attach API failed to launch", TargetManager.waitForAttachApiInitialization()); //$NON-NLS-1$
+		vmId = TargetManager.getVmId();
+		skipTest = !System.getProperty("java.vm.vendor").contains("OpenJ9");  //$NON-NLS-1$//$NON-NLS-2$


Is this really required? I'm not sure that it is, since IBM isn't listed as an <impl> in the playlist.

test/functional/Java8andUp/src/org/openj9/test/attachAPI/TestJps.java

pshipton · 2019-02-22T20:12:19Z

Windows - all passed
jdk8: https://ci.eclipse.org/openj9/view/Test/job/Test-Grinder/314/
jdk8 32-bit: https://ci.eclipse.org/openj9/view/Test/job/Test-Grinder/313/
jdk11: https://ci.eclipse.org/openj9/view/Test/job/Test-Grinder/312/

pdbain-ibm · 2019-02-22T20:27:35Z

Ran a personal build on Windows, Java 8 and 11 passed.

Add Java code to implement the main function Update Attach API to return JVM arguments Add unit test [ci skip] Signed-off-by: Peter Bain <peter_bain@ca.ibm.com>

pdbain-ibm · 2019-02-22T20:40:56Z

searchResult can be used here.

Right.

Should there be a tgtMgr2.syncWithTarget()?

Yes.

Is this really required? I'm not sure that it is, since IBM isn't listed as an <impl> in the playlist.

I added it for the case where someone is running testNG without the test tooling. I removed the flag and changed the check to an assertion so the test fails if you try to run with a different JVM.

pdbain-ibm · 2019-02-22T20:58:17Z

Test code updated.

pshipton · 2019-02-22T21:10:22Z

Validation test https://ci.eclipse.org/openj9/view/Test/job/Test-Grinder/315/

pshipton · 2019-02-22T21:37:46Z

Previous grinder passed.

pdbain-ibm force-pushed the attach_jps branch 2 times, most recently from 5a17cdd to 760fd23 Compare February 7, 2019 18:07

pdbain-ibm mentioned this pull request Feb 7, 2019

Add test utility methods to get attach API information #4639

Merged

pdbain-ibm force-pushed the attach_jps branch from 760fd23 to 94a24de Compare February 7, 2019 19:28

pshipton added comp:vm depends:doc labels Feb 7, 2019

pshipton added this to the Release 0.13.0 (Java 12) milestone Feb 7, 2019

pdbain-ibm mentioned this pull request Feb 7, 2019

Implement Java diagnostic utilities #4655

Open

6 tasks

pdbain-ibm mentioned this pull request Feb 7, 2019

Document jps eclipse-openj9/openj9-docs#210

Closed

pshipton reviewed Feb 7, 2019

View reviewed changes

keithc-ca requested changes Feb 7, 2019

View reviewed changes

pdbain-ibm added a commit to pdbain-ibm/openj9-openjdk-jdk11 that referenced this pull request Feb 8, 2019

Add launcher definition for jps

0dcfb9a

Requires eclipse-openj9/openj9#4650 Signed-off-by: Peter Bain <peter_bain@ca.ibm.com>

pdbain-ibm added a commit to pdbain-ibm/openj9-openjdk-jdk11 that referenced this pull request Feb 8, 2019

Add launcher definition for jps

64d7c7b

Requires eclipse-openj9/openj9#4650 Signed-off-by: Peter Bain <peter_bain@ca.ibm.com>

pdbain-ibm added a commit to pdbain-ibm/openj9-openjdk-jdk8 that referenced this pull request Feb 8, 2019

Add launcher for jps

4be9fbb

Depends on eclipse-openj9/openj9#4650 Signed-off-by: Peter Bain <peter_bain@ca.ibm.com>

pdbain-ibm force-pushed the attach_jps branch 2 times, most recently from 81dd587 to 48b2b84 Compare February 8, 2019 15:43

pdbain-ibm added a commit to pdbain-ibm/openj9-openjdk-jdk11 that referenced this pull request Feb 8, 2019

Add launcher definition for jps

df28617

Requires eclipse-openj9/openj9#4650 Signed-off-by: Peter Bain <peter_bain@ca.ibm.com>

pdbain-ibm mentioned this pull request Feb 8, 2019

Add launcher definition for jps ibmruntimes/openj9-openjdk-jdk11#130

Merged

pdbain-ibm force-pushed the attach_jps branch from f5a256c to 1f59efc Compare February 20, 2019 19:59

pdbain-ibm force-pushed the attach_jps branch 3 times, most recently from 347d99b to 68853e7 Compare February 21, 2019 20:56

pdbain-ibm changed the title ~~WIP: Implement jps: java process status~~ Implement jps: java process status Feb 22, 2019

pdbain-ibm force-pushed the attach_jps branch from 68853e7 to 129b19f Compare February 22, 2019 13:35

pdbain-ibm force-pushed the attach_jps branch from 129b19f to 24314b7 Compare February 22, 2019 17:30

pshipton reviewed Feb 22, 2019

View reviewed changes

Implement jps: java process status

142da72

Add Java code to implement the main function Update Attach API to return JVM arguments Add unit test [ci skip] Signed-off-by: Peter Bain <peter_bain@ca.ibm.com>

pdbain-ibm force-pushed the attach_jps branch from 24314b7 to 142da72 Compare February 22, 2019 20:40

pshipton approved these changes Feb 23, 2019

View reviewed changes

pshipton merged commit 655bf0d into eclipse-openj9:master Feb 23, 2019

pdbain-ibm deleted the attach_jps branch February 25, 2019 14:20

pshipton mentioned this pull request Apr 4, 2019

Final prep for milestone 1 #5355

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement jps: java process status #4650

Implement jps: java process status #4650

pdbain-ibm commented Feb 7, 2019 •

edited by pshipton

Loading

pdbain-ibm commented Feb 7, 2019

pdbain-ibm commented Feb 7, 2019

pshipton commented Feb 7, 2019

pshipton commented Feb 7, 2019

pshipton commented Feb 7, 2019 •

edited

Loading

pshipton commented Feb 7, 2019

pshipton commented Feb 7, 2019 •

edited

Loading

pdbain-ibm commented Feb 8, 2019

pdbain-ibm commented Feb 8, 2019

keithc-ca commented Feb 8, 2019

pdbain-ibm commented Feb 8, 2019

pshipton commented Feb 8, 2019

pdbain-ibm commented Feb 20, 2019

pshipton commented Feb 20, 2019

pdbain commented Feb 20, 2019

pdbain-ibm commented Feb 22, 2019

pshipton commented Feb 22, 2019 •

edited

Loading

pdbain-ibm commented Feb 22, 2019

pdbain-ibm commented Feb 22, 2019 •

edited

Loading

pshipton commented Feb 22, 2019

pshipton commented Feb 22, 2019 •

edited

Loading

pshipton commented Feb 22, 2019

pshipton Feb 15, 2019

pshipton Feb 22, 2019

pshipton commented Feb 22, 2019 •

edited

Loading

pdbain-ibm commented Feb 22, 2019

pdbain-ibm commented Feb 22, 2019

pdbain-ibm commented Feb 22, 2019

pshipton commented Feb 22, 2019

pshipton commented Feb 22, 2019

Implement jps: java process status #4650

Implement jps: java process status #4650

Conversation

pdbain-ibm commented Feb 7, 2019 • edited by pshipton Loading

pdbain-ibm commented Feb 7, 2019

pdbain-ibm commented Feb 7, 2019

pshipton commented Feb 7, 2019

pshipton commented Feb 7, 2019

pshipton commented Feb 7, 2019 • edited Loading

pshipton commented Feb 7, 2019

pshipton commented Feb 7, 2019 • edited Loading

pdbain-ibm commented Feb 8, 2019

pdbain-ibm commented Feb 8, 2019

keithc-ca commented Feb 8, 2019

pdbain-ibm commented Feb 8, 2019

pshipton commented Feb 8, 2019

pdbain-ibm commented Feb 20, 2019

pshipton commented Feb 20, 2019

pdbain commented Feb 20, 2019

pdbain-ibm commented Feb 22, 2019

pshipton commented Feb 22, 2019 • edited Loading

pdbain-ibm commented Feb 22, 2019

pdbain-ibm commented Feb 22, 2019 • edited Loading

pshipton commented Feb 22, 2019

pshipton commented Feb 22, 2019 • edited Loading

pshipton commented Feb 22, 2019

pshipton Feb 15, 2019

Choose a reason for hiding this comment

pshipton Feb 22, 2019

Choose a reason for hiding this comment

pshipton commented Feb 22, 2019 • edited Loading

pdbain-ibm commented Feb 22, 2019

pdbain-ibm commented Feb 22, 2019

pdbain-ibm commented Feb 22, 2019

pshipton commented Feb 22, 2019

pshipton commented Feb 22, 2019

pdbain-ibm commented Feb 7, 2019 •

edited by pshipton

Loading

pshipton commented Feb 7, 2019 •

edited

Loading

pshipton commented Feb 7, 2019 •

edited

Loading

pshipton commented Feb 22, 2019 •

edited

Loading

pdbain-ibm commented Feb 22, 2019 •

edited

Loading

pshipton commented Feb 22, 2019 •

edited

Loading

pshipton commented Feb 22, 2019 •

edited

Loading