Improve DDR support #3099

keithc-ca · 2018-10-02T17:08:34Z

DDR support in OpenJ9 makes use of the ddrgen tool from OMR which operates on debug information generated by compilers.

Not all compilers retain references to typedefs. For example a field involving type UDATA may instead be recorded as using uint32_t or uint64_t depending on the definition of uintptr_t. This means that we cannot rely on generating DDR pointer classes at build time because they would reflect the type definitions in that build which would be incompatible with core files produced by a VM with a different pointer size.

This makes the necessary foundation changes to allow DDR pointer classes to be created dynamically according to the core file being examined. Small changes will be required in the extension repositories to adopt this new capability.

doc issue eclipse-openj9/openj9-docs#97
Fixes #2507

* remove most type overrides for IDATA/UDATA * remove unnecessary struct tags * J9SRP(void) -> J9SRP Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

pshipton · 2018-10-02T17:46:43Z

@keithc-ca please create an issue at https://github.com/eclipse/openj9-docs in order to coordinate updating the release notes

pshipton · 2018-10-03T12:18:41Z

runtime/ddr/types.cpp

+#include "ddrhelp.h"
+
+/*
+ * This struct has fields correspnding to StructureTypeManager.TYPE_XXX categories


I think this comment could include a little more context, i.e. mention DDR.

Perhaps the fact that this file is in a folder called ddr provides that context?

pshipton · 2018-10-03T17:39:38Z

debugtools/DDR_VM/src/com/ibm/j9ddr/vm29/j9/OptInfo.java

@@ -220,15 +221,15 @@ private static String getOption(J9ROMClassPointer romClass, long option) throws
 	}

 	private static VoidPointer getStructure(J9ROMClassPointer romClass, long option) throws CorruptDataException {
-		SelfRelativePointer ptr = getSRPPtr(romClass.optionalInfo(), romClass.optionalFlags(), option);
+		SelfRelativePointer ptr = getSRPPtr(J9ROMClassHelper.optionalInfo(romClass), romClass.optionalFlags(), option);


What happens if there is code outside this project using optionalInfo(), cpShapeDescription()? jdmpview supports adding debug extensions at runtime via the plugins command. There are many extensions that exist outside of this project.

I don't think there's a way to avoid that problem - it's a consequence of the behavior of ddrgen (being unable to distinguish among U32Pointer, U64Pointer and UDATAPointer).

What will happen?

Keith created #3131 to deal with this.

pshipton · 2018-10-03T18:19:34Z

debugtools/DDR_VM/src/com/ibm/j9ddr/BytecodeGenerator.java

+	 */
+	private void doArrayMethod(FieldDescriptor field) {
+		String fieldType = field.getType();
+		String componentType = fieldType.substring(0, fieldType.lastIndexOf('[')).trim();


Assumes arrays are always single dimension.

No. If the field type is Whatever[4][5] it will analyze Whatever[4] and notice the component type has type TYPE_ARRAY. This is the same as the behavior found in PointerGenerator.java.

Include fields corresponding to StructureTypeManager.TYPE_XXX categories that don't appear elsewhere so that all paths in BytecodeGenerator and PointerGenerator are exercised. Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

* add these methods which throw NullPointerDereference -- add AbstractPointer.nonNullAddress() -- add StructurePointer.nonNullFieldEA(long offset) Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

BytecodeGenerator * 'structure' classes are generated by a new helper class * helper classes have been added to generate 'flags' and 'pointers' J9DDRClassLoader * delegate loading pointer classes to StructureReader * generate pointer classes only for version 29 or newer (if not present in j9ddr.jar) StructureAliases29.dat * rename legacy version and add copyright notice * add new version for use with blobs produced by ddrgen StructureReader: * add StructureTypeManager * add PackageNameType.POINTER_PACKAGE_SLASH_NAME * simplify implementation of getPackageName(PackageNameType) * choose the appropriate set of aliases to use Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

cpShapeDescription optionalInfo Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

With the new DDR tooling, these may be indistinguishable from IDATA/UDATA. Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

* add alias GC_ArrayletObjectModel=GC_ArrayObjectModel Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

pshipton · 2018-10-03T20:15:37Z

jenkins test all win,zlinux jdk8

pshipton · 2018-10-03T22:18:55Z

@keithc-ca note the test failures

keithc-ca · 2018-10-04T00:59:20Z

I see two failures of the form

[FindVMOutputParser] [ERROR] ParserUtil: missing line J9WSRP(struct J9PoolPuddleList) puddleList

where in the new output the struct tag is omitted: J9WSRP(J9PoolPuddleList): I think the test should be adjusted to match either format.

keithc-ca · 2018-10-04T16:27:55Z

The newest commit fixes the test failure.

* now accepts either of: `J9WSRP(J9PoolPuddleList) puddleList` `J9WSRP(struct J9PoolPuddleList) puddleList` * no longer accepts obsolete pattern: `J9WSRP(struct J9PoolPuddle) puddleList` * enhance ParserUtil to allow using patterns Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

pshipton · 2018-10-05T14:54:46Z

jenkins test all win,zlinux jdk8

pshipton · 2018-10-05T17:28:02Z

Looks like the win jdk8 sanity failure is a jenkins issue and all the testing passed.

Clean up type overrides

95c25b6

* remove most type overrides for IDATA/UDATA * remove unnecessary struct tags * J9SRP(void) -> J9SRP Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

keithc-ca requested a review from pshipton October 2, 2018 17:09

keithc-ca mentioned this pull request Oct 2, 2018

Use new DDR ClassGenerator tool ibmruntimes/openj9-openjdk-jdk11#28

Merged

pshipton added comp:vm depends:doc labels Oct 2, 2018

This was referenced Oct 2, 2018

Remove note about restriction on DDR use eclipse-openj9/openj9-docs#97

Closed

Compile DDR_VM code twice ibmruntimes/openj9-openjdk-jdk8#117

Merged

pshipton reviewed Oct 3, 2018

View reviewed changes

keithc-ca added 7 commits October 3, 2018 14:55

Add helper methods for generated code

6ced858

* add these methods which throw NullPointerDereference -- add AbstractPointer.nonNullAddress() -- add StructurePointer.nonNullFieldEA(long offset) Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

Add new methods to J9ROMClassHelper

0b5bd56

cpShapeDescription optionalInfo Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

Adjust references to fields of type I32/U32

056b9d2

With the new DDR tooling, these may be indistinguishable from IDATA/UDATA. Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

Handle GC_ArrayletObjectModel in legacy blob files

87fd777

* add alias GC_ArrayletObjectModel=GC_ArrayObjectModel Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

Add ClassGenerator to generate classes at build time

d757682

Signed-off-by: Keith W. Campbell <keithc@ca.ibm.com>

keithc-ca mentioned this pull request Oct 3, 2018

Provide migration advice for DDR debug extensions #3131

Open

keithc-ca force-pushed the ddr branch from 820d49f to d757682 Compare October 3, 2018 19:54

keithc-ca force-pushed the ddr branch from 4d49f61 to fc00f71 Compare October 4, 2018 17:03

pshipton approved these changes Oct 5, 2018

View reviewed changes

pshipton merged commit f69c425 into eclipse-openj9:master Oct 5, 2018

keithc-ca deleted the ddr branch October 9, 2018 14:11

babsingh mentioned this pull request Apr 5, 2019

Extend support to all POSIX signals on Unix platforms eclipse-omr/omr#3713

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve DDR support #3099

Improve DDR support #3099

keithc-ca commented Oct 2, 2018 •

edited by pshipton

Loading

pshipton commented Oct 2, 2018

pshipton Oct 3, 2018

keithc-ca Oct 3, 2018

pshipton Oct 3, 2018

keithc-ca Oct 3, 2018

pshipton Oct 3, 2018

pshipton Oct 3, 2018

pshipton Oct 3, 2018

keithc-ca Oct 3, 2018

pshipton commented Oct 3, 2018

pshipton commented Oct 3, 2018

keithc-ca commented Oct 4, 2018

keithc-ca commented Oct 4, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

Improve DDR support #3099

Improve DDR support #3099

Conversation

keithc-ca commented Oct 2, 2018 • edited by pshipton Loading

pshipton commented Oct 2, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pshipton commented Oct 3, 2018

pshipton commented Oct 3, 2018

keithc-ca commented Oct 4, 2018

keithc-ca commented Oct 4, 2018

pshipton commented Oct 5, 2018

pshipton commented Oct 5, 2018

keithc-ca commented Oct 2, 2018 •

edited by pshipton

Loading