-
Notifications
You must be signed in to change notification settings - Fork 738
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new DDR commands for modularity__Command 1 and 2 #1987
Conversation
@@ -274,7 +274,7 @@ private void generateClass(StructureDescriptor structure) throws IOException { | |||
File javaFile = new File(outputDir, structure.getPointerName() + ".java"); | |||
|
|||
// Do not create Pointer class for Constant only structures (this includes enums) | |||
if (structure.getFields().size() == 0 && structure.getConstants().size() != 0 && superClassName.equals("StructurePointer")) { | |||
if (structure.getFields().size() == 0 && structure.getConstants().size() > 1 && superClassName.equals("StructurePointer")) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
take out these changes Keith has submitted a fix for this
} | ||
} | ||
|
||
/** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fix indentation
@Ailloviee please post an example of what the two commands output in this PR |
@dmitripivkine Please take a look at this PR, it refactors some of the checkEngine and ObjectModel code |
J9ModulePointer modulePtr = null; | ||
while (poolIterator.hasNext()) { | ||
modulePtr = poolIterator.next(); | ||
if (ObjectModel.isPointerInHeap(vm, modulePtr.moduleName())) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add a comment saying why this check must be done
out.printf("%s !j9module %s\n", moduleName, hexAddress); | ||
} | ||
} | ||
} catch (Exception/* CorruptDataException */ e) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if CorruptDataException
is the only exception that can be thrown in the try block then specify it in the catch block
@Ailloviee can you fix the ObjectModel diff, its too large |
Sorry I tried to squash the commits but I did it wrong and there was merge conflicts |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GC DDR changes LGTM
6ce3e24
to
794a26d
Compare
c7dffa6
to
6cf6c22
Compare
return null; | ||
} | ||
|
||
// TODO kmt : this doesn't belong here |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove this comment, no longer applies since you've moved it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also add javadocs
* any abstract pointer | ||
* @return If the AbstractPointer pointer is in stored in heap. | ||
*/ | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove this space
@@ -362,4 +369,69 @@ public static VoidPointer getElementAddress(J9IndexableObjectPointer indexableOb | |||
{ | |||
return gcObjectModel.getElementAddress(indexableObjectPointer, elementIndex, elementSize); | |||
} | |||
|
|||
public static GCHeapRegionDescriptor findRegionForPointer(J9JavaVMPointer javaVM, GCHeapRegionManager hrm, AbstractPointer pointer, GCHeapRegionDescriptor region) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add javadocs for this
} catch (CorruptDataException cde) {} | ||
return null; | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add javadocs
* <p> | ||
* @param javaVM | ||
* J9JavaVMPointer | ||
* pointer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
formatting looks off here, add the @param
for each parameter
* java.prefs !j9module 0x000001305F478928 | ||
* .....(A list of every module loaded by the runtime) | ||
*/ | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove space
/* | ||
* Since both packages and modules could be in the pool, | ||
* the iterator checks on the region and make sure that it is | ||
* pointing to a module. (For modules, the pointer must be in heap) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
try to be more specific and mention that we distinguish between module and j9package by checking if the first field is in the heap. Since the first field for j9module is an object and the first field for j9package is a J9UTF8
public static GCHeapRegionDescriptor regionForAddress(J9JavaVMPointer javaVM, GCHeapRegionManager hrm, AbstractPointer pointer) | ||
{ | ||
try { | ||
if(null == hrm) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should be a space after the if
if (null ...
|
||
public static boolean isPointerInHeap(J9JavaVMPointer javaVM, AbstractPointer pointer) | ||
{ | ||
if(findRegionForPointer(javaVM, null, pointer, null) != null) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should be space after if
if(findRegionForPointer(javaVM, null, pointer, null) != null) { | ||
return true; | ||
} | ||
else { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
} else {
6f70363
to
a8df173
Compare
@@ -358,8 +365,104 @@ public static UDATA getHeaderSize(J9ObjectPointer object) throws CorruptDataExce | |||
return gcObjectModel.getHeaderSize(object); | |||
} | |||
|
|||
/** | |||
* Returns the address of an element. | |||
* <p> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dont need this
|
||
/** | ||
* Returns the heap region of a pointer. | ||
* <p> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dont need this
|
||
/** | ||
* Returns the heap region for address of a pointer. | ||
* <p> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same for the rest
@keithc-ca can you please review these changes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have the new commands been tested with a core file produced by Java 8?
* J9UTF8) | ||
*/ | ||
if (ObjectModel.isPointerInHeap(vm, modulePtr.moduleName())) { | ||
String moduleName = J9ObjectHelper.stringValue(modulePtr.moduleName()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please pull the expression modulePtr.moduleName()
out of the loop.
{ | ||
public FindModuleByNameCommand() | ||
{ | ||
addCommand("findmodulebyname","<moduleName>","find the modules corresponding to its module name"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit (formatting): space after comma
while (poolIterator.hasNext()) { | ||
modulePtr = poolIterator.next(); | ||
if (ObjectModel.isPointerInHeap(vm, modulePtr.moduleName()) | ||
&& pattern.isMatch(J9ObjectHelper.stringValue(modulePtr.moduleName()))) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please extract the expression modulePtr.moduleName()
into a local variable.
@keithc-ca So far I've only tested with core file generated by Java 10, can you please teach me how to generate core file using Java 8? |
What are you trying? -Xdump works the same on Java 8 as it does on Java 10, although Java 8 doesn't support modules. I assume Keith is asking you to check the commands fail gracefully on a Java 8 core. |
@Ailloviee when you are testing with Java8 make sure its a J9 or Openj9 and not hotspot |
@pshipton is right: I want to know that you've tried the commands on a core file from Java 8, that it behaves 'gracefully', and what that graceful behavior is. |
@pshipton @keithc-ca for now it fails but not gracefully, working on the version handling now |
29613cb
to
6218399
Compare
Jenkins signed off by check |
@keithc-ca @pshipton New commands now fail gracefully on Java 8, example:
|
import com.ibm.j9ddr.vm29.types.UDATA; | ||
|
||
/** | ||
* JavaVersionHelper helps check if the JVM version is new enough for the modularity DDR commands |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove the part about "modularity DDR commands". This class should be general enough for other purposes
public final static int J2SE_18 = 0x0800; | ||
public final static int J2SE_19 = 0x0900; | ||
public final static int J2SE_V10 = 0x0A00; | ||
public final static int J2SE_V11 = 0x0B00; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to see this written so it doesn't need attention with every new version.
The comments in j2sever.h suggest vm.j2seVersion().bitAnd(J2SE_VERSION_MASK)
will give the right answer until Java version 255. The J2SE_* constants for 8-11 are thus not necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
vm.j2seVersion().bitAnd(J2SE_VERSION_MASK)
returns the hex version like 0x0800
, is it fine if I just print it out like that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The version will be vm.j2seVersion().bitAnd(J2SE_VERSION_MASK).shiftRight(J2SE_JAVA_SPEC_VERSION_SHIFT)
.
We should probably arrange that those constants are available via generated DDR code. It could be that only a DDR annotation is required in j2sever.h.
UDATA javaVersion = vm.j2seVersion().bitAnd(J2SE_SERVICE_RELEASE_MASK); | ||
if (javaVersion.lt(J2SE_19)) { | ||
out.printf("This command only works with Java version 9 or higher\n" | ||
+ "The current Java version is: %s\n", convertVersionToString(javaVersion.intValue())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The message should make it clearer that it's the version of the VM that created the core file being examined that is relevant (not the SDK being used to examine that core file).
return true; | ||
} | ||
|
||
private static String convertVersionToString(int version) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should also be generalized so it won't need attention with each Java version.
if (ObjectModel.isPointerInHeap(vm, moduleNameObjPtr)) { | ||
String moduleName = J9ObjectHelper.stringValue(moduleNameObjPtr); | ||
String hexAddress = modulePtr.getHexAddress(); | ||
out.printf("%s !j9module %s\n", moduleName, hexAddress); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Strings should not have internal tabs; a better solution would be to use %-20s
(or other suitable width).
* iterator checks on the region and make sure that it is | ||
* pointing to a module. (For modules, the first field must be | ||
* in heap since it is an object while for j9package it is a | ||
* J9UTF8) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: Please finish the sentence with a period.
J9ObjectPointer moduleNameObjPtr = null; | ||
PatternString pattern = new PatternString(searchModuleName); | ||
|
||
out.println(String.format("Searching for modules named '%1$s' in VM=%2$s", searchModuleName, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why use %$1s
instead of just %s
? The former is only helpful when arguments are used out of order or are used repeatedly.
} | ||
} | ||
|
||
/** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: Formatting and inconsistent indenting.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand what the indentation issue is here..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks good now, so whatever I was commenting about has been addressed.
/* Even if hrm is null, all helpers that use it will null check it and attempt to allocate it and handle the failure */ | ||
MM_HeapRegionManagerPointer hrmPtr = MM_GCExtensionsPointer.cast(_javaVM.gcExtensions()).heapRegionManager(); | ||
_hrm = GCHeapRegionManager.fromHeapRegionManager(hrmPtr); | ||
} catch (CorruptDataException cde) {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This exception ought not to be ignored: I expect many NullPointerExceptions or similar problems if _hrm
is null.
fb08b31
to
4d41e58
Compare
} | ||
} | ||
|
||
/** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks good now, so whatever I was commenting about has been addressed.
|
||
String moduleName = J9ObjectHelper.stringValue(moduleNameObjPtr); | ||
String hexAddress = modulePtr.getHexAddress(); | ||
out.printf("%-30s!j9module %s\n", moduleName, hexAddress); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There should be at least one space between %-30s
and !j9module
so a module name of 30 characters or more is still separated from the command. You should also use %n
instead of \n
.
if (ObjectModel.isPointerInHeap(vm, moduleNameObjPtr)) { | ||
String moduleName = J9ObjectHelper.stringValue(moduleNameObjPtr); | ||
String hexAddress = modulePtr.getHexAddress(); | ||
out.printf("%-30s!j9module %s\n", moduleName, hexAddress); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a space between the fields and use %n
for the line-end.
out.printf("%-30s!j9module %s\n", moduleName, hexAddress); | ||
} | ||
} | ||
out.println(String.format("Found %d module(s) named %s", hitCount, searchModuleName)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please replace %s
with '%s'
for consistency with earlier message.
*/ | ||
private void printUsage(PrintStream out) | ||
{ | ||
out.println("findmodulebyname <moduleName> - find the module corresponding to its module name"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The help and this usage message should make it clear that the argument is a name pattern.
deba172
to
5568562
Compare
int javaVersion = vm.j2seVersion().bitAnd(J2SE_SERVICE_RELEASE_MASK).intValue() >> J2SE_JAVA_SPEC_VERSION_SHIFT; | ||
if (javaVersion < J2SE_19) { | ||
out.printf("This command only works with core file created by VM with Java version 9 or higher\n" | ||
+ "The current VM Java version is: %s\n", javaVersion); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please %n
instead of \n
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please replace the remaining \n
with %n
.
J9ObjectPointer moduleNameObjPtr = null; | ||
PatternString pattern = new PatternString(searchModuleName); | ||
|
||
out.println(String.format("Searching for modules named '%s' in VM=%s", searchModuleName, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please use printf
(with %n
) instead of println
and format
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please fix the other instance of this pattern.
*/ | ||
private void printUsage(PrintStream out) | ||
{ | ||
out.println("findmodulebyname <moduleNamePattern> - find the module corresponding to its given name pattern (like 'java.base')"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The help can be improved, I suggest:
findmodulebyname - find the modules corresponding to a name pattern (e.g. 'java.*')
Notice it makes it clear that more than one module might be found and shows the form of pattern used (I assume 'java.' is the same as the Java pattern 'java..').
{ | ||
public FindModuleByNameCommand() | ||
{ | ||
addCommand("findmodulebyname", "<moduleName>", "find the modules corresponding to its module name"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The help here should be similar to that displayed by the usage method below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please address my most recent comments.
Signed-off-by: Charles_Zheng <Juntian.Zheng@ibm.com>
6aff8eb
to
a1a647c
Compare
jenkins test sanity plinux |
Added two commands:
!findallmodules Outputs all the modules loaded by the runtime
findmodulebyname <moduleName> find the modules corresponding to its module name
Related issue: Add new DDR commands for modularity #1859
Refactored
ObjectModel.java
andCheckEngine.java
to make some pointer check helper functions public static.Made change in PointerGenerator.java as DDR wouldn't compile without the change, for detail look at: #1973
Signed-off-by: Charles_Zheng Juntian.Zheng@ibm.com
Examples:
findallmodules:
findmodulebyname:
Also, added test cases for the two commands