OutOfMemoryError: lower memory usage for M heap dumps

[pyricau]

When parsing an Android M heap dump with perflib, LeakCanary uses 50mb. The heap dump itself is 13Mb.

I've been playing with perflib and with MAT to figure this out.

MAT says that the heap dump has 48172 unique roots. An object can be a GC root for more than one reason, so here's the decomposition:

THREAD_OBJ=15
NATIVE_STACK=295
SYSTEM_CLASS=197490
UNKNOWN=44338

Perflib finds ~200K GC Roots, which corresponds.

I started digging, and I found that there are only 3802 unique objects that are a gc root as a "System Class" at least once.

In other words, the heap dump contains the same information over and over again, on average 50 times per class.

[pyricau]

Here's a diff to android_platform_tool_base to deduplicate the loading of system class roots, per heap:

diff --git a/perflib/src/main/java/com/android/tools/perflib/heap/Heap.java b/perflib/src/main/java/com/android/tools/perflib/heap/Heap.java
index 3d617e6..3426dcc 100644
--- a/perflib/src/main/java/com/android/tools/perflib/heap/Heap.java
+++ b/perflib/src/main/java/com/android/tools/perflib/heap/Heap.java
@@ -17,14 +17,14 @@
 package com.android.tools.perflib.heap;

 import com.android.annotations.NonNull;
-
-import java.util.ArrayList;
-import java.util.Collection;
-
-import com.google.common.collect.*;
+import com.google.common.collect.ArrayListMultimap;
+import com.google.common.collect.Multimap;
 import gnu.trove.TIntObjectHashMap;
+import gnu.trove.TLongHashSet;
 import gnu.trove.TLongObjectHashMap;
 import gnu.trove.TObjectProcedure;
+import java.util.ArrayList;
+import java.util.Collection;

 public class Heap {

@@ -45,6 +45,10 @@ public class Heap {
     @NonNull
     ArrayList<RootObj> mRoots = new ArrayList<RootObj>();

+    // Deduplicates system class roots
+    @NonNull
+    private TLongHashSet rootSystemClassIds = new TLongHashSet();
+
     //  List of threads
     @NonNull
     TIntObjectHashMap<ThreadObj> mThreads = new TIntObjectHashMap<ThreadObj>();
@@ -103,6 +107,12 @@ public class Heap {
     }

     public final void addRoot(@NonNull RootObj root) {
+        if (root.getRootType() == RootType.SYSTEM_CLASS) {
+            boolean modified = rootSystemClassIds.add(root.getId());
+            if (!modified) {
+                return;
+            }
+        }
         root.mIndex = mRoots.size();
         mRoots.add(root);
     }

[pyricau]

Here's the memory on an Android device when loading the heap dump, before and after:

[pyricau]

We gain 20Mb by simply deduplicating system class roots per heap, which gives us way more leg room considering the 64Mb limit.

[pyricau]

I run the heap dump through a bunch of for loops. About 50% of the System Class roots are arrays and show up as "UNDEFINED" (they are array instances, not classes). I grouped the array by size and then counted the number of array of that type and size, it surprising how many arrays have the same size. Also not clear why those are GC roots.

Regarding actual System Class roots, there's a ton of duplication. They're all in the same heap: default. I grouped the system class roots by class name, and counted the number for each.

Here's the output: https://gist.github.com/pyricau/8becbc14aeb670368ff8

[Maragues]

I'm getting this failed report on 1.4-beta, Genymotion Nexus 4 Android 5.1.0

It's the same message as in #279, which is closed