Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash during GC on arm64 #36906

Closed
mkustermann opened this issue May 9, 2019 · 3 comments
Closed

Crash during GC on arm64 #36906

mkustermann opened this issue May 9, 2019 · 3 comments
Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. gardening

Comments

@mkustermann
Copy link
Member

From this failure:

/============================================================================\
| standalone_2/no_lazy_dispatchers_test broke (Pass -> Crash, expected Pass) |
\============================================================================/

--- Command "vm" (took 11.000228s):
DART_CONFIGURATION=ReleaseXARM64 out/ReleaseXARM64/dart --no_lazy_dispatchers --ignore-unrecognized-flags --packages=/b/s/w/ir/.packages /b/s/w/ir/tests/standalone_2/no_lazy_dispatchers_test.dart

exit code:
-6

stderr:
===== CRASH =====
si_signo=Segmentation fault(11), si_code=1, si_addr=0xff7974696360
version=2.3.0-edge.13fa51ed541280d916c27e10cb1a47f6831b7135 (Wed May 8 14:49:45 2019 +0000) on "linux_arm64"
thread=13328, isolate=kernel-service(0xaaaaf7197b00)
  pc 0x0000aaaac5251230 fp 0x0000ffff80dfe690 out/ReleaseXARM64/dart+0x18a3230
  pc 0x0000aaaac5258a6c fp 0x0000ffff80dfe6c0 dart::Scavenger::VisitObjectPointers(dart::ObjectPointerVisitor*) const
  pc 0x0000aaaac524fb70 fp 0x0000ffff80dfe770 dart::GCMarker::IterateRoots(dart::ObjectPointerVisitor*)
  pc 0x0000aaaac52513f8 fp 0x0000ffff80dfe800 out/ReleaseXARM64/dart+0x18a33f8
  pc 0x0000aaaac50fbc3c fp 0x0000ffff80dfe840 dart::ThreadPool::Worker::Loop()
  pc 0x0000aaaac50fba6c fp 0x0000ffff80dfe880 dart::ThreadPool::Worker::Main(unsigned long)
  pc 0x0000aaaac5063d00 fp 0x0000ffff80dfe930 out/ReleaseXARM64/dart+0x16b5d00
  pc 0x0000ffff89583fc4 fp 0x0000ffff80dfe940 /lib/aarch64-linux-gnu/libpthread.so.0+0x6fc4
-- End of DumpStackTrace

--- Re-run this test:
python tools/test.py -n dartk-linux-release-arm64 standalone_2/no_lazy_dispatchers_test


/============================================================================\
| vm/dart/generic_field_invocation_test broke (Pass -> Crash, expected Pass) |
\============================================================================/

--- Command "vm" (took 08.000789s):
DART_CONFIGURATION=ReleaseXARM64 out/ReleaseXARM64/dart --no-lazy-dispatchers --ignore-unrecognized-flags --packages=/b/s/w/ir/.packages /b/s/w/ir/runtime/tests/vm/dart/generic_field_invocation_test.dart

exit code:
-6

stderr:
===== CRASH =====
si_signo=Segmentation fault(11), si_code=1, si_addr=0x6c697542646460
version=2.3.0-edge.13fa51ed541280d916c27e10cb1a47f6831b7135 (Wed May 8 14:49:45 2019 +0000) on "linux_arm64"
thread=1764, isolate=kernel-service(0xaaab20a55200)
  pc 0x0000aaaaea548230 fp 0x0000ffff990be6b0 out/ReleaseXARM64/dart+0x18a3230
  pc 0x0000aaaaea54fa6c fp 0x0000ffff990be6e0 dart::Scavenger::VisitObjectPointers(dart::ObjectPointerVisitor*) const
  pc 0x0000aaaaea546b70 fp 0x0000ffff990be790 dart::GCMarker::IterateRoots(dart::ObjectPointerVisitor*)
  pc 0x0000aaaaea547d7c fp 0x0000ffff990be800 out/ReleaseXARM64/dart+0x18a2d7c
  pc 0x0000aaaaea3f2c3c fp 0x0000ffff990be840 dart::ThreadPool::Worker::Loop()
  pc 0x0000aaaaea3f2a6c fp 0x0000ffff990be880 dart::ThreadPool::Worker::Main(unsigned long)
  pc 0x0000aaaaea35ad00 fp 0x0000ffff990be930 out/ReleaseXARM64/dart+0x16b5d00
  pc 0x0000ffff9b49ffc4 fp 0x0000ffff990be940 /lib/aarch64-linux-gnu/libpthread.so.0+0x6fc4
-- End of DumpStackTrace

--- Re-run this test:
python tools/test.py -n dartk-linux-release-arm64 vm/dart/generic_field_invocation_tes

There seem to be cores available for this, see isolate server

/cc @rmacnak-google
@mkustermann mkustermann added area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. gardening labels May 9, 2019
@mkustermann
Copy link
Member Author

@mraleph

@alexmarkov
Copy link
Contributor

Still crashing:

 /============================================================================\
 | vm/dart/generic_field_invocation_test broke (Pass -> Crash, expected Pass) |
 \============================================================================/
 --- Command "vm" (took 05.000303s):
 DART_CONFIGURATION=ReleaseXARM64 out/ReleaseXARM64/dart --no-lazy-dispatchers --ignore-unrecognized-flags --packages=/b/s/w/ir/.packages /b/s/w/ir/runtime/tests/vm/dart/generic_field_invocation_test.dart
 exit code:
 -6
 stderr:
 ===== CRASH =====
 si_signo=Segmentation fault(11), si_code=1, si_addr=0x65666552797260
 version=2.8.0-edge.168cf90131a44024f3d6d2c93ce6ac9a17c22f47 (Mon Dec 9 12:51:07 2019 +0000) on "linux_arm64"
 thread=22070, isolate=kernel-service(0xaaaace228400)
   pc 0x0000aaaabb5693fc fp 0x0000ffff8affe6b0 out/ReleaseXARM64/dart+0x1ad73fc
   pc 0x0000aaaabb5716b0 fp 0x0000ffff8affe6e0 dart::Scavenger::VisitObjectPointers(dart::ObjectPointerVisitor*) const
   pc 0x0000aaaabb568184 fp 0x0000ffff8affe790 dart::GCMarker::IterateRoots(dart::ObjectPointerVisitor*)
   pc 0x0000aaaabb56955c fp 0x0000ffff8affe800 out/ReleaseXARM64/dart+0x1ad755c
   pc 0x0000aaaabb415474 fp 0x0000ffff8affe840 dart::ThreadPool::Worker::Loop()
   pc 0x0000aaaabb415278 fp 0x0000ffff8affe870 dart::ThreadPool::Worker::Main(unsigned long)
   pc 0x0000aaaabb395d20 fp 0x0000ffff8affe930 out/ReleaseXARM64/dart+0x1903d20
   pc 0x0000ffff8c6fbfc4 fp 0x0000ffff8affe940 /lib/aarch64-linux-gnu/libpthread.so.0+0x6fc4
 -- End of DumpStackTrace
 --- Re-run this test:
 python tools/test.py -n dartk-linux-release-arm64 vm/dart/generic_field_invocation_test

log

@rmacnak-google
Copy link
Contributor

While iterating new-space, the iteration has gotten confused and thinks the next object starts in the middle of a string object.

                ------------------
0xffff86f802b8:	0x3be08d68004b0304 OneByteString, heapsize=3units, new-bit
         	0x0000000000000026 size=19 code units
0xffff86f802c8:	0x6972745364616572 "readStringReference"
           	0x657265666552676e
0xffff86f802d8:	0x0000ffff8c65636e 
        	0x0000ffff8c280041 (object null)
                ------------------
0xffff86f802e8:	0x20bd6b52004b0204 OneByteString, heapsize=2units, new-bit
              	0x000000000000001c size=14 code units
0xffff86f802f8:	0x6f43676e69727453 "StringConstant"                          <= current
           	0x0000746e6174736e      
                ------------------
0xffff86f80308:	0x00000001004b0204 OneByteString, heapsize=2units, new-bit
         	0x0000000000000000 size=0
0xffff86f80318:	0x0000ffff8c280041 (object null)
           	0x0000ffff8c280041 (object null)
                ------------------

The near-by objects look correct. I'll add some instrumentation to remember the previous object during iteration to try to narrow down how we've gotten here.

dart-bot pushed a commit that referenced this issue Dec 10, 2019
@rmacnak-google
[vm, gc] As a debugging aid, remember the previous object during heap…
617933c 
… iteration.

Bug: #36906
Change-Id: Id7214aaa475929e33e5e5ae5bc58c99eb927b74b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/127743
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
dart-bot pushed a commit that referenced this issue Dec 12, 2019
@rmacnak-google
[vm, arm64] Fix heap corruption in PushArrayOfArguments.
a8d8a18 
The store barrier may clobber condition flags, causing the loop to copy past the last argument.

Bug: #36906
Change-Id: Ia863ec88aaa26c4193cadba26df62b43e68c377e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128108
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. gardening
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mkustermann @rmacnak-google @alexmarkov