BOLT is crashing when optimizing python

[nadavrot]

root@4a75b2eea103:/workspace/scripts# /usr/local/bin/llvm-bolt -instrument /workspace/python386-base/bin/python3.8 -o p3
BOLT-INFO: shared object or position-independent executable detected
BOLT-INFO: Target architecture: x86_64
BOLT-INFO: BOLT version: 87e45c9
BOLT-INFO: first alloc address is 0x0
BOLT-INFO: creating new program header table at address 0x400000, offset 0x400000
BOLT-WARNING: debug info will be stripped from the binary. Use -update-debug-sections to keep it.
BOLT-INFO: enabling relocation mode
BOLT-INFO: forcing -jump-tables=move for instrumentation
BOLT-INFO: enabling -align-macro-fusion=all since no profile was specified
BOLT-INFO: enabling lite mode
BOLT-WARNING: split function detected on input : pymain_main.cold/1. The support is limited in relocation mode.
llvm-bolt: /home/bolt/BOLT/bolt/src/BinaryContext.cpp:812: const llvm::MCSymbol* llvm::bolt::BinaryContext::getOrCreateJumpTable(llvm::bolt::BinaryFunction&, uint64_t, llvm::bolt::JumpTable::JumpTableType): Assertion `JT->Parent == &Function && "cannot re-use jump table of a different function"' failed.
#0 0x0000558d725c4575 PrintStackTraceSignalHandler(void*) Signals.cpp:0:0
#1 0x0000558d725c211e SignalHandler(int) Signals.cpp:0:0
#2 0x00007fe36400e520 (/lib/x86_64-linux-gnu/libc.so.6+0x46520)
#3 0x00007fe364062808 pthread_kill (/lib/x86_64-linux-gnu/libc.so.6+0x9a808)
#4 0x00007fe36400e476 gsignal (/lib/x86_64-linux-gnu/libc.so.6+0x46476)
#5 0x00007fe363ff47b7 abort (/lib/x86_64-linux-gnu/libc.so.6+0x2c7b7)
#6 0x00007fe363ff46db (/lib/x86_64-linux-gnu/libc.so.6+0x2c6db)
#7 0x00007fe364005e26 (/lib/x86_64-linux-gnu/libc.so.6+0x3de26)
#8 0x0000558d7171690b llvm::bolt::BinaryContext::getOrCreateJumpTable(llvm::bolt::BinaryFunction&, unsigned long, llvm::bolt::JumpTable::JumpTableType) (/usr/local/bin/llvm-bolt+0x2f890b)
#9 0x0000558d7171afcb llvm::bolt::BinaryContext::handleAddressRef(unsigned long, llvm::bolt::BinaryFunction&, bool) (/usr/local/bin/llvm-bolt+0x2fcfcb)
#10 0x0000558d71732939 llvm::bolt::BinaryFunction::disassemble()::'lambda'(llvm::MCInst&, unsigned long, unsigned long)::operator()(llvm::MCInst&, unsigned long, unsigned long) const BinaryFunction.cpp:0:0
#11 0x0000558d71748fff llvm::bolt::BinaryFunction::disassemble() (/usr/local/bin/llvm-bolt+0x32afff)
#12 0x0000558d717faef8 llvm::bolt::RewriteInstance::disassembleFunctions() (/usr/local/bin/llvm-bolt+0x3dcef8)
#13 0x0000558d718559d9 llvm::bolt::RewriteInstance::run() (/usr/local/bin/llvm-bolt+0x4379d9)
#14 0x0000558d71606372 main (/usr/local/bin/llvm-bolt+0x1e8372)
#15 0x00007fe363ff5fd0 (/lib/x86_64-linux-gnu/libc.so.6+0x2dfd0)
#16 0x00007fe363ff607d __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2e07d)
#17 0x0000558d71660d45 _start (/usr/local/bin/llvm-bolt+0x242d45)
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0. Program arguments: /usr/local/bin/llvm-bolt -instrument /workspace/python386-base/bin/python3.8 -o p3
Aborted (core dumped)

The relocations are available in the file.
$objdump python3.8_built_with_gcc -r | wc -l
724336

0000000000247680 R_X86_64_32 sysmodule.c.94ab4c58+0x00000000000067e8
0000000000247684 R_X86_64_32 .debug_loclists+0x00000000002fb38a
0000000000247688 R_X86_64_32 .debug_loclists+0x00000000002fb388
000000000024768e R_X86_64_64 .text+0x00000000000744b4
000000000024769d R_X86_64_32 unicodeobject.c.c792f956+0x000000000001100f
00000000002476a1 R_X86_64_64 .text+0x00000000000744c4
00000000002476b4 R_X86_64_32 unicodeobject.c.c792f956+0x0000000000004cd8
00000000002476b8 R_X86_64_64 .text+0x00000000000744ce

File info:

python3.8_built_with_gcc: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=3c6c49d795968eb31a310ba011b006529dd01245, for GNU/Linux 3.2.0, with debug_info, not stripped

ldd /workspace/python386-base/bin/python3
linux-vdso.so.1 (0x00007ffef3d8d000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1c558ab000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1c55683000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1c55d0a000)

[aaupov]

Looks like the issue is with jump tables in split functions produced by gcc8+

BOLT-WARNING: split function detected on input : pymain_main.cold/1. The support is limited in relocation mode.
#8 0x0000558d7171690b llvm::bolt::BinaryContext::getOrCreateJumpTable(llvm::bolt::BinaryFunction&, unsigned long, llvm::bolt::JumpTable::JumpTableType) (/usr/local/bin/llvm-bolt+0x2f890b)

Currently we recommend disabling this optimization in gcc with -fno-reorder-blocks-and-partition. BOLT performs function splitting so there's no regression.

Generally, this case of jump tables in split functions in non-stripped binaries is in my task list but with a low priority (if the binary is not stripped, it's likely compiled from scratch -> easy to recompile with disabled option).

[nhuhuan]

Hi @nadavrot, the issue has been fixed and is now upstream:

I've added a manual test to replicate the error. Thanks.

[aaupov]

Thanks Huan!
The upstream commit: llvm/llvm-project@28b1dcb

[nadavrot]

Thank you @nhuhuan and @aaupov