LLVM assertion failure on Wasm exceptions

[purplesyringa]

Code

fn main() {}

:/

Meta

I'm building rustc on the current master (bcd0683) with LLVM assertions enabled. I'm compiling the program above like this:

RUSTFLAGS="-C panic=unwind" cargo +stage1 build --target wasm32-wasip1 -Z build-std

I'm using the 23.0 WASI sysroot, downloaded from their GitHub releases, if that matters.

Error output

rustc: /checkout/src/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:1378: void mapWasmLandingPadIndex(MachineBasicBlock *, const CatchPadInst *): Assertion `IntrFound && "wasm.landingpad.index intrinsic not found!"' failed.

like, everywhere, on all sorts of standard crates like libc and unwind.

Disabling assertions hides the bug, and user code works fine. This also does not reproduce on rustup's nightly, probably for the same reason. I have no idea, really. Please tell me I'm doing something wrong.

For context, this seems more important because Emscripten is supposed to switch to Wasm exceptions soon-ish, and I imagine the same problem occurs there (but I haven't checked). Unlike wasm32-wasip1, unwinding is enabled there by default.

@rustbot label +A-LLVM

[workingjubilee]

cc @juntyr @hoodmane

[purplesyringa]

I also get ud2 inside LLVM even with assertions disabled on some complex code that directly uses core::intrinsics::catch_unwind. The code worked perfectly fine on nightly, it works just fine on master on x86-64, and it was mostly copied from std anyway, so I doubt (but will check twice) this isn't my bug. It seems to me like something has gone very wrong with Wasm unwinding in rustc at some point. Any pointers are appreciated.

[purplesyringa]

Downgrading my local repo to 419b3e2, which is what my nightly is based on, does not remove the assertion failures. So perhaps the only reason nightly is working is that assertions are disabled by default.

I have not yet confirmed that disabling the assertions makes the complex code (to anyone who wants to play around with this, run LITHIUM_BACKEND=itanium ci/cargo-wasi test --release --target wasm32-wasip1 in https://github.com/iex-rs/lithium/tree/dev; add +stage1 to the cargo-wasi script beforehand) compile and work on this older commit. This is a sanity check to make sure it's not just my machine that's affected. I expect that it will compile and work, indicating that the cause of the ud2 is just #135484. Perhaps changes in LLVM caused the broken IR to lead to even more brokenness?

[workingjubilee]

I could be mistaken but I don't believe we necessarily support enabling unwinding on targets that are abort-by-default?

[workingjubilee]

Do these problems actually surface for the emscripten target on nightly if you use...

hm, we didn't document it...? we should document it here, probably... https://doc.rust-lang.org/nightly/rustc/platform-support/wasm32-unknown-emscripten.html

@purplesyringa on nightly, with

--target wasm32-unknown-emscripten \
-Zbuild-std \
-Zemscripten-wasm-eh

[purplesyringa]

I have not yet confirmed that disabling the assertions makes the complex code compile and work on this older commit.

So: rustc does compile it, but it miscompiles the code. Which is a known issue (#132416) I'm trying to fix, and applying the fix leads to the ud2 all over again. :/ No idea what's going on yet.

[purplesyringa]

I could be mistaken but I don't believe we necessarily support enabling unwinding on targets that are abort-by-default?

Wasm EH has been a thing in rustc for quite a while, so I'm assuming it was at the very least considered as something we want to support.

Do these problems actually surface for the emscripten target on nightly if you use...

So there's two problems: 1) assertion failure in general, and 2) ud2 on manual unwinding. The first one can't be reproduced on nightly under any conditions, because nightly doesn't enable assertions (is that correct?). The second one does not reproduce on nightly because it's hidden by the miscompilation from #132416 due to nounwind on the wasm intrinsic. The way I see it, what you're asking for doesn't make sense, because neither issue explicitly surfaces on nightly at all. Can you elaborate please?

[workingjubilee]

Wasm EH has been a thing in rustc for quite a while, so I'm assuming it was at the very least considered as something we want to support.

As part of wasip1?

I don't believe so, no.

cc @alexcrichton

[purplesyringa]

^ sorry for the wrong label

Here's a short reproducer of the ud2 bug:

#![feature(core_intrinsics, wasm_exception_handling_intrinsics, rustc_attrs)]

fn main() {
    unsafe {
        core::intrinsics::catch_unwind(
            |_| {
                core::arch::wasm32::throw::<0>(core::ptr::null_mut());
            },
            core::ptr::null_mut(),
            #[rustc_nounwind]
            |_, _| {},
        );
    };
}

It reproduces on both the nightly commit and master if you remove the bit in rustc_codegen_ssa/src/codegen_attrs.rs that makes the llvm.wasm.throw intrinsic nounwind. (We do want that change, or catching panics will be broken if the optimizer feels like it.)

The crash affects both wasm32-wasip1 with -C panic=unwind and wasm32-unknown-emscripten with -Z emscripten_wasm_eh.

I don't see anything wrong in this snippet -- it's pretty much what std does, barring null pointers, but that's not the problem. Thoughts?

[workingjubilee]

My memory says we do produce a variant of the nightly builds that has assertions enabled, but I can't remember unfortunately exactly how to obtain it.

[purplesyringa]

I can also confirm that the assertion is triggered on emscripten (on empty code, on master, without any rustc modifications).

[alexcrichton]

I unfortunately know very little about the state of unwinding/exceptions in LLVM. My (very dated) impression is that LLVM supports the what's now called "legacy exceptions" proposal in wasm and is getting support for the official proposal. I don't know how battle-tested either are in practice though in terms of robustness of the LLVM backend.

[hoodmane]

I think it has support for both legacy and new exceptions now. As I understand it, Emscripten still is asking llvm to emit the legacy exceptions and then uses binaryen to convert to the new exceptions. But there is a PR to fix this:
emscripten-core/emscripten#23469

cc @aheejin

[aheejin]

@purplesyringa Can you provide a bitcode (ll) reproducer? I don't have rustc working with the local LLVM source build set up in my local machine.

Also I don't know what ud2 is, but I would also appreciate a bitcode reproducer with the expected behavior (and the current non-expected behavior).

[purplesyringa]

I compiled fn main() {} like this:

$ RUSTFLAGS="-Z emscripten_wasm_eh -C save-temps -C codegen-units=1" cargo +stage1 rustc --target wasm32-unknown-emscripten -Z build-std

This failed when compiling compiler_builtins. I then disassembled the compiler_builtins rcgu:

compiler_builtins.ll.txt

This reproduces the llc crash with assertions enabled:

$ llc compiler_builtins.ll -o test.s --exception-model=wasm -wasm-enable-eh
llc: /checkout/src/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:1378: void mapWasmLandingPadIndex(MachineBasicBlock *, const CatchPadInst *): Assertion `IntrFound && "wasm.landingpad.index intrinsic not found!"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /home/purplesyringa/rust/build/host/ci-llvm/bin/llc compiler_builtins.ll -o test.s --exception-model=wasm -wasm-enable-eh
1.	Running pass 'Function Pass Manager' on module 'compiler_builtins.ll'.
2.	Running pass 'WebAssembly Instruction Selection' on function '@_ZN4core10intrinsics19copy_nonoverlapping18precondition_check17hc5243bdde1bb1f76E'
 #0 0x00007e9503418167 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/purplesyringa/rust/build/x86_64-unknown-linux-gnu/ci-llvm/bin/../lib/libLLVM.so.19.1-rust-1.86.0-nightly+0x5018167)
 #1 0x00007e950341857c SignalHandler(int) (/home/purplesyringa/rust/build/x86_64-unknown-linux-gnu/ci-llvm/bin/../lib/libLLVM.so.19.1-rust-1.86.0-nightly+0x501857c)
 #2 0x00007e94fe24c1d0 (/usr/lib/libc.so.6+0x3d1d0)
 #3 0x00007e94fe2a53f4 (/usr/lib/libc.so.6+0x963f4)
 #4 0x00007e94fe24c120 raise (/usr/lib/libc.so.6+0x3d120)
 #5 0x00007e94fe2334c3 abort (/usr/lib/libc.so.6+0x244c3)
 #6 0x00007e94fe2333df (/usr/lib/libc.so.6+0x243df)
 #7 0x00007e94fe244177 (/usr/lib/libc.so.6+0x35177)
 #8 0x00007e950732d5e5 llvm::SelectionDAGISel::PrepareEHLandingPad() (.cold) bolt-pseudo.o:0:0
 #9 0x00007e9506210a30 llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) (/home/purplesyringa/rust/build/x86_64-unknown-linux-gnu/ci-llvm/bin/../lib/libLLVM.so.19.1-rust-1.86.0-nightly+0x7e10a30)
#10 0x00007e95060e9203 llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) (/home/purplesyringa/rust/build/x86_64-unknown-linux-gnu/ci-llvm/bin/../lib/libLLVM.so.19.1-rust-1.86.0-nightly+0x7ce9203)
#11 0x00007e95060ec6f3 llvm::SelectionDAGISelLegacy::runOnMachineFunction(llvm::MachineFunction&) (/home/purplesyringa/rust/build/x86_64-unknown-linux-gnu/ci-llvm/bin/../lib/libLLVM.so.19.1-rust-1.86.0-nightly+0x7cec6f3)
#12 0x00007e95060ef9f0 llvm::FPPassManager::runOnFunction(llvm::Function&) (/home/purplesyringa/rust/build/x86_64-unknown-linux-gnu/ci-llvm/bin/../lib/libLLVM.so.19.1-rust-1.86.0-nightly+0x7cef9f0)
#13 0x00007e95060ee7b6 llvm::FPPassManager::runOnModule(llvm::Module&) (/home/purplesyringa/rust/build/x86_64-unknown-linux-gnu/ci-llvm/bin/../lib/libLLVM.so.19.1-rust-1.86.0-nightly+0x7cee7b6)
#14 0x00007e95067a3993 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/home/purplesyringa/rust/build/x86_64-unknown-linux-gnu/ci-llvm/bin/../lib/libLLVM.so.19.1-rust-1.86.0-nightly+0x83a3993)
#15 0x00005795b2dc2c43 main (/home/purplesyringa/rust/build/host/ci-llvm/bin/llc+0x15c43)
#16 0x00007e94fe234e08 (/usr/lib/libc.so.6+0x25e08)
#17 0x00007e94fe234ecc __libc_start_main (/usr/lib/libc.so.6+0x25ecc)
#18 0x00005795b2dbff28 _start (/home/purplesyringa/rust/build/host/ci-llvm/bin/llc+0x12f28)
Aborted (core dumped)

I have considered minimizing it, but applying llvm-reduce uncovers more and more odd assertion failures that might be genuine, so I decided against it.

[purplesyringa]

ud2 is due to the second part of llvm/llvm-project#124710.

[aheejin]

I don't know much about how rustc frontend generates the EH instructions such as catchpad and cleanupad. Also the current LLVM backend has only been used with Clang and WasmEHPrepare, which adds Wasm EH intrinsics to be processed by the iSel and the backend.
(To be precise, WasmEHPrepare needs some intrinsics placed by Clang to run. And it adds / removes more intrinsics.)

Judging by the bitcode, it looks rustc currently generates catchpads and cleanuppads but none of those intrinsics expected by the backend. I'm not sure whether those intrinsics make sense for Rust either. To support Rust exceptions correctly with Wasm EH, rustc either needs to support those intrinsics or, if they don't make sense, we need to make another way that makes the backend work.

[purplesyringa]

@aheejin rustc emits (pseudocode):

declare i32 @rust_try(%try_func, %data, %catch_func) {
    %slot = alloca i8*
    invoke %try_func(%data) to label %normal unwind label %catchswitch

normal:
    ret i32 0

catchswitch:
    %cs = catchswitch within none [%catchpad] unwind to caller

catchpad:
    %tok = catchpad within %cs [null]
    %ptr = call @llvm.wasm.get.exception(token %tok)
    %sel = call @llvm.wasm.get.ehselector(token %tok)
    call %catch_func(%data, %ptr)
    catchret from %tok to label %caught

caught:
    ret i32 1
}

This seems to match the input WasmEHPrepare expects. Are you saying that it looks like, for whatever reason, the WasmEHPrepare pass is not run? Or is this IR wrong in some way?

[purplesyringa]

Oh, I see now: this is not the IR we emit in compiler_builtins. We emit something like

  %7 = catchswitch within none [label %8] unwind to caller

8:
  %9 = catchpad within %7 [ptr null, i32 64, ptr null]
  call void @llvm.trap()
  unreachable

which WasmEHPrepare obviously doesn't rewrite.

[purplesyringa]

Asserts should now pass, but the LLVM-side issue (which has been referred above as the ud2 bug) is still unresolved.

[purplesyringa]

llvm/llvm-project#124710 is now fixed -- this issue will probably be resolved by an LLVM update.

@rustbot label +llvm-fixed-upstream