[BROKEN] 32-bit x86 (i386) support

[sulix]

I've made a few attempts to get Rust-for-Linux going under 32-bit x86, and have hit some crashes (I think with printk) that I haven't been able to resolve. (Interestingly, it seems to work on 32-bit UML, so that's interesting…)

This currently consists of two patches:

• An implementation of __udivdi3/__umoddi3 to provide 64-bit integer division. This probably should ultimately live in compiler_builtins instead.
• Support in generate_rust_target.rs and the various Makefiles.

I've not yet added either CI support or updated the arch-support.rst documentation.

At the moment, it's crashing in rust_fmt_argument whenever printk is called from Rust:

BUG: kernel NULL pointer dereference, address: 00000414
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
*pde = 00000000
Oops: 0000 [#1] PREEMPT
CPU: 0 PID: 66 Comm: kunit_try_catch Tainted: G                 N 6.1.0-rc1-63059-gc5878a6654a6-dirty #15
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.0-0-gd239552-rebuilt.opensuse.org 04/01/2014
EIP: rust_fmt_argument+0x1e/0x70
Code: 90 90 90 90 90 90 90 90 90 90 90 90 55 89 e5 83 ec 28 8b 45 10 8b 4d 0c 8b 55 08 89 55 f0 89 55 f4 89 4d f8 8d 4d f0 89 4d fc <8b> 48 14 89 4d ec 8b 48 10 89 4d e8 8b 48 0c 89 4d e4 8b 48 08 89
EAX: 00000400 EBX: ffff0a00 ECX: c1097cd8 EDX: c1097dd7
ESI: df11a6ae EDI: c1097e6c EBP: c1097ce8 ESP: c1097cc0
DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00000086
CR0: 80050033 CR2: 00000414 CR3: 1f2ca000 CR4: 00000690
Call Trace:
 ? console_emit_next_record+0x5e/0x410
 pointer+0x4f9/0x6a0
 vsnprintf+0x25d/0x5f0
 ? sched_clock_cpu+0x17/0x290
 vprintk_store+0x125/0x3f0
 ? vprintk_default+0x10/0x20
 vprintk_emit+0x5c/0x1e0
 vprintk_default+0x10/0x20
 vprintk+0x49/0x50
 _printk+0x1f/0x37
 rust_begin_unwind+0x4b/0x70
 ? <&u8 as core[b62776744ad43b56]::fmt::Debug>::fmt+0x60/0x60
 core[b62776744ad43b56]::panicking::panic_fmt+0x2f/0x40
 core[b62776744ad43b56]::result::unwrap_failed+0x6c/0x80
 ? krealloc+0x116/0x200
 ? __switch_to_asm+0xca/0xf0
 ? <&&str as core[b62776744ad43b56]::fmt::Display>::fmt+0x20/0x20
 ? <&[u8; 4: usize] as core[b62776744ad43b56]::fmt::Debug>::fmt+0xd0/0xd0
 rust_kernel_doctest_kasync_executor_workqueue_rs_174_0+0x71c/0x740
 ? __switch_to_asm+0x34/0xf0
 ? __switch_to_asm+0x2e/0xf0
 ? __switch_to_asm+0x28/0xf0
 ? __switch_to_asm+0x22/0xf0
 ? __schedule+0x335/0x3c0
 kunit_try_run_case+0x37/0x90
 kunit_generic_run_threadfn_adapter+0x11/0x20
 kthread+0xa6/0xc0
 ? kunit_try_catch_run+0x130/0x130
 ? kthread_unuse_mm+0x60/0x60
 ret_from_fork+0x1c/0x28
CR2: 0000000000000414
---[ end trace 0000000000000000 ]---

(Out of curiosity, is there a reason rust_fmt_argument is not declared extern "C"? Doing so doesn't fix this issue, but since it's called by the C printk code when %pA appears, I'd've thought it needed to use a C ABI?)

In any case, that issue doesn't seem to show up under UML, and all of the doctest KUnit tests pass:
# rust_kernel_doctests: pass:92 fail:0 skip:0 total:92

I'm testing this with KUnit's tooling, using the following commands:
QEMU/i386:
./tools/testing/kunit/kunit.py run --arch i386 --make_options LLVM=1 --kconfig_add CONFIG_RUST=y
UML/i386:
./tools/testing/kunit/kunit.py run --make_options LLVM=1 --kconfig_add CONFIG_RUST=y --kconfig_add CONFIG_64BIT=n

If anyone has any ideas what could be causing these problems, I'd love to hear them!

[bjorn3]

Out of curiosity, is there a reason rust_fmt_argument is not declared extern "C"? Doing so doesn't fix this issue, but since it's called by the C printk code when %pA appears, I'd've thought it needed to use a C ABI?

It should be extern "C". Good catch!

[bjorn3]

At the moment, it's crashing in rust_fmt_argument whenever printk is called from Rust:

It looks like the test tries to pr_log! a string. When doing so the Display for str impl has an unwrap that fails causing a panic (in the rust sense, so an oops in terms of linux kernel terminology), but when it panics, the code to format the panic message performs a null pointer dereference.

[ojeda]

For x86 32-bit we need -mregparm=3 support. I took a look recently on adding it to rustc, but Clang appeared to mark parameters manually with inreg, with some of the logic being in Clang rather than LLVM. I wanted to discuss it with @nickdesaulniers et al. to see whether there is some cleaner way of doing it without replicating that logic.

[sulix]

For x86 32-bit we need -mregparm=3 support. I took a look recently on adding it to rustc, but Clang appeared to mark parameters manually with inreg, with some of the logic being in Clang rather than LLVM. I wanted to discuss it with @nickdesaulniers et al. to see whether there is some cleaner way of doing it without replicating that logic.

Ah, yup, that'd explain it!

Adding __attribute__((regparm(0))) to rust_fmt_argument fixes the printk crashes (there's still a bunch of panics in individual tests, etc, but the print.rs ones pass now). It's definitely not sensible to try to remove the regparm stuff from the C code more generally, though, so I guess implementing it properly in rustc/llvm is the only real way forward now…

[ojeda]

Yeah, exactly.

[nickdesaulniers]

I wanted to discuss it with @nickdesaulniers et al. to see whether there is some cleaner way of doing it without replicating that logic.

Probably not too bad to re-implement that logic in the front end. I don't see how we simplify the IR for this. Removing inreg is going to be fairly invasive.

[ojeda]

Thanks Nick. I was thinking of adding a module/target option so that LLVM would handle it (while keeping inreg and giving some semantics when specifying both). Or perhaps moving the mregparm= logic that decides whether to add inreg or not from Clang to LLVM, letting Clang call LLVM when it needs to decide, and then call it from rustc too when a -Cregparm or similar is passed. Not sure if either of those would make much sense for LLVM, but if it does, it would be ideal.

[bjorn3]

Rustc handles most abi calculation itself independent of the codegen backend in rustc_target::abi::call. The advantage of this is that it allow sharing the code between all codegen backends.

[nickdesaulniers]

rustc could add inreg for all targets, it just wouldn't meaningfully change codegen unless LLVM knows that inreg means "abi difference from standard calling convention" for the given target.

i.e. if -Cregparm=3 is set, then rustc_target::abi::call should set inreg on the return value and first 2 parameters. Whether LLVM does anything with that parameter attribute is target specific.