Cross-compiling raw-dylib doesn't work on Windows

[mati865]

I tried this code:

#![feature(raw_dylib)]

fn main() {
    unsafe {
        MessageBoxA(0, b"hello\0".as_ptr(), "world\0".as_ptr(), 0);
    }
}

#[link(name = "user32", kind = "raw-dylib")]
extern "system" {
    fn MessageBoxA(hwnd: usize, lptext: *const u8, lpcaption: *const u8, flags: u32) -> i32;
}

I expected to see this happen:
Successful compilation

Instead, this happened:

$ PATH=/d/msys64/mingw32/bin:$PATH /c/Users/mateusz/.cargo/bin/rustc test.rs --target i686-pc-windows-gnu
error: Error calling dlltool: program not found

error: aborting due to previous error

Worth mentioning it works fine when building natively from x86_64 to x86_64 target.

Meta

rustc --version --verbose:

rustc 1.67.0-nightly (edf018221 2022-11-02)
binary: rustc
commit-hash: edf0182213a9e30982eb34f3925ddc4cf5ed3471
commit-date: 2022-11-02
host: x86_64-pc-windows-gnu
release: 1.67.0-nightly
LLVM version: 15.0.4

[bjorn3]

rust/compiler/rustc_codegen_llvm/src/back/archive.rs

Lines 404 to 409 in 31d754a

	// We are cross-compiling, so we need the tool with the prefix matching our target
	if sess.target.arch == "x86" {
	"i686-w64-mingw32-dlltool"
	} else {
	"x86_64-w64-mingw32-dlltool"
	}

Looks like rustc expects a toolchain that cross compiles from the host (x86_64 mingw) to the target (i686 mingw) rather than a toolchain native to the target (i686 mingw) like in your case. I guess it could try both executable names.

[mati865]

On Windows toolchains commonly provide only dlltool without prefix but it's possible to cross-generate import library using dlltool for any host with trick discovered by glandium: microsoft/windows-rs@cb8d6a9 (#2016)

[mati865]

I have it on my radar but cannot tell when when I can find free time to do it.

[dpaoliello]

@rustbot claim

[eiderdaus]

I am unsure whether the linked pull request above actually fixed this issue for cross-compilation, but I get the same exact error with the stable-x86_64-pc-windows-gnu toolchain version 1.71.0. In this version, the feature should have been stabilized. Should I open a new issue because this one only regards cross-compilation?

[dpaoliello]

I am unsure whether the linked pull request above actually fixed this issue for cross-compilation, but I get the same exact error with the stable-x86_64-pc-windows-gnu toolchain version 1.71.0. In this version, the feature should have been stabilized. Should I open a new issue because this one only regards cross-compilation?

@eiderdaus what is your host environment (Windows? Linux?) and what dlltool(s) do you have on your PATH?

[mati865]

Looking at tokio-rs/mio#1632 (comment) your issue is a bit different. How did you install mingw-w64 toolchain?
MSYS2 and mingw-w64 builds provide dlltool.exe but other toolchains might provide x86_64-w64-mingw32-dlltool.exe instead. In such case the easiest workaround is to symlink dlltool.exe pointing to your *-dlltool.exe.

[mati865]

Alternatively you could try RUSTFLAGS='-Cdlltool=<your-dlltool-bin>'.

[eiderdaus]

Hi, thanks for the prompt responses! My host is Windows, there's no dlltool on my %PATH%. I used Rustup to install Rust and I found a dlltool.exe in the following folder:

%USERPROFILE%.rustup\toolchains\stable-x86_64-pc-windows-gnu\lib\rustlib\x86_64-pc-windows-gnu\bin\self-contained

I would expect that this should be enough for building working Rust applications. I tried passing this dlltool via RUSTFLAGS and got an error

error: Dlltool could not create import library:
       [redacted]\.rustup\toolchains\stable-x86_64-pc-windows-gnu\lib\rustlib\x86_64-pc-windows-gnu\bin\self-contained\dlltool.exe: CreateProcess

How did you install mingw-w64 toolchain?

I never mentioned any mingw-w64 but actually I do have MSYS2 at my disposal. It's not on my %PATH% at the moment. I just used RUSTFLAGS=-Cdlltool=C:\msys64\clang64\bin\dlltool.exe with success - however I would call this a workaround, not a solution!

[dpaoliello]

I never mentioned any mingw-w64 but actually I do have MSYS2 at my disposal. It's not on my %PATH% at the moment. I just used RUSTFLAGS=-Cdlltool=C:\msys64\clang64\bin\dlltool.exe with success - however I would call this a workaround, not a solution!

Correct, the use of dlltool is considered to be a temporary workaround until the minimum required binutils version supports the import libraries that LLVM generates: #104218 (comment)

I the meantime, having dlltool on your PATH, or using -Cdlltool is required.

[mati865]

I would expect that this should be enough for building working Rust applications.

I'd expect that as well.

I tried passing this dlltool via RUSTFLAGS and got an error

error: Dlltool could not create import library:
       [redacted]\.rustup\toolchains\stable-x86_64-pc-windows-gnu\lib\rustlib\x86_64-pc-windows-gnu\bin\self-contained\dlltool.exe: CreateProcess

This definitely doesn't seem right, might be worth to create issue for having self-contained dlltool working by the default.

[eiderdaus]

I the meantime, having dlltool on your PATH, or using -Cdlltool is required.

Alright, thanks a lot for this information. To be honest I am a little bit disenchanted after the great feature announcement on the Rust blog - I don't normally expect a stabilized feature to still require such workarounds.

This definitely doesn't seem right, might be worth to create issue for having self-contained dlltool working by the default.

Not sure if another issue makes much sense when the whole dlltool invocation should be removed either way if I understood correctly?

[mati865]

Not sure if another issue makes much sense when the whole dlltool invocation should be removed either way if I understood correctly?

Removal of dlltool might take years, in the meantime bundled dlltool should be fixed and used by the default.

[eiderdaus]

Removal of dlltool might take years, in the meantime bundled dlltool should be fixed and used by the default.

I see. Just so I fully understand: If what I described is a new issue in your opinion, then why is this one still open?

[ChrisDenton]

It's been a year since this issue was originally reported. Can we bump the self-contained binutils at least?

[mati865]

Can we bump the self-contained binutils at least?

That's my plan for the near future but it's easier said than done. Though, at current pace it should hit nightly before 2024.

[ChrisDenton]

Thanks for the quick reply! And sorry that updating is still an ordeal.

[kleisauke]

Just wondering, perhaps we could use https://github.com/messense/implib-rs to generate a GNU binutils flavored import library? It seems to work well with cargo-c for the x86_64-pc-windows-gnu target, although the i686-pc-windows-gnu target required a minor patch to function properly.

[mati865]

Does it output import libraries in legacy format in GNU mode?
Naming the modes GNU and MSVC is confusing because GCC didn't invent its own format. It just implemented what MSVC was using two decades ago. Currently, MSVC uses a short import library format that other tools like LLVM mimic.

Ideally, Rust could say that it only works with Binutils that properly support short import libraries, that is 2.40 released almost exactly one year ago. That would free it from all this dlltool nonsense and just rely on the backend like other Windows targets do.

[ChrisDenton]

We'd probably need an MCP to document a minimum supported binutils version but personally I'd be very much in favour of that.

[kleisauke]

Does it output import libraries in legacy format in GNU mode?

Yes, it does. LLVM (and likely MSVC) can link to this without any issues. See for example the simplified testcase here:
https://gist.github.com/kleisauke/38f4a33b77e9f09669c23c8a35f64752

However, the implib crate does not support Arm64EC/Arm64X. So, bumping the minimum supported version of Binutils in Rust to 2.40 is probably the best solution.

[mati865]

Yes, it does. LLVM (and likely MSVC) can link to this without any issues. See for example the simplified testcase here:
gist.github.com/kleisauke/38f4a33b77e9f09669c23c8a35f64752

MSVC and LLVM can use both legacy and short import libs without a problem, I'm worried only about ld.bfd. Some ld.bfd bugs happen only in non-obvious scenarios, like when an import library is included within another library (fixed in 2,40 by Martin).

However, the implib crate does not support Arm64EC/Arm64X. So, bumping the minimum supported version of Binutils in Rust to 2.40 is probably the best solution.

That wouldn't be a problem if implib was used only on windows-gnu.