Promote the wasm32-wasip2 target to Tier 2

[alexcrichton]

This commit promotes the wasm32-wasip2 Rust target to tier 2 as proposed in rust-lang/compiler-team#760. There are two major changes in this PR:

1. The dist-various-2 container, which already produces the other WASI targets, now has an extra target added for wasm32-wasip2.
2. A new wasm-component-ld binary is added to all host toolchains when LLD is enabled. This is the linker used for the wasm32-wasip2 target.

This new linker is added for all host toolchains to ensure that all host toolchains can produce the wasm32-wasip2 target. This is similar to how rust-lld was originally included for all host toolchains to be able to produce WebAssembly output when the targets were first added. The new linker is developed here and is pulled in via a crates.io-based dependency to the tree here.

[rustbot]

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[alexcrichton]

I'll note that rust-lang/compiler-team#760 is not complete so this shouldn't merge yet. I'm opening this up ahead of time to get feedback on the more technical side of things as opposed to the MCP-side of things.

One specific question I'd have is that this PR is adding more exceptions to the license check for crates that have an Apache-2.0 WITH LLVM-exception license. I understand it's easiest to stick to MIT OR Apache-2.0 like most other Rust crates and such, and to that effect we're going to try to relicense the wasm-tools repository to Apache-2.0 WITH LLVM-exception OR MIT OR Apache-2.0 soon. In lieu of this though, as that will probably take a bit, two questions:

• Is it ok to add more exceptions here in the meantime?
• Is it necessary to go through the relicensing effort? There are a number of other crates such as ar_archive_writer and rustc_apfloat and Cranelift crates which have the same Apache-2.0 WITH LLVM-exception. I don't expect Wasmtime's own license (and thus Cranelift's) to change, it's just the wasm-tools-family of crates that may change. If this is an acceptable state, should we consider it ok to not relicense wasm-tools?

Also I realize that licensing decisions are probably best not made in PRs, so this can also be considered as mostly highlighting the issue and if you know a better forum/location to discuss this I'm happy to raise something there.

[Mark-Simulacrum]

One specific question I'd have is that this PR is adding more exceptions to the license check for crates that have an Apache-2.0 WITH LLVM-exception license

My sense is that we're fine with extra dependencies with that license, especially in the same pool (i.e., wasm-related crates). cc @wesleywiser @davidtwco -- does that align with your expectations? Can you help drive the licensing story with the Foundation's help if needed?

I'm not sure whether re-licensing is worth the trouble or not as such. Though long-term it does likely hurt our ability to move code across the boundary of these crates, so if there's expectations that might be warranted it might be worth investing while there's less people involved (and so it's easier).

---

This is similar to how rust-lld was originally included for all host toolchains to be able to produce WebAssembly output when the targets were first added. The new linker is developed here and is pulled in via a crates.io-based dependency to the tree here.

Can you say how big the extra binary is? I don't remember the rationale back then, but it feels plausible that the linker should be part of the wasm target component... maybe I'm missing something obvious? (lld is distinct in that lots of targets use it -- at least now -- whereas this one seems like that may not be true for).

[bors]

☔ The latest upstream changes (presumably #127133) made this pull request unmergeable. Please resolve the merge conflicts.

[alexcrichton]

Ok sounds good on the licensing front. I opened bytecodealliance/wasm-tools#1637 and bytecodealliance/wit-bindgen#987 earlier anyway to start the process and see generally if it can be done, so that's progressing in parallel too.

As for binary sizes the linker locally is ~5M (4.6 on macos and 5.2 on linux). Making this part of the wasm component would require having a binary for each supported host architeture in the component as well which I think would be pretty difficult build-wise and also would make the wasm component significantly larger. IIRC we wrestled with this at the time when adding rust-lld as well and the decision was to ship it with all host components as that was the simplest solution at the time. Now that was also guided by the fact that we wanted to eventually use LLD for more targets one day (as has since happened) and this won't ever be used for any other target. I can look into size-optimizing the binary if 5M is too large.

[wesleywiser]

My sense is that we're fine with extra dependencies with that license, especially in the same pool (i.e., wasm-related crates). cc @wesleywiser @davidtwco -- does that align with your expectations? Can you help drive the licensing story with the Foundation's help if needed?

Yes, I think that's ok. Obviously, it would be more ideal if we had the standard licensing applied so relicensing would still be preferable but not a blocker. Thanks @alexcrichton for starting that conversation in the bytecodealliance projects!

[rustbot]

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

The list of allowed third-party dependencies may have been modified! You must ensure that any new dependencies have compatible licenses before merging.

cc @davidtwco, @wesleywiser

[alexcrichton]

Ok I've redone a bit how the build is done to fix the build error from before, notably unconditionally using the stage0 compiler for the wasm-component-ld linker to ensure that it avoids the nightly issue with the ahash dependency. The FCP has finished here and otherwise the relicensing efforts are underway and should hopefully be complete in a month or so with new published versions at which point I'll update this to remove the exceptions.

In the meantime this should otherwise be ready for review and I'm happy to dig further into the binary size if that's a concern. The easiest win is to probably compile with -Cpanic=abort which I can try to figure out how to plumb through if desired.

[Mark-Simulacrum]

@bors r+

[bors]

📌 Commit 4cd6eee has been approved by Mark-Simulacrum

It is now in the queue for this repository.

[Mark-Simulacrum]

Let's land this. I think we can update the licensing and consider further improvements in binary size in the future, no need to block on that.

[RalfJung]

There's an incomplete sentence here (and still seems to be there on master).

[alexcrichton]

Oh oops, I'll work on fixing this.

EDIT: #130034