Create an ABI version for wasm object format

[sbc100]

@sunfishcode proposed that we add an ABI version to the wasm object format.

In fact we probably want several concepts of version:

1. An ABI version. This will allows the linker issue warning when we make changes to the C ABI. This information can come from the clang front-end and probably be stored as metadata in the bit code.

2. Feature flags: To allow optional features in the ABI and for these optional features to be recording in the object format

3. A version for the linking metadata. When we make changes to https://github.com/WebAssembly/tool-conventions/blob/master/Linking.md we would like the linker to be able issue warnings or errors on incompatible object files.

[sbc100]

@sunfishcode does this make sense to you?

[sunfishcode]

Yes, this looks good. It's worth mentioning one of the key motivations here, multiple return values, which we won't be able to use until they're standardized and widely supported, but which we'll want to incorporate into the core C ABI for returning structs by value.

[sbc100]

Do we want to include the target arch too (i.e. WASM32 vs WASM64?) Maybe not since I guess the format itself will need to have some kind of bit for that (if we need to we could always use a flag for 64-bit if the format doesn't grow an indicator)

[sunfishcode]

There isn't a wasm64 proposal yet, but I think the expected direction is that there would be a flag for linear memory declarations specifying an address width. So yeah, we could reuse that here, or we could add our own flag for wasm64.

[sbc100]

I decided to punt on the ABI information initially and land the metadata version field first. It seems logically separate to me.

[dschuff]

I just realized that we never landed ABI versioning, other than for the linking metadata. We should pick it up again.

We have some options on the table:

1. a single ABI version number. Presumably we'd refuse to link objects with different version numbers.
2. Several ABI version numbers (e.g. C, C++, or perhaps even sub-ABIs for C++). Presumably the linker would refuse to link modules where any of the fields didn't match (or, weren't present? the rules could get complex quickly).

[sbc100]

Agreed. It looks like we won't get this landed for LLVM 8.0.0 so perhaps out initial ABI can be represented a missing ABI information ?

[dschuff]

After thinking a bit more, I think our initial ABI can be represented as metadata version 1.
It means C ABI version 1.

When we want to make any change (e.g. new C ABI that includes threads or multi-value) we'll have to finalize whatever scheme we've been discussing here and in #56. That will mean a different metatdata format, so we'll have to update the metadata version too. Then after that we can update the other versions independently.

[alexcrichton]

I've been thinking about wasm ABI issues recently in the context of multiple return values. I know it's been long-expected that the C ABI will change in the future to incorporate multiple return values, but given the progress of multiple returns (now merged in the spec and implemented in many engines) I figured it'd be a good time to think about this again.

My interest currently is to expose the ability to import and export functions from a wasm module that use multiple return values in Rust. I'm less interested in whether the module internally uses multiple return values, but I would like to make sure that the module can still communicate with C code as well (e.g. everyone uses the same clang version). This is not possible in stable Rust today, unfortunately, and I'm trying to figure out what this story might look like.

In Rust it's possible to specify a function's ABI in it's type signature (e.g. extern "C" foo() vs extern "fastcall" foo() or things like that), so one idea I had about exposing the ability for multiple returns as extern "wasm" foo() where "wasm" is a made-up ABI for WebAssembly which does whatever's necessary at the LLVM IR level to do multi-return things. This however involves Rust defining an ABI and I'd prefer to avoid doing that because it seems like C/C++ projects want the same ability to import/export functions with multiple return values.

This leads me to posting here to get others' thoughts on this situation. It seems like we're not comfortable yet turning on multi-value by default, but I think it would be great to support multi-value somehow anyway. I could imagine, for example, that a new "v2" ABI is created here for C which specifies what the ABI would look like with multiple returns (and I would selfishly try to also address #88 at the same time). LLVM could add support for this "v2" ABI but it would not be on-by-default. Rust could then add extern "wasm-v2" which maps to this ABI. Eventually in the future when v2 becomes the default then Rust could add extern "wasm-v1" and extern "wasm-v2" would become equivalent to extern "C". (or something like that)

A situation like this though (rolling out a feature gradually) means that ABIs are a function-level concern rather than a module-level concern. For example we, today, want to ship a precompiled wasi-libc with Rust (and a precompiled Rust libstd) which does not use multivalue to be maximally compatible, but users who want to use multivalue should be able to opt-in to it (e.g. via extern "wasm-v2").

I guess that's sort of a really long-winded way of saying two things:

• Is now the right time to start designing a "v2" ABI which supports multiple return values and address other existing issues?
• I think ABI versions in an object format would want to be on a symbol level rather than a per-module level so new ABIs could be gradually rolled out. That way you're not forced to move the whole world all at once if you don't want.

[sbc100]

I think the original motivation for this issue was to allow the object format itself to evolve, which is somewhat independent of the C ABI. I.e. you can imagine an object format v2 which doesn't all kinds of things differently but has no effect at all on the C ABI or calling convention.

Addressing just the second part of your question regarding the C ABI and calling convention and its granularity and agree that it makes sense to have this apply at the function level. Isn't this already how things like "fastcall" work in LLVM? I imagine they function attributes or something like that in LLVM. However, I'm not so sure we would need to persist that information in the object format. I don't think calling convention information existing ELF or other contains formats does it? One advantage we have in the WebAssembly world is that if you attempt to link two object files which disagree about the declared calling convention of a given function its very likely you will see a link time signature check failure.. which is something ELF can't do.

[alexcrichton]

Ah ok I may have pasted this in the wrong location then! Would it be better if I opened a new issue about getting design of a new C ABI rolling for incorporating mulitple returns?

AFAIK yeah ELF has no mechanism for detecting mismatched ABIs. If that's not a goal of wasm-ld that seems fine by me, this is something I would generally consider to be handled at a higher-level anyway (e.g. in rustc or C typechecking).