Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: cmd/compile: relax wasm/wasm32 function import signature type constraints #66984

Open
johanbrandhorst opened this issue Apr 23, 2024 · 52 comments

Comments

@johanbrandhorst
Copy link
Member

johanbrandhorst commented Apr 23, 2024

Background

#59149 removed the package restrictions on the use of go:wasmimport, but established strict constraints on the types that can be used as input and result parameters. The motivation for this was that supporting rich types between the host and the client would require sophisticated and expensive runtime type conversions because of the mismatch between the 64 bit architecture of the client and the 32 bit architecture of the host.

With the upcoming 32 bit wasm port, this problem goes away, as both client and host will use 32 bit pointers. However, we can also support a limited set of types on 64 bit platforms, where no runtime conversion is necessary.

Proposal

Relax the constraints on types that can be used as input and result parameters with the go:wasmimport compiler directive. The exact allowed types would depend on whether wasm or wasm32 is used.

We define the "small integer types" group as the set of types described by [u]int[8|16]. The following types would be allowed as input parameters to any wasmimport/wasmexport function:

  • bool
  • int32, uint32, int64, uint64
  • float32, float64
  • string
  • uintptr, unsafe.Pointer, *T where T is an allowed parameter type or one of the small integer types.
  • *struct. All fields of the *struct must be allowed parameter types, struct, [...]T, or one of the small integer types.
    • If the struct has any fields, it must embed structs.HostLayout (see structs: add HostLayout "directive" type #66408).
    • Any struct fields must also embed structs.HostLayout (recursively).
    • *T and string types are only allowed in *struct on wasm32.
  • *[...]T where T is an allowed type or one of the small integer types.

All input parameter types except string are also allowed as result parameter types.

The following types would remain disallowed as input and output parameter types:

  • chan T
  • complex64, complex128
  • func
  • interface
  • map[T]U
  • []T
  • struct
  • [...]T

The conventions established for use of pointers in CGO will be required when using pointers with wasmimport/wasmexport, e.g. the host can read Go memory, can write pointerless data (like the content of a byte buffer) but cannot write Go pointers to Go memory, and cannot hold on to Go pointers unless they are pinned.

Discussion

Compatibility guarantees

The Go spec does not specify the struct layout and leaves it up to implementations to decide. As such, we cannot provide a guaranteed ABI without having to change the spec or force future layout changes to provide runtime conversion of data. This proposal suggests making it clear to users through documentation that there are no guarantees of compatibility across versions of the Go compiler.

Type conversion rules

The following conversion rules would be automatically applied by the compiler for the respective parameter type:

Go Type Parameter type (per Wasm spec)
bool i32
int32, uint32

int64, uint64

i32, i32

i64, i64

float32, float64 f32, f64
string Assigned to two call parameters as a (i32, i32) tuple of (pointer, len). Only allowed for input parameters.
uintptr, unsafe.Pointer, *T, *struct, *[...]T i32, i32, i32, i32, i32

Strings

Strings are not allowed as result parameters as Wasm practically does not allow more than 1 result parameter.

Supporting GOARCH=wasm

The wasm architecture uses 64 bit pointers and integer sizes. As the host uses 32 bit pointers, this makes it impossible to allow certain types without costly runtime conversions, such as *struct types containing pointer fields. Since string types are also pointer types, *struct types containing string fields are also disallowed.

Supporting [u]int, [u]int8, [u]int16 as concrete parameters

The [u]int types are problematic as the size of them are not precisely defined, and may cause confusion when used with strictly 32 bit or 64 bit integers. The [u]int8 and [u]int16 types are problematic because we would be forced to automatically convert them to/from the i32 wasm representation, with potential loss of precision or overflow. They are still allowed as pointer type, array elements and struct fields.

Supporting slices, maps

Both slices and maps are disallowed because of the uncertainty around the memory underlying these types and interactions with struct and array rules. Users who wish to use slices can manually use (&slice, len(slice)) or unsafe.Pointer. There is no clear way to support passing or returning map data from the host other than by using unsafe.Pointer and making assumptions about the underlying data.

Related proposals

struct.Hostlayout

#66408 proposes a way for users to request that struct layout is host compatible. Our proposal depends on the definitions put forward in this proposal for struct parameters.

Future work

WASI Preview 2 (AKA WASI 0.2)

WASI Preview 2 defines its API in terms of the Component Model, with a rich type system and an IDL language, WIT. The Component Model also defines a Canonical ABI with a specification for lifting and lowering Component Model types into and out of linear memory. This proposal does not attempt to define the ABI for any hypothetical wasip2 target, and would leave such decisions for any future wasip2 proposal.

Supporting struct and [...]T by value

A previous version of this proposal included support for passing struct and [...]T types by value by expanding each field recursively into call parameters. This was removed in favor of a simpler initial implementation but could be re-added if users require it.

Contributors

@johanbrandhorst, @evanphx, @achille-roussel, @dgryski, @ydnar

CC @cherrymui @golang/wasm

@gopherbot gopherbot added this to the Proposal milestone Apr 23, 2024
@dr2chase
Copy link
Contributor

dr2chase commented Apr 23, 2024

"This follows the C struct value semantics" is just a hair vague; are 8-byte quantities (float64, int64, uint64) stored at a 4-byte or 8-byte alignment? It was my understanding (and the purpose of #66408) to specify a 4-byte alignment for fields of those types when they occur in structs passed to wasm32 (tagged structs.HostLayout).

(edited to note error, the host alignment for 8-byte integers and floats is 8 bytes).

@ydnar
Copy link

ydnar commented Apr 23, 2024

Ideally 8-byte values would always be 8-byte aligned in the wasm32 port.

@evanphx
Copy link
Contributor

evanphx commented Apr 23, 2024

@dr2chase Looking at what clang does, it uses 8-byte alignment on 64bit quantities so we'd match that.

@dr2chase
Copy link
Contributor

dr2chase commented Apr 23, 2024

You are right, I got it backwards. But that is what you are expecting for anything that has pointers-to-it passed to the wasm host platform, yes?

@cherrymui
Copy link
Member

Thanks for the proposal! A few questions:

  • 8-byte alignment for 64-bit values, as mentioned above. cmd/compile: create GOARCH=wasm32 #63131 doesn't seem to have a definitive answer, and currently it seems the CL doesn't implement 64-bit alignment.
  • If we don't always align 64-bit value to 8 bytes (which differs from current all 32-bit architectures we support and probably requires quite some work), we should align 64-bit value to 8 bytes when structs.HostLayout is specified. So structs: add HostLayout "directive" type #66408 is very related.
  • structs and arrays. What is the ABI specification exactly? The C ABI on, say ELF AMD64, is pretty complex for passing structs and arrays. Small fields may be packed into one word. Large structs may be passed indirectly (stored on stack, passing a pointer to the callee). Do we have a specification for this?
  • string. What does a string look like on Wasm/WASI side? I couldn't find its specification on WASI P1 doc. On Component Model doc https://github.com/WebAssembly/component-model/blob/main/design/mvp/CanonicalABI.md (which I guess is for WASI P2, not P1?), it specifies string is two i32, which is similar to Go's string, which is good. But also it allows three encodings, UTF-8, UTF-16, and "latin1+utf16" differentiated by a high bit. The second and third encoding are not compatible with Go strings. Do we require UTF-8 encoding? Or we don't allow passing Go strings directly?

Besides, for structs, arrays of structs, and pointer to structs, I would suggest we allow only structs with structs.HostLayout to be passed. The reason is that in the Go spec we don't require struct fields to be laid out in memory in source order, and it may well change in a future Go release. structs.HostLayout specifies a fixed layout. Structs without that marker can change. This gives a clear way to say which structs should have a fixed layout, which are okay to change.

Thanks.

@dr2chase
Copy link
Contributor

Two other questions, first:

type w32thing struct {
    _ structs.HostLayout
    a uint8
    b uint16
}

Is this laid out a_bb or is it aaaabbbb? What sizes do I use for struct fields? I assume it is the smaller ones, but I wanted to verify this else it would be a problem.

Second, passing pointers to 8-byte primitive types to the host will be tricky unless those references come from fields in structures tagged with HostLayout -- otherwise, they may not be aligned. So

type wx struct {
   _ structs.HostLayout
  x int64
}
func f(x int64, w wx) {
  someWasmFunc(&x) // might not work, x might not be 8-byte aligned
  someWasmFunc(&w.x) // this will work because w is a wx and its x field is 8-byte aligned
  someOtherWasmFunc(&w) // if it used *wx for its parameter type instead of *int64
}

@johanbrandhorst
Copy link
Member Author

johanbrandhorst commented Apr 25, 2024

Thanks for the quick feedback! I've tried to answer each question:

structs and arrays. What is the ABI specification exactly? The C ABI on, say ELF AMD64, is pretty complex for passing structs and arrays. Small fields may be packed into one word. Large structs may be passed indirectly (stored on stack, passing a pointer to the callee). Do we have a specification for this?

The specification falls out of the table of transformations (I think?). There current plan isn't to introduce any sort of magic around large structs or field packing. Structs fields are added as call parameters, from the first field to the last, according to the conversion rules for the type of the field. Examples:

type foo struct {
    a int
    b string
    c [2]float32
}

With a function signature of

//go:wasmimport some_module some_function
func wasmFunc(in foo) int

Would roughly translate to (in WAT format)

// $a is of type `i32` holding the value of `a`
// $b_addr is of type `i32` and is a pointer to the start of the bytes for the Go string `b`
// $b_len is of type `i32` and is the length in bytes to read from `$b_addr` to get the whole string
// $c_0 is of type `f32` and is the value of `c[0]`
// $c_1 is of type `f32` and is the value of `c[1]`
call $some_function (local.get $a) (local.get $b_addr) (local.get $b_len) (local.get $c_0) (local.get $c_1)

Struct fields would be expanded into call parameters before subsequent fields at the same level.

What does a string look like on Wasm/WASI side?

For wasip1, we will treat Go string parameters simply as a (*byte, int) tuple. There will be no encoding constraints, just as with regular Go strings. To the Wasm host, it will look identical to using struct { a *byte; b int } as a parameter. For wasip2, those constraints would have to be considered in a hypothetical future wasip2 proposal.

Making structs.HostLayout required for structs, arrays of structs and pointers to structs

This sounds like a great idea, and we should also extend it to pointers to 8 byte sized primitive types to guarantee alignment, as suggested by @dr2chase's last question. This would avoid any question around alignment issues for pointers. It hurts the ergonomics a little bit but that's a price worth paying, I think.

type w32thing struct {
_ structs.HostLayout
a uint8
b uint16
}

Is this laid out a_bb or is it aaaabbbb? What sizes do I use for struct fields? I assume it is the smaller ones, but I wanted to verify this else it would be a problem.

I'm a little confused by the question to be honest. If this type was used as an input to a Wasm call, it would look like this:

// $a is of type `i32`
// $b is of type `i32`
call $some_function (local.get $a) (local.get $b)

I suppose that might mean the memory looks like this: a___bb__? We're not passing a pointer to the struct or the fields, so we'd need to copy the values into locals, which will be of type i32 (I think)? Admittedly my grasp of this exact part of the code is a bit weak so I appreciate corrections.

@cherrymui
Copy link
Member

Thanks for the response!

Structs fields are added as call parameters, from the first field to the last, according to the conversion rules for the type of the field.

This sounds like a reasonable choice. Is this ABI specified anywhere in Wasm/WASI docs? Or the Wasm side has to define the function taking parameters element-wise?

For wasip1, we will treat Go string parameters simply as a (*byte, int) tuple. There will be no encoding constraints, just as with regular Go strings. To the Wasm host, it will look identical to using struct { a *byte; b int } as a parameter.

This sounds reasonable as well. Is it specified anywhere in Wasm/WASI docs?

Thanks.

@johanbrandhorst
Copy link
Member Author

This sounds like a reasonable choice. Is this ABI specified anywhere in Wasm/WASI docs? Or the Wasm side has to define the function taking parameters element-wise?

I don't know about this being an official ABI so much as just a consequence of the Wasm spec around function calls and how we can apply Go semantics to it. We're limited to the i32, i64, f32 and f64 value types, and the call instruction takes a function index and arguments from the stack. In order to simulate pass-by-value for structs, we have to flatten each field to one of the allowed value types.

This sounds reasonable as well. Is it specified anywhere in Wasm/WASI docs?

Not sure there's a doc anywhere, but practically, definitions like path_create_directory, which take a string parameter, use this pattern: https://cs.opensource.google/go/go/+/refs/tags/go1.22.2:src/syscall/fs_wasip1.go;l=230.

ydnar added a commit to ydnar/tinygo that referenced this issue May 4, 2024
@dr2chase
Copy link
Contributor

dr2chase commented May 6, 2024

I guess my question is whether a pointer-to-struct is ever passed from Go to the WASM platform, and therefore, what expectations the WASM side has about the layout of the fields of that structure. structs.HostLayout is intended to obtain those expectations, but (1) do we even need to do this? We thought we did, and (2) we need to know what the expectations are. I think it was just that 64-bit floats and ints get 64-bit alignment.

I don't think this is for specifying the layout that gets passed to a WASM call if the struct is passed by value.

@cherrymui
Copy link
Member

I don't know about this being an official ABI so much as just a consequence of the Wasm spec around function calls and how we can apply Go semantics to it. We're limited to the i32, i64, f32 and f64 value types, and the call instruction takes a function index and arguments from the stack. In order to simulate pass-by-value for structs, we have to flatten each field to one of the allowed value types.

As the ABI doesn't have a way to pass struct by value, do we need to support it? If users on the other (non-Go) side have to define the function as taking arguments element-wise with primitive types and pointers, it would probably be better to define the same way on the Go side. Does any other language have a Wasm/WASI interface that allows passing struct by value?

(Same applies for arrays. Pointer to struct/array is fine.)

@johanbrandhorst
Copy link
Member Author

I guess my question is whether a pointer-to-struct is ever passed from Go to the WASM platform, and therefore, what expectations the WASM side has about the layout of the fields of that structure.

I think the biggest concern around this is that all 64 bit values use 8 byte alignment, as you say. We definitely want this, so I think that on its own makes the case for structs.HostLayout. For other values, I think we want to just use "natural alignment" (4 byte for 4 byte values, etc). As far as we know, there is no strict enforcement of this in Wasm generally, but this is the approach taken by LLVM, so it probably makes sense for us to keep it the same.

I also don't know that it's an important question for this proposal in particular, since the answer is pretty clear regarding what we should be passing in the call instruction when encountering a pointer (an i32). I'm happy to weigh in on #66408 if needed to have this discussion though.

As the ABI doesn't have a way to pass struct by value, do we need to support it?

It's fair to say that we can just not support structs and arrays by value, their use are likely to be limited (why not use a pointer?), and it would significantly simplify the implementation. We can come back to it if we need to later. I'll update the proposal.

@aykevl
Copy link

aykevl commented May 13, 2024

On the TinyGo side we're working on an implementation of this proposal, so here's my perspective on it from TinyGo:

  • Always passing structs by reference (pointer) seems like a good idea. Passing structs by value gets complicated quickly and if we agree on something in the future it's easy enough to add it in a new proposal. Structs in memory are relatively well defined in comparison.
  • Strings in TinyGo are already of the format specified in this proposal, so that's nice. We won't need to do anything special there.
  • TinyGo uses the host layout (it's based on LLVM) so TinyGo needs no struct.HostLayout pragma. But it would be relatively simple to add a check like that to ensure compatibility, and if that is what the proposal ends up with I'll make sure we will have the same strict checking.
  • TinyGo has always had a 32-bit wasm implementation (int, uintptr and pointers are 32-bit). Therefore, it would make sense to allow these values at all times. That's a possible compatibility concern, but in essence we're already incompatible so I'm not sure how much of an issue this is. Thoughts?

Question: what fields would be allowed in these structs? I would assume a struct with a chan field would be disallowed, for example. This isn't part of the proposal yet though, so perhaps this can be added?
Something like this:

Structs may not be passed by value, but pointers to structs are allowed. Every field in a struct must be one of the allowed parameter types, or be a struct (recursively).

@cherrymui
Copy link
Member

TinyGo has always had a 32-bit wasm implementation (int, uintptr and pointers are 32-bit). Therefore, it would make sense to allow these values at all times.

I think this is fine. And we should allow them in Go gc toolchain for the "wasm32" port.

Structs may not be passed by value, but pointers to structs are allowed. Every field in a struct must be one of the allowed parameter types, or be a struct (recursively).

Yeah, something along this line makes sense. And also for arrays. I'd say a struct field or a struct pointed by a field should also have the HostLayout marker (because the marker is not recursive).

@johanbrandhorst
Copy link
Member Author

Thanks for your thoughts Ayke, it's always appreciated.

TinyGo has always had a 32-bit wasm implementation (int, uintptr and pointers are 32-bit). Therefore, it would make sense to allow these values at all times. That's a possible compatibility concern, but in essence we're already incompatible so I'm not sure how much of an issue this is. Thoughts?

As Cherry says, these values will be allowed since this proposal is restricted to the wasm32 architecture. The wasm architecture will not have these new relaxations applied. I'm not sure I understand the incompatibility?

Structs may not be passed by value, but pointers to structs are allowed. Every field in a struct must be one of the allowed parameter types, or be a struct (recursively).

I'd say a struct field or a struct pointed by a field should also have the HostLayout marker (because the marker is not recursive).

I've added some clarifying words to the proposal, please take a look!

@aykevl
Copy link

aykevl commented May 17, 2024

The updated proposal looks good to me!
(If I'm very pedantic, it doesn't explicitly say that a struct in a struct is allowed, though it clearly should be. Right now it says *struct is allowed but struct isn't).

However, I have to say that @ydnar has pointed out that the Canonical ABI also allows structs, and it would be nice to have them supported in //go:wasmimport. That said, if I'm reading the specs correctly, the Canonical ABI and the C ABI are incompatible when it comes to structs: the C ABI passes structs by value only when it contains only one field after flattening, while the Canonical ABI passes records (similar to structs) by value if the number of fields is 16 or less after flattening. So that means //go:wasmimport would have to choose between the C ABI and the Canonical ABI.

As Cherry says, these values will be allowed since this proposal is restricted to the wasm32 architecture. The wasm architecture will not have these new relaxations applied. I'm not sure I understand the incompatibility?

Nevermind, TinyGo doesn't even support GOOS=js GOARCH=wasm tinygo ..., it just uses tinygo -target=wasm. So in essence tinygo -target=wasm ... is equivalent to GOOS=js GOARCH=wasm32 go .... Basically it has always been a GOOS=wasm32 implementation and never supported what would be GOOS=wasm (with 64-bit pointers).

I'd say a struct field or a struct pointed by a field should also have the HostLayout marker (because the marker is not recursive).

Seems like a good idea. It's easier to remove such a restriction in the future (if it turns out to be unnecessary) than it is to introduce it later. But I don't know Go internals well enough to say it is needed.

@johanbrandhorst
Copy link
Member Author

To comment quickly on the Canonical ABI: it doesn't relate to this proposal directly as this proposal only targets the wasip1 port, and the Canonical ABI is a preview 2 document (as far as I know). A hypothetical wasip2 proposal would have to tackle type constraints for go:wasmimport (and go:wasmexport) as they relate to the Canonical ABI.

@cherrymui
Copy link
Member

I'm okay with supprting passing structs by value if there is a widely used ABI that is not too complex (if it is as complex as the ELF C ABI on amd64, I'm not sure). If currently there is no widely agreed ABI for structs, we can wait. We can always add things later.

@ydnar
Copy link

ydnar commented May 18, 2024

Hi, original author of the relaxed type constraints proposed here.

The TinyGo PR where this originated depends on LLVM to flatten structs and arrays. This works in practice most of the time, except when it doesn't: namely the Component Model and WASI 0.2 extensively uses tagged unions (variant types in WIT).

The code generator (wit-bindgen-go) implements the flattening rules as specified in the Canonical ABI, which then leans on LLVM to flatten the Go structs that represent variant types.

The CABI flattening rules are per-field, so if a variant has a case that includes a 64-bit wide field, then the flattened representation of the variant must use an i64 at that position.

Given that the compiler is ignorant of the CABI layout, this strategy cannot correctly represent these variant types when passed by value.

@cherrymui: LLVM does correctly flatten structs and arrays consistent with the CABI spec (my sense is the former informed the latter). If we want to start with a more constrained set of types now and relax later, we can make that work.

@aykevl
Copy link

aykevl commented May 20, 2024

LLVM does correctly flatten structs and arrays consistent with the CABI spec (my sense is the former informed the latter).

Not exactly. If you pass a LLVM struct like {i32, i32} by value, LLVM will happily flatten the struct and pass it as values. But if you do that in C, Clang will pass the struct by reference, not by value: it will reserve some space on the stack and pass a pointer instead.
See: https://godbolt.org/z/YjKj5o3c4

I believe this is why the Component Model lowers everything to bare i32/i64/f32/f64/pointer values in function signatures, which have no ambiguity in what ABI they should have on the WebAssembly level.

ydnar added a commit to ydnar/tinygo that referenced this issue Jun 23, 2024
ydnar added a commit to ydnar/tinygo that referenced this issue Jun 23, 2024
This reflects the relaxed type restrictions proposed here (except for structs.HostLayout):
golang/go#66984
@ydnar
Copy link

ydnar commented Jun 23, 2024

With the exception of the check for structs.HostLayout, I’ve implemented the rules as defined in this proposal here in TinyGo: dgryski/tinygo@6921ce6

Initial (shallow) tests against WASI 0.2 APIs seem to work.

@johanbrandhorst
Copy link
Member Author

I've updated the proposal to explicitly state that [...]T are allowed as struct fields when a pointer of that type is used as an input parameter.

@cherrymui
Copy link
Member

The updated proposal looks good. Thanks!

For consistency, it may still be reasonable to apply similar relaxations on GOARCH=wasm. wasip1/wasm is probably not going to go away very soon (especially given that wasm32 isn't available yet).

  • For most primitive types (bool, int8, uint8, int16, uint16, int32, uint32, int64, uint64) it is straightforward.
  • int and uint are 64-bit on GOARCH=wasm, so they probably should be converted to Wasm i64. Or we continue disallowing them for now.
  • Pointers are 64-bit on GOARCH=wasm, but we convert to 32-bit on the host side. We currently allow unsafe.Pointer and apply the conversion. We could also allow *T and do the same conversion.
  • For *struct, if it contains a pointer field, the field would be 64-bit on the Go side and 32-bit on the host side, so there is a mismatch. HostLayout probably should not shrink the field size, either. So I think we'll disallow structs with pointers on GOARCH=wasm.
  • For *struct containing only allowed fields and no pointers, we can allow it if it embeds HostLayout.
  • strings could be allowed if directly used as arguments or results. We apply the same conversion to (i32, i32) on the host side. Strings contains pointers, so for the reason above string-containing structs are not allowed.

@johanbrandhorst
Copy link
Member Author

Thanks for the thorough summary, I agree that this would clearly provide value to users. I've updated the proposal to detail the aim to support both wasm and wasm32. For now, we allow precise primitive types (excluding int and uint), and *struct not containing pointer or string fields. Just as on wasm32, struct types and any struct fields must recursively embed HostLayout.

@johanbrandhorst johanbrandhorst changed the title proposal: cmd/compile: relax wasm32 function import signature type constraints proposal: cmd/compile: relax wasm/wasm32 function import signature type constraints Sep 11, 2024
@rsc
Copy link
Contributor

rsc commented Sep 11, 2024

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

The proposal is #66984 (comment)

@aclements
Copy link
Member

I think this is looking really good. A few thoughts:

  • I'm slightly concerned about allowing int and uint. The argument is that on wasm32, they're 32 bits in Go, so can be safely passed as i32, but the size of int and uint aren't specified by the language (the spec only says that they can be 32 or 64 bits). Of course, the platform is what defines the size of these, so I could see an argument that wasm32 defines them to be 32 bits and thus interchangeable as an i32. But by the logic of that argument, wasm should define them to be 64 bits and thus interchangeable as an i64.
  • What happens if the user passes a *struct containing a smaller integer type like uint8? If uint8 is passed as a parameter, the proposal defines that it gets expanded to an i32, but we can't change the size of the field in the struct. Is wasm prepared to receive a pointer to a struct containing a field smaller than 4 bytes?
  • If Go calls a wasm function that returns, say, a Go uint8, this proposal forces a narrowing from wasm's i32 result value to the Go uint8. (There's a symmetric case with wasm passing an i32 to a Go function that takes a uint8.) In regular Go code, we always require an explicit conversion to convert between different size integer types. It seems like we should follow that same design principal here, which would suggest that [u]int8 and [u]int16 should not be allowed as parameter types.

@ydnar
Copy link

ydnar commented Sep 19, 2024

I think this is looking really good. A few thoughts:

  • I'm slightly concerned about allowing int and uint. The argument is that on wasm32, they're 32 bits in Go, so can be safely passed as i32, but the size of int and uint aren't specified by the language (the spec only says that they can be 32 or 64 bits). Of course, the platform is what defines the size of these, so I could see an argument that wasm32 defines them to be 32 bits and thus interchangeable as an i32. But by the logic of that argument, wasm should define them to be 64 bits and thus interchangeable as an i64.

If developer chooses to use int, uint or uintptr in a wasmimport or wasmexport signature, I think we can assume they know what they’re doing, and expect a 32-bit or 64-bit representation depending on GOARCH.

Edit: this proposal constraints int and uint to GOARCH=wasm32, so until there’s wasm64, I think this is moot?

  • What happens if the user passes a *struct containing a smaller integer type like uint8? If uint8 is passed as a parameter, the proposal defines that it gets expanded to an i32, but we can't change the size of the field in the struct. Is wasm prepared to receive a pointer to a struct containing a field smaller than 4 bytes?

The struct isn’t directly passed over the ABI, so it doesn’t matter. That’s where structs.HostLayout comes in. Some context: #63131 (comment)

@aclements
Copy link
Member

Edit: this proposal constraints int and uint to GOARCH=wasm32, so until there’s wasm64, I think this is moot?

Since I'm not deep in wasm, it took me a bit to understand your argument here. I think the key context is that we've defined GOARCH=wasm to be "32-bit wasm external interfaces, but it looks like a 64-bit CPU to Go programs" and int and uint are a place where these conflict, so we define that they cannot be passed on GOARCH=wasm. Is that right?

The struct isn’t directly passed over the ABI, so it doesn’t matter. That’s where structs.HostLayout comes in. Some context: #63131 (comment)

Thanks. The struct layout is still part of the ABI, but you're right that HostLayout will tell Go to use whatever the wasip2 ABI for struct layout is, including for these smaller types (which happens to align with Go's current layout rules except on 64-bit types).

@johanbrandhorst
Copy link
Member Author

A bit late to the latest discussion, but I'm happy to restrict the use of int and uint on both GOARCH=wasm and GOARCH=wasm32 to avoid any ambiguity. We've already disallowed it on GOARCH=wasm. I've updated the proposal.

@johanbrandhorst
Copy link
Member Author

I missed the question about [u]int8 and [u]int16, I think that's also fair (disallowing them to avoid confusion). We can come back to these if the prove to be useful to the community.

@ydnar
Copy link

ydnar commented Sep 20, 2024

Removing [u]uint8 and [u]int16 may have had the unintended side-effect of disallowing a common use case: passing *byte to a wasmimport call.

The constraints for *T limit T to allowed types, or structs that contain only allowed types. By removing [u]int8 and [u]int16 from the allowed type list, *T where T is [u]int8 or [u]int16 is no longer allowed. Nor would structs containing these types. Is that our intention here?

Also, how should the proposal deal with string (a tuple of *uint8 and uintptr)?

@johanbrandhorst
Copy link
Member Author

Great points Randy, I've updated the proposal to allow [u]int[8|16] as pointer types and struct fields. That will avoid the implicit conversion necessary to support these types as concrete parameters while allowing their use for structs and pointers.

@aclements
Copy link
Member

I think *[...]T where T is an allowed type. should also allow [u]int{8,16}. It might be worth just giving a name to the set of types that are allowed via indirection.

@johanbrandhorst
Copy link
Member Author

Thanks, I defined the small integer types group and made it allowed for array elements.

@ydnar
Copy link

ydnar commented Sep 27, 2024

Can we consider limiting the structs.Hostlayout requirement to structs with > 1 field? This would allow:

  • structs with fields of type struct{}
  • structs that embed a single struct with structs.HostLayout

@johanbrandhorst
Copy link
Member Author

johanbrandhorst commented Sep 27, 2024

I don't see a problem with allowing *struct{} without embedding structs.HostLayout. Not sure about structs with a single field. I've updated the proposal to allow *struct{} without the embedding.

@aclements
Copy link
Member

Thanks for the updates!

One nit, in "Strings are not allowed as result parameters as Wasm practically does not allow more than 1 result parameter." the link points to HEAD, which has drifted. It looks like you meant to link to https://go.googlesource.com/go/+/refs/tags/go1.23.0/src/cmd/internal/obj/wasm/wasmobj.go#219.

@johanbrandhorst
Copy link
Member Author

Thanks for the updates!

One nit, in "Strings are not allowed as result parameters as Wasm practically does not allow more than 1 result parameter." the link points to HEAD, which has drifted. It looks like you meant to link to https://go.googlesource.com/go/+/refs/tags/go1.23.0/src/cmd/internal/obj/wasm/wasmobj.go#219.

Thanks for the thorough check, updated 😄

@aclements
Copy link
Member

The proposal is #66984 (comment). Have all remaining concerns about this proposal been addressed?

@cherrymui
Copy link
Member

Thanks for the discussion and update, @johanbrandhorst , @ydnar , and @aclements !

bool is also somewhat similar to the "small integer types", in that it is i32 on the Wasm side (although converting a bool to a (u)int32 needs more code than a simple conversion expression). Should we treat bool one of the "small integer types"? Or is it too inconvenient to disallow bool to be passed directly?

@ydnar
Copy link

ydnar commented Oct 4, 2024

I can see reasonable arguments for and against allowing small integers and bool values to be included in the list of allowed param types.

The conversion between i32 and bool values is straightforward and can be documented, as can the expansion or truncation of small integers and i32.

LLVM handles this for TinyGo today, so we have a pattern to follow. We have working code that uses bool today (as uint8 either 0 or 1).

If simplifying this proposal is preferred, maybe removing bool from allowed param list is acceptable.

@aclements
Copy link
Member

My sense is that bool is okay. My main concern with int8/int16 was that we'd be performing implicit widening and narrowing, which seemed against Go's general requirement that any change in the domain of a numeric value requires an explicit conversion. With bool <-> i32, they're entirely different kinds, rather than just different domains of the same kind, so it doesn't seem likely a programmer would accidentally pass a number instead of a boolean.

ydnar added a commit to ydnar/tinygo that referenced this issue Oct 5, 2024
golang/go#66984

- Remove int and uint as allowed types in params, results, pointers, or struct fields
- Only allow small integers in pointers, arrays, or struct fields
ydnar added a commit to bytecodealliance/wasm-tools-go that referenced this issue Oct 5, 2024
uint was removed from the list of permitted types in golang/go#66984.
@ydnar
Copy link

ydnar commented Oct 5, 2024

I implemented the current version of this spec in TinyGo, and found one issue in an upstream repo, fixed here (uintuintptr).

Edit: Another way to address this could be to explicitly declare that the size of int and uint always equal the size of uintptr on wasm32, which would allow int and uint as valid types in this proposal. This would be consistent with how string is explicitly permitted (which implicitly permits int).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Likely Accept
Development

No branches or pull requests

9 participants