Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support reference types in the C API #1996

Merged
merged 17 commits into from
Jul 13, 2020

Conversation

fitzgen
Copy link
Member

@fitzgen fitzgen commented Jul 8, 2020

See each commit message for details. The first commit in particular has a bit of rippling out due to wasm_val_t not being POD anymore.

@fitzgen fitzgen requested a review from alexcrichton July 8, 2020 21:50
@github-actions github-actions bot added the wasmtime:c-api Issues pertaining to the C API. label Jul 8, 2020
@github-actions
Copy link

github-actions bot commented Jul 8, 2020

Subscribe to Label Action

cc @peterhuene

This issue or pull request has been labeled: "wasmtime:c-api"

Thus the following users have been cc'd because of the following labels:

  • peterhuene: wasmtime:c-api

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

crates/c-api/src/func.rs Outdated Show resolved Hide resolved
crates/c-api/src/global.rs Outdated Show resolved Hide resolved
Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! (especially the doc updates and examples)

I'm left with a feeling though that this is a bit overly-complicated for externref in the C API. I'm a bit worried about the multiple layers of allocations and indirection with Option and such. I think it's also a bit confusing to try to represent both functions and externref in wasm_ref_t just because that's the way the C API is right now.

A possible alternative that I've been thinking of would look something like this:

  • The wasm_ref_t is still either a funcref or externref, but it's not tagged internally with what it is.
  • Usage of *mut wasm_ref_t is given meaning in wasm_val_t based on the union's tag
  • Usage of *mut wasm_ref_t in wasm_table_* APIs is given meaning based on the table's type (this is a vector for UB since we're assuming you got things right).
  • We could add wasmtime_table_* functions which operate on wasm_val_t (similar to our own APIs in the Rust API) to "fix" the UB
  • The representation of funcref would exactly be wasm_func_t, so the wasm_ref_t pointer would simply be a casted wasm_func_t
  • The representation of externref would ideally be the *mut VMExternData underlying pointer, but we'd have to add Rust APIs for this (which while doing that seems plausible to me feels like we can defer that to later and do Box for now).
  • The wasm_ref_* APIs would probably all continue to just abort the process.

In my opinion the existing APIs in wasm.h are basically not suitable for extenref/funcref right now. They might make more sense in a future world with anyref but it feels premature to design around that possible future.

crates/c-api/include/wasmtime.h Outdated Show resolved Hide resolved
crates/c-api/include/wasmtime.h Outdated Show resolved Hide resolved
crates/c-api/src/func.rs Outdated Show resolved Hide resolved
crates/c-api/src/ref.rs Outdated Show resolved Hide resolved
crates/c-api/src/ref.rs Outdated Show resolved Hide resolved
crates/c-api/src/ref.rs Show resolved Hide resolved
crates/c-api/src/ref.rs Outdated Show resolved Hide resolved
crates/c-api/src/ref.rs Outdated Show resolved Hide resolved
crates/c-api/src/val.rs Outdated Show resolved Hide resolved
@fitzgen
Copy link
Member Author

fitzgen commented Jul 9, 2020

I think I am going to go only one step towards the "raw" wasm_val_t and wasm_ref_t representation you're describing: removing the double Option bits, but not getting rid of WasmRefInner and its type tagging. This way, wasm_ref_copy and wasm_ref_delete can still be safely implemented, which I think is valuable, even if a bunch of the other wasm_ref_* stuff is not worthwhile.

@alexcrichton
Copy link
Member

Hm ok that can work, but trying to think through this, presumably the reason is that wasm_table_* take/return wasm_ref_t, right? For wasm_table_set I think we should not take ownership anyway, so it's less of an issue there, but for wasm_table_get you'd need some way to delete the reference that you acquire. Without tagging we don't know, in isolation, what to delete there.

Given that it seems that to support that API we'd need to have internal tagging. I'll raise an issue on the wasm-c-api repo as well.

crates/c-api/src/val.rs Outdated Show resolved Hide resolved
examples/externref.c Outdated Show resolved Hide resolved
This required that `wasm_val_t` have a `Drop` implementation, an explicit
`Clone` implementation, and no longer be `Copy`, which rippled out through the
crate a bit.

Additionally, `wasm_func_call` and friends were creating references to
uninitialized data for its out pointers and assigning to them. As soon as
`wasm_val_t` gained a `Drop` impl and tried to drop the old value of the
assignment (which is uninitialized data), then things blew up. The fix is to
properly represent the out pointers with `MaybeUninit`, and use `ptr::write` to
initialize them without dropping the old data.

Part of bytecodealliance#929
This commit adds APIs to create new `externref` values (with and without
finalizers) and to get an `externref`'s wrapped data.
The C API prefers not to return structs by value.

Same for `wasmtime_externref_new_with_finalizer`.
…erence

Same for `wasm_table_grow` and `wasm_table_new` and their `init` values.
Use `anyhow` for nice errors and provide error context on commands that we run.
It is created by the `run-examples` crate.
The canonical examples tests will be the examples run via `cargo run -p
run-examples` from now on. Those tests have two advantages over the ones being
deleted:

1. They will automatically pick up new examples and run/test them.
2. They support all of Linux, MacOS, and Windows, while the tests being deleted
   are Linux only.
@fitzgen fitzgen force-pushed the ref-types-in-c-api branch 2 times, most recently from c53a6a6 to 509092d Compare July 13, 2020 16:14
We were accidentally always running the `fib-debug/main` example because of
shenanigans with alphabetical ordering and hard coding "main.exe" as the command
we run. Now we properly detect which example we built and run the appropriate
executable.
@fitzgen fitzgen merged commit 9bafb17 into bytecodealliance:main Jul 13, 2020
@fitzgen fitzgen deleted the ref-types-in-c-api branch July 13, 2020 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wasmtime:c-api Issues pertaining to the C API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants