Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moves WASI status to the top-level README and reformats motivation #389

Merged
merged 1 commit into from
Mar 17, 2022
Merged

Moves WASI status to the top-level README and reformats motivation #389

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 150 additions & 48 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,48 +71,6 @@ r := wazero.NewRuntimeWithConfig(wazero.NewRuntimeConfigInterpreter())

The below support policy focuses on compatability concerns of those embedding wazero into their Go applications.

### WebAssembly

#### Core API
wazero supports the only WebAssembly specification which has reached W3C
Recommendation (REC) phase: [WebAssembly Core Specification 1.0 (20191205)](https://www.w3.org/TR/2019/REC-wasm-core-1-20191205).

Independent verification is possible via the [WebAssembly spec test suite](https://github.com/WebAssembly/spec/tree/wg-1.0/test/core),
which we run on every change and against all supported platforms.

One current limitation of wazero is that it doesn't fully implement the Text
Format, yet, e.g. compiling `%.wat` files. The intent is to finish this, and
meanwhile users can work around this using tools such as `wat2wasm` to compile
the text format into the binary format.

#### Post 1.0 Features

The last W3C REC was at the end of 2019. There were other features that didn't
make the cut or were developed in the years since. The community unofficially
refers to these as [Finished Proposals](https://github.com/WebAssembly/proposals/blob/main/finished-proposals.md).

To ensure compatability, wazero does not opt-in to any non-standard feature by
default. However, the following status covers what's currently possible with
`wazero.RuntimeConfig`.

| Feature | Status |
|:-------------------------------------:|:------:|
| mutable-global | ✅ |
| nontrapping-float-to-int-conversions | ❌ |
| sign-extension-ops | ✅ |
| multi-value | ❌ |
| JS-BigInt-integration | N/A |
| reference-types | ❌ |
| bulk-memory-operations | ❌ |
| simd | ❌ |

Note: While the above are specified in a WebAssembly GitHub repository, they
are not W3C recommendations (standards). This means they can change further
between now and any future update to the W3C WebAssembly core specification.
Due to this, we cannot guarantee future compatability. Please encourage the
[WebAssembly community](https://www.w3.org/community/webassembly/) to formalize
features you rely on, specifically to read the W3C recommendation (REC) phase.

### wazero

wazero is an early project, so APIs are subject to change until version 1.0.
Expand Down Expand Up @@ -150,14 +108,158 @@ embedded in an application that doesn't use an operating system. For example,
via Docker `FROM scratch`. While we don't currently have tests for this, it is
an intended use case and a major differentiator between wazero and alternatives.

## Background
## Standards Compliance

The [WebAssembly Core Specification 1.0 (20191205)](https://www.w3.org/TR/2019/REC-wasm-core-1-20191205)
is the only part of the WebAssembly ecosystem that is a W3C recommendation.

In practice, this specification is not enough. Most compilers that target Wasm
rely both on features that are not yet W3C recommendations, such as
`bulk-memory-operations` and the [WebAssembly System Interface (WASI)](https://github.com/WebAssembly/WASI),
whose stable point was a snapshot released at the end of 2020. The aim of this
section is to familiarize you with what wazero complies with, and through that
understand the current state of interop that considers both standards (W3C
recommendations) and non-standard features your tooling may use.

### WebAssembly Core
wazero supports the only WebAssembly specification which has reached W3C
Recommendation (REC) phase: [WebAssembly Core Specification 1.0 (20191205)](https://www.w3.org/TR/2019/REC-wasm-core-1-20191205).

Independent verification is possible via the [WebAssembly spec test suite](https://github.com/WebAssembly/spec/tree/wg-1.0/test/core),
which we run on every change and against all supported platforms.

One current limitation of wazero is that it doesn't fully implement the Text
Format, yet, e.g. compiling `.wat` files. The intent is to finish this, and
meanwhile users can work around this using tools such as `wat2wasm` to compile
the text format into the binary format.

#### Post 1.0 Features

The last W3C REC was at the end of 2019. There were other features that didn't
make the cut or were developed in the years since. The community unofficially
refers to these as [Finished Proposals](https://github.com/WebAssembly/proposals/blob/main/finished-proposals.md).

If you want to provide Wasm host environments in your Go programs, currently there's no other choice than using CGO and leveraging the state-of-the-art runtimes written in C++/Rust (e.g. V8, Wasmtime, Wasmer, WAVM, etc.), and there's no pure Go Wasm runtime out there. (There's only one exception named [wagon](https://github.com/go-interpreter/wagon), but it was archived with the mention to this project.)
To ensure compatability, wazero does not opt-in to any non-standard feature by
default. However, the following status covers what's currently possible with
`wazero.RuntimeConfig`.

First, why do you want to write host environments in Go? You might want to have plugin systems in your Go project and want these plugin systems to be safe/fast/flexible, and enable users to
write plugins in their favorite languages. That's where Wasm comes into play. You write your own Wasm host environments and embed Wasm runtime in your projects, and now users are able to write plugins in their own favorite lanugages (AssembyScript, C, C++, Rust, Zig, etc.). As a specific example, you maybe write proxy severs in Go and want to allow users to extend the proxy via [Proxy-Wasm ABI](https://github.com/proxy-wasm/spec). Maybe you are writing server-side rendering applications via Wasm, or [OpenPolicyAgent](https://www.openpolicyagent.org/docs/latest/wasm/) is using Wasm for plugin system.
| Feature | Status |
|:-------------------------------------:|:------:|
| mutable-global | ✅ |
| nontrapping-float-to-int-conversions | ❌ |
| sign-extension-ops | ✅ |
| multi-value | ❌ |
| JS-BigInt-integration | N/A |
| reference-types | ❌ |
| bulk-memory-operations | ❌ |
| simd | ❌ |

However, experienced Golang developers often avoid using CGO because it introduces complexity. For example, CGO projects are larger and complicated to consume due to their libc + shared library dependency. Debugging is more difficult for Go developers when most of a library is written in Rustlang. [_CGO is not Go_](https://dave.cheney.net/2016/01/18/cgo-is-not-go)[ -- _Rob_ _Pike_](https://www.youtube.com/watch?v=PAAkCSZUG1c&t=757s) dives in deeper. In short, the primary motivation to start wazero was to avoid CGO.
Note: While the above are specified in a WebAssembly GitHub repository, they
are not W3C recommendations (standards). This means they can change further
between now and any future update to the W3C WebAssembly core specification.
Due to this, we cannot guarantee future compatability. Please encourage the
[WebAssembly community](https://www.w3.org/community/webassembly/) to formalize
features you rely on, specifically to read the W3C recommendation (REC) phase.

wazero compiles WebAssembly modules into native assembly (JIT) by default. You may be surprised to find equal or better performance vs mature JIT-style runtimes because [CGO is slow](https://github.com/golang/go/issues/19574). More specifically, if you make large amount of CGO calls which cross the boundary between Go and C (stack) space, then the usage of CGO could be a bottleneck.
### WebAssembly System Interface (WASI)

As of early 2022, the WebAssembly System Interface (WASI) has never reached the
recommendation phase of the W3C. The closest to stability is a draft ([snapshot-01](https://github.com/WebAssembly/WASI/tree/snapshot-01))
released in late 2020. Some functions of this draft are used in practice while
some features are not known to be used at all. Further confusion exists because
some compilers (ex GrainLang) import functions not used. Finally, `snapshot-01`
includes features such as "rights" that [will be removed](https://github.com/WebAssembly/WASI/issues/469#issuecomment-1045251844).

For all of these reasons, wazero will not implement all WASI features, just to
complete the below chart. If you desire something not yet implemented, please
[raise an issue](https://github.com/tetratelabs/wazero/issues/new) and include
your use case (ex which language you are using to compile, a.k.a. target Wasm).

| Function | Status | Known Usage |
|:-----------------------:|:------:|---------------:|
| args_get | ✅ | TinyGo |
| args_sizes_get | ✅ | TinyGo |
| environ_get | ✅ | TinyGo |
| environ_sizes_get | ✅ | TinyGo |
| clock_res_get | ❌ | |
| clock_time_get | ✅ | TinyGo |
| fd_advise | ❌ | |
| fd_allocate | ❌ | |
| fd_close | ✅ | TinyGo |
| fd_datasync | ❌ | |
| fd_fdstat_get | ✅ | TinyGo |
| fd_fdstat_set_flags | ❌ | |
| fd_fdstat_set_rights | ❌ | |
| fd_filestat_get | ❌ | |
| fd_filestat_set_size | ❌ | |
| fd_filestat_set_times | ❌ | |
| fd_pread | ❌ | |
| fd_prestat_get | ✅ | TinyGo,`fs.FS` |
| fd_prestat_dir_name | ✅ | TinyGo |
| fd_pwrite | ❌ | |
| fd_read | ✅ | TinyGo,`fs.FS` |
| fd_readdir | ❌ | |
| fd_renumber | ❌ | |
| fd_seek | ✅ | TinyGo |
| fd_sync | ❌ | |
| fd_tell | ❌ | |
| fd_write | ✅ | `fs.FS` |
| path_create_directory | ❌ | |
| path_filestat_get | ❌ | |
| path_filestat_set_times | ❌ | |
| path_link | ❌ | |
| path_open | ✅ | TinyGo,`fs.FS` |
| path_readlink | ❌ | |
| path_remove_directory | ❌ | |
| path_rename | ❌ | |
| path_symlink | ❌ | |
| path_unlink_file | ❌ | |
| poll_oneoff | ✅ | TinyGo |
| proc_exit | ✅ | AssemblyScript |
| proc_raise | ❌ | |
| sched_yield | ❌ | |
| random_get | ✅ | |
| sock_recv | ❌ | |
| sock_send | ❌ | |
| sock_shutdown | ❌ | |


## History of wazero

wazero was originally developed by [Takeshi Yoneda](https://github.com/mathetake)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😄

as a hobby project in mid 2020. In late 2021, it was sponsored by Tetrate as a
top-level project. That said, Takeshi's original motivation is as relevant
today as when he started the project, and worthwhile reading:

If you want to provide Wasm host environments in your Go programs, currently
there's no other choice than using CGO andleveraging the state-of-the-art
runtimes written in C++/Rust (e.g. V8, Wasmtime, Wasmer, WAVM, etc.), and
there's no pure Go Wasm runtime out there. (There's only one exception named
[wagon](https://github.com/go-interpreter/wagon), but it was archived with the
mention to this project.)

First, why do you want to write host environments in Go? You might want to have
plugin systems in your Go project and want these plugin systems to be
safe/fast/flexible, and enable users to write plugins in their favorite
languages. That's where Wasm comes into play. You write your own Wasm host
environments and embed Wasm runtime in your projects, and now users are able to
write plugins in their own favorite lanugages (AssembyScript, C, C++, Rust,
Zig, etc.). As a specific example, you maybe write proxy severs in Go and want
to allow users to extend the proxy via [Proxy-Wasm ABI](https://github.com/proxy-wasm/spec).
Maybe you are writing server-side rendering applications via Wasm, or
[OpenPolicyAgent](https://www.openpolicyagent.org/docs/latest/wasm/) is using
Wasm for plugin system.

However, experienced Golang developers often avoid using CGO because it
introduces complexity. For example, CGO projects are larger and complicated to
consume due to their libc + shared library dependency. Debugging is more
difficult for Go developers when most of a library is written in Rustlang.
[_CGO is not Go_](https://dave.cheney.net/2016/01/18/cgo-is-not-go)[ -- _Rob_ _Pike_](https://www.youtube.com/watch?v=PAAkCSZUG1c&t=757s) dives in deeper.
In short, the primary motivation to start wazero was to avoid CGO.

wazero compiles WebAssembly modules into native assembly (JIT) by default. You
may be surprised to find equal or better performance vs mature JIT-style
runtimes because [CGO is slow](https://github.com/golang/go/issues/19574). More
specifically, if you make large amount of CGO calls which cross the boundary
between Go and C (stack) space, then the usage of CGO could be a bottleneck.