Webassembly Linux Interface (WALI)

This repo serves to prototype an implementation of the WebAssembly Linux Interface. For current range of support, refer here

Overview

WALI is a complete(ish?) abstraction over Linux for WebAssembly that aims to push lightweight virtualization down to even low-level system applications. WALI adopts a layering approach to API design, establishing infrastructure for facilitating WebAssembly-oriented research by virtualizing high-level APIs and seamless build-run-deploy workflows of arbitrary applications. We create a custom modified C standard library (musl libc) that uses WALI and produce a baseline implementation in WAMR

Skip the talk, I want to run some WALI apps!

1. Install dependencies

• Ninja
• Make
• Cmake
• GCC
• WABT

If using apt, run sudo ./apt-install-deps.sh to install above depedencies

2. Build a WALI runtime following these instructions

3. The wasm-apps directory has several popular applications like Bash, Lua, and Sqlite with sample scripts/data for each app. As an example, to run sqlite3:

# Increase the stack size if the program runs out of space
./iwasm -v=0 --stack-size=524288 wasm-apps/sqlite/sqlite3.wasm

Building the Toolchain

Before proceeding, make sure all dependencies are up to date, as detailed in previous section:

There are four major toolchain components:

1. WALI runtime
2. Custom Clang compiler (C -> Wasm-WALI)
3. C-standard library for WALI
4. (Optional) AoT Compiler for WAMR (Wasm-WALI -> WAMR AoT)

If compiling WALI applications is not required and step 1 is required.

Building WALI runtime

We produce a baseline implementation in WAMR. For details on how to implement these native APIs in WAMR, refer here

To build the WALI-enabled WAMR runtime:

git submodule update --init wasm-micro-runtime
make iwasm

An iwasm symlink executable should be generated in the root directory

Building the WALI LLVM compiler

To build the llvm-18 suite:

git submodule update --init llvm-project
make wali-compiler

This is essential for future steps that rely on this build.

Building WALI libc

The wali-musl submodule has detailed information on prerequisites and steps for compiling libc

To build libc:

git submodule update --init wali-musl
make libc

We currently support 64-bit architectures for x86-64, aarch64, and riscv64, with hopes to expand to more architectures.

(Optional) WAMR AoT Compiler

Refer here on steps to build the AoT compiler.

Once completed, you can create a symlink from the root directory:

ln -sf wasm-micro-runtime/wamr-compiler/wamrc wamrc

Compiling Applications to WALI

C Standalones

To compile C to WASM, refer to compile-wali-standalone.sh:

# Compile standalone C file
<path-to-WALI-clang> \
  --target=wasm32-wasi-threads -O3 -pthread \
  `# Sysroot and lib search path` \
  --sysroot=<path-to-wali-sysroot> -L<path-to-wali-sysroot>/lib \
  `# Enable wasm extension features`  \
  -matomics -mbulk-memory -mmutable-globals -msign-ext  \
  `# Linker flags for shared mem + threading` \
  -Wl,--shared-memory -Wl,--export-memory -Wl,--max-memory=67108864 \
  <input-c-file> -o <output-wasm-file>

Since changes are yet to be made to clang/wasm-ld for the wali toolchain, we are using support enabled in wasi-threads target. This will change once a wasm32-linux target is added for WALI.

To indepedently specify compile and link flags, refer to compile-wali.sh used for the test suite

Building the Test Suite

make tests

WALI executables are located in tests/wasm. Native ELF files for the same in tests/elf can be used to compare against the WASM output

WASM Bytecode -> AoT Compilation

Use the WAMR compiler wamrc with the --enable-multi-thread flag to generate threaded code

Running WALI-WASM code

Use any Webassembly runtime that implements WALI to execute the above generated WASM code.

If you built the baseline WAMR implementation from the Makefile, you can use ./iwasm <path-to-wasm-file> to execute the code.

The wasm-apps directory has several popular prebuilt binaries to run. You may also run the test suite binaries detailed here

Compiler Ports

Rust

We support a custom Rust compiler with a wasm32-wali64-linux-musl target. Existing cargo and rustup and required for a successful build. To build rustc, run:

git submodule update --init compiler_ports/rust
make rustc

This adds a new toolchain to rustup named wali with the new target. To compile applications:

cargo +wali build --target=wasm32-wali64-linux-musl

NOTE: Many applications will currently require a custom libc to be patched into Cargo.toml until potential upstreaming is possible.

Miscellaneous

Run WASM code like an ELF binary!

Most Linux distros will allow registration of miscellaneous binary formats. This will greatly simplify all toolchain builds for WALI out-of-the-box and is highly recommended. This is required to compile some applications in our repo To enable this, run the following:

cd misc
source gen_iwasm_wrapper.sh
# Default binfmt_register does not survive reboots in the system
# Specify '-p' option to register with systemd-binfmt for reboot survival
sudo ./binfmt_register.sh

This essentially points Linux to our iwasm interpreter to invoke any WASM/AoT file. More information about miscellaneous binary formats and troubleshooting can be found here

Notable Changes to Application Operation at Runtime

Wasm enforces type-checking unlike C, so type-unsafe C code may not produce the desired output at runtime. In particular, we have observed the following common unsafe occurences:

• Indirect function invocation with function pointers: Unlike C, Wasm's call_indirect performs a runtime type-check which will fail on function invocation with mismatched signatures.
• Variadic function types: Functions that use variadic arguments must ensure argument type consistency (e.g. syscall arguments must all be typecast to long).

Resources

Syscall Information Table

This paper (https://cseweb.ucsd.edu/~dstefan/pubs/johnson:2022:wave.pdf) and its related work section, especially the bit labeled "Modeling and verifying system interfaces"