Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C-API: add functions to overwrite stdin / stdout / stderr handlers #3145

Merged
merged 80 commits into from
Oct 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
f3fc438
c-api: add functions to overwrite stdin and to write to it
epilys Jul 21, 2022
b1c77a1
Refactor overriding stdin / stdout / stderr into wasi_console_io_over…
fschutt Jul 25, 2022
759b2f6
Add (non-working) unit test
fschutt Jul 26, 2022
8e5ff7d
Remove wasi_env_write_stdin
fschutt Jul 26, 2022
dccaa3f
Fix C API
fschutt Jul 26, 2022
d0ccb34
Make capi unit test functional, finalize C API
fschutt Jul 26, 2022
660f866
Fix double free() in deallocation of WasiConsoleMemoryOverride
fschutt Jul 29, 2022
127e06d
Rename "wasi_console_io_override_t" to "wasi_console_out_t"
fschutt Jul 29, 2022
5103306
Added API for wasi_stdin_t
fschutt Aug 1, 2022
8c36f06
Finalize API for stdin API rewrite
fschutt Aug 1, 2022
dfe4e8e
Try debuggint unit test test_wasi_stdin_set
fschutt Aug 1, 2022
f4ff4bc
Fixed stdin-capi unit test not running due to missing imports
fschutt Aug 2, 2022
5e83b80
Fix C-API stdin PR: read correct amount of bytes and fix stdout cursor
fschutt Aug 2, 2022
ed4fb72
cargo fmt
fschutt Aug 2, 2022
aa27a66
Fix __u_long to size_t in C API
fschutt Aug 2, 2022
2ff8d34
Change size_t to unsigned long
fschutt Aug 3, 2022
1224700
Use int64_t / uintptr_t
fschutt Aug 5, 2022
f1a3350
cargo update
fschutt Aug 5, 2022
ac127d8
Fix clippy errors in lib/c-api/src/wasm_c_api/wasi/mod.rs
fschutt Aug 5, 2022
18709f2
Merge branch 'master' into capi-wasi-overwrite-stdin
fschutt Aug 17, 2022
838d932
Use wasi::Pipe
fschutt Aug 17, 2022
df05679
First work to translate from WasiConsoleMemoryOverride to Pipe
fschutt Aug 17, 2022
662be61
Implement console_io_memory using wasi::Pipe
fschutt Aug 18, 2022
9c4bc82
Merge branch 'master' into capi-wasi-overwrite-stdin
fschutt Aug 18, 2022
16f7ff7
Address review changes
fschutt Aug 18, 2022
b7f30c9
Fix errors in make test-capi
fschutt Aug 18, 2022
a93f589
Merge branch 'master' into capi-wasi-overwrite-stdin
fschutt Aug 18, 2022
6e944fa
cargo fmt && make lint
fschutt Aug 18, 2022
63d709e
Adress review comment: rename wasi_console_out_new -> wasi_pipe_new
fschutt Aug 22, 2022
61ff302
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Aug 30, 2022
67ed624
Fix make lint
fschutt Aug 30, 2022
a9f1cc8
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Sep 2, 2022
fb1fadb
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Sep 2, 2022
b046463
Implement Write + Seek for WasiPipe
fschutt Sep 6, 2022
fac6d0f
Added WasiPipePair to make the transition to two pipes easier
fschutt Sep 6, 2022
8bf26a3
cargo fmt
fschutt Sep 6, 2022
dcadaf2
Added intial API for creating two-channel pipes
fschutt Sep 6, 2022
2a5d267
Added WasiPipe test to test reading / writing
fschutt Sep 6, 2022
d26edf9
C-API: Implement stdio override with WasiPipe instead of custom impl
fschutt Sep 6, 2022
815da05
Rename wasi_console_out_t to wasi_pipe_t
fschutt Sep 6, 2022
1f4c46b
Fix stdio tests for WasiPipe (currently still failing)
fschutt Sep 7, 2022
d840b5c
Fix unit test with proper compiled .wasm file
fschutt Sep 7, 2022
d8e19ba
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Sep 8, 2022
6f35fb5
"Fix" WasiPipe to allow multiple-write-single-read buffers
fschutt Sep 9, 2022
6b6785b
Fix "make lint"
fschutt Sep 9, 2022
c4a16ae
Fix WasiPipe + examples
fschutt Sep 9, 2022
2c76d73
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Sep 19, 2022
2484db9
Address review comments to prevent leak with Arc::try_unwrap
fschutt Sep 19, 2022
aa4b1d0
Add bytes_available function
fschutt Sep 19, 2022
1825e60
Rename inherit_stdout -> stdout
fschutt Sep 19, 2022
0e53d2d
Rename WasiPipePair -> WasiBidirectionalPipePair
fschutt Sep 19, 2022
238aef3
Remove debug eprintln
fschutt Sep 19, 2022
6439997
Reexport WasiBidirectionalSharedPipePair as Pipe
fschutt Sep 19, 2022
f21689d
Added set_blocking method to control blocking behaviour for WasiPipe
fschutt Sep 19, 2022
72acbc8
cargo fmt
fschutt Sep 19, 2022
497592a
Allow dead code (?)
fschutt Sep 19, 2022
8aa69d2
Try fixing failing JS tests
fschutt Sep 19, 2022
0e5b55f
Set blocking = false for all tests
fschutt Sep 19, 2022
6a91ed7
cargo fmt
fschutt Sep 19, 2022
f1fe308
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Sep 22, 2022
7d44c13
Prevent CI from locking up when running WasiTest
fschutt Sep 22, 2022
3089d4d
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Sep 27, 2022
a90dea6
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Sep 27, 2022
703dbce
Fix ambigouusly named "config" and "stdin" params
fschutt Sep 28, 2022
28c0e9c
cargo fmt
fschutt Sep 28, 2022
14d0614
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Oct 4, 2022
2414636
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Oct 11, 2022
68bc7cc
Fix merge issues
fschutt Oct 11, 2022
3eb2008
cargo fmt
fschutt Oct 11, 2022
0a50a17
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Oct 11, 2022
b20b88b
WIP: prepare buffered reads to Pipe instead of reading data on-demand
fschutt Oct 11, 2022
6b5c746
Fix PR review comments from john
fschutt Oct 12, 2022
e38aca6
Merge branch 'master' into capi-wasi-overwrite-stdin-3
fschutt Oct 14, 2022
2422888
Fix CI issues
fschutt Oct 14, 2022
ace2d9f
cargo fmt
fschutt Oct 14, 2022
67d6501
cargo clippy --fix
fschutt Oct 14, 2022
ba5c1b8
Try fixing CI issue w. TokenStream
fschutt Oct 14, 2022
8951c9d
Use into() instead of TokenStream::from
fschutt Oct 14, 2022
f6fa5b7
"config" -> "wasi_config"
fschutt Oct 14, 2022
05d74ea
Fix WASI pipe to properly store read bytes in temp_buffer
fschutt Oct 17, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions examples/wasi_pipes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
use std::io::{Read, Write};
use wasmer::{Instance, Module, Store};
use wasmer_compiler_cranelift::Cranelift;
use wasmer_wasi::{Pipe, WasiState};
use wasmer_wasi::{WasiBidirectionalSharedPipePair, WasiState};

fn main() -> Result<(), Box<dyn std::error::Error>> {
let wasm_path = concat!(
Expand All @@ -36,8 +36,8 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {

println!("Creating `WasiEnv`...");
// First, we create the `WasiEnv` with the stdio pipes
let mut input = Pipe::new();
let mut output = Pipe::new();
let mut input = WasiBidirectionalSharedPipePair::new().with_blocking(false);
let mut output = WasiBidirectionalSharedPipePair::new().with_blocking(false);
let wasi_env = WasiState::new("hello")
.stdin(Box::new(input.clone()))
.stdout(Box::new(output.clone()))
Expand Down
46 changes: 20 additions & 26 deletions lib/api/tests/externals.rs
Original file line number Diff line number Diff line change
Expand Up @@ -210,28 +210,25 @@ fn memory_grow() -> Result<(), String> {
fn function_new() -> Result<(), String> {
let mut store = Store::default();
let function = Function::new_typed(&mut store, || {});
assert_eq!(
function.ty(&mut store).clone(),
FunctionType::new(vec![], vec![])
);
assert_eq!(function.ty(&mut store), FunctionType::new(vec![], vec![]));
let function = Function::new_typed(&mut store, |_a: i32| {});
assert_eq!(
function.ty(&mut store).clone(),
function.ty(&mut store),
FunctionType::new(vec![Type::I32], vec![])
);
let function = Function::new_typed(&mut store, |_a: i32, _b: i64, _c: f32, _d: f64| {});
assert_eq!(
function.ty(&mut store).clone(),
function.ty(&mut store),
FunctionType::new(vec![Type::I32, Type::I64, Type::F32, Type::F64], vec![])
);
let function = Function::new_typed(&mut store, || -> i32 { 1 });
assert_eq!(
function.ty(&mut store).clone(),
function.ty(&mut store),
FunctionType::new(vec![], vec![Type::I32])
);
let function = Function::new_typed(&mut store, || -> (i32, i64, f32, f64) { (1, 2, 3.0, 4.0) });
assert_eq!(
function.ty(&mut store).clone(),
function.ty(&mut store),
FunctionType::new(vec![], vec![Type::I32, Type::I64, Type::F32, Type::F64])
);
Ok(())
Expand All @@ -246,14 +243,11 @@ fn function_new_env() -> Result<(), String> {
let my_env = MyEnv {};
let env = FunctionEnv::new(&mut store, my_env);
let function = Function::new_typed_with_env(&mut store, &env, |_env: FunctionEnvMut<MyEnv>| {});
assert_eq!(
function.ty(&mut store).clone(),
FunctionType::new(vec![], vec![])
);
assert_eq!(function.ty(&mut store), FunctionType::new(vec![], vec![]));
let function =
Function::new_typed_with_env(&mut store, &env, |_env: FunctionEnvMut<MyEnv>, _a: i32| {});
assert_eq!(
function.ty(&mut store).clone(),
function.ty(&mut store),
FunctionType::new(vec![Type::I32], vec![])
);
let function = Function::new_typed_with_env(
Expand All @@ -262,13 +256,13 @@ fn function_new_env() -> Result<(), String> {
|_env: FunctionEnvMut<MyEnv>, _a: i32, _b: i64, _c: f32, _d: f64| {},
);
assert_eq!(
function.ty(&mut store).clone(),
function.ty(&mut store),
FunctionType::new(vec![Type::I32, Type::I64, Type::F32, Type::F64], vec![])
);
let function =
Function::new_typed_with_env(&mut store, &env, |_env: FunctionEnvMut<MyEnv>| -> i32 { 1 });
assert_eq!(
function.ty(&mut store).clone(),
function.ty(&mut store),
FunctionType::new(vec![], vec![Type::I32])
);
let function = Function::new_typed_with_env(
Expand All @@ -277,7 +271,7 @@ fn function_new_env() -> Result<(), String> {
|_env: FunctionEnvMut<MyEnv>| -> (i32, i64, f32, f64) { (1, 2, 3.0, 4.0) },
);
assert_eq!(
function.ty(&mut store).clone(),
function.ty(&mut store),
FunctionType::new(vec![], vec![Type::I32, Type::I64, Type::F32, Type::F64])
);
Ok(())
Expand All @@ -294,35 +288,35 @@ fn function_new_dynamic() -> Result<(), String> {
&function_type,
|_values: &[Value]| unimplemented!(),
);
assert_eq!(function.ty(&mut store).clone(), function_type);
assert_eq!(function.ty(&mut store), function_type);
let function_type = FunctionType::new(vec![Type::I32], vec![]);
let function = Function::new(
&mut store,
&function_type,
|_values: &[Value]| unimplemented!(),
);
assert_eq!(function.ty(&mut store).clone(), function_type);
assert_eq!(function.ty(&mut store), function_type);
let function_type = FunctionType::new(vec![Type::I32, Type::I64, Type::F32, Type::F64], vec![]);
let function = Function::new(
&mut store,
&function_type,
|_values: &[Value]| unimplemented!(),
);
assert_eq!(function.ty(&mut store).clone(), function_type);
assert_eq!(function.ty(&mut store), function_type);
let function_type = FunctionType::new(vec![], vec![Type::I32]);
let function = Function::new(
&mut store,
&function_type,
|_values: &[Value]| unimplemented!(),
);
assert_eq!(function.ty(&mut store).clone(), function_type);
assert_eq!(function.ty(&mut store), function_type);
let function_type = FunctionType::new(vec![], vec![Type::I32, Type::I64, Type::F32, Type::F64]);
let function = Function::new(
&mut store,
&function_type,
|_values: &[Value]| unimplemented!(),
);
assert_eq!(function.ty(&mut store).clone(), function_type);
assert_eq!(function.ty(&mut store), function_type);

// Using array signature
let function_type = ([Type::V128], [Type::I32, Type::F32, Type::F64]);
Expand Down Expand Up @@ -356,39 +350,39 @@ fn function_new_dynamic_env() -> Result<(), String> {
&function_type,
|_env: FunctionEnvMut<MyEnv>, _values: &[Value]| unimplemented!(),
);
assert_eq!(function.ty(&mut store).clone(), function_type);
assert_eq!(function.ty(&mut store), function_type);
let function_type = FunctionType::new(vec![Type::I32], vec![]);
let function = Function::new_with_env(
&mut store,
&env,
&function_type,
|_env: FunctionEnvMut<MyEnv>, _values: &[Value]| unimplemented!(),
);
assert_eq!(function.ty(&mut store).clone(), function_type);
assert_eq!(function.ty(&mut store), function_type);
let function_type = FunctionType::new(vec![Type::I32, Type::I64, Type::F32, Type::F64], vec![]);
let function = Function::new_with_env(
&mut store,
&env,
&function_type,
|_env: FunctionEnvMut<MyEnv>, _values: &[Value]| unimplemented!(),
);
assert_eq!(function.ty(&mut store).clone(), function_type);
assert_eq!(function.ty(&mut store), function_type);
let function_type = FunctionType::new(vec![], vec![Type::I32]);
let function = Function::new_with_env(
&mut store,
&env,
&function_type,
|_env: FunctionEnvMut<MyEnv>, _values: &[Value]| unimplemented!(),
);
assert_eq!(function.ty(&mut store).clone(), function_type);
assert_eq!(function.ty(&mut store), function_type);
let function_type = FunctionType::new(vec![], vec![Type::I32, Type::I64, Type::F32, Type::F64]);
let function = Function::new_with_env(
&mut store,
&env,
&function_type,
|_env: FunctionEnvMut<MyEnv>, _values: &[Value]| unimplemented!(),
);
assert_eq!(function.ty(&mut store).clone(), function_type);
assert_eq!(function.ty(&mut store), function_type);

// Using array signature
let function_type = ([Type::V128], [Type::I32, Type::F32, Type::F64]);
Expand Down
10 changes: 5 additions & 5 deletions lib/api/tests/reference_types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -367,7 +367,7 @@ pub mod reference_types {
let global: &Global = instance.exports.get_global("global")?;
{
let er = ExternRef::new(&mut store, 3usize);
global.set(&mut store, Value::ExternRef(Some(er.clone())))?;
global.set(&mut store, Value::ExternRef(Some(er)))?;
}
let get_from_global: TypedFunction<(), Option<ExternRef>> = instance
.exports
Expand Down Expand Up @@ -398,7 +398,7 @@ pub mod reference_types {

let er = ExternRef::new(&mut store, 3usize);

let result = pass_extern_ref.call(&mut store, Some(er.clone()));
let result = pass_extern_ref.call(&mut store, Some(er));
assert!(result.is_err());

Ok(())
Expand Down Expand Up @@ -442,7 +442,7 @@ pub mod reference_types {
let result = grow_table_with_ref.call(&mut store, Some(er1.clone()), 10_000)?;
assert_eq!(result, -1);

let result = grow_table_with_ref.call(&mut store, Some(er1.clone()), 8)?;
let result = grow_table_with_ref.call(&mut store, Some(er1), 8)?;
assert_eq!(result, 2);

for i in 2..10 {
Expand All @@ -454,15 +454,15 @@ pub mod reference_types {
}

{
fill_table_with_ref.call(&mut store, Some(er2.clone()), 0, 2)?;
fill_table_with_ref.call(&mut store, Some(er2), 0, 2)?;
}

{
table2.set(&mut store, 0, Value::ExternRef(Some(er3.clone())))?;
table2.set(&mut store, 1, Value::ExternRef(Some(er3.clone())))?;
table2.set(&mut store, 2, Value::ExternRef(Some(er3.clone())))?;
table2.set(&mut store, 3, Value::ExternRef(Some(er3.clone())))?;
table2.set(&mut store, 4, Value::ExternRef(Some(er3.clone())))?;
table2.set(&mut store, 4, Value::ExternRef(Some(er3)))?;
}

{
Expand Down
Loading