Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use versioning enum for WASM executor input and output #1830

Merged
merged 3 commits into from
Apr 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Description of the upcoming release here.

### Changed

- [#1830](https://github.com/FuelLabs/fuel-core/pull/1830): Use versioning enum for WASM executor input and output.
- [#1816](https://github.com/FuelLabs/fuel-core/pull/1816): Updated the upgradable executor to fetch the state transition bytecode from the database when the version doesn't match a native one. This change enables the WASM executor in the "production" build and requires a `wasm32-unknown-unknown` target.
- [#1812](https://github.com/FuelLabs/fuel-core/pull/1812): Follow-up PR to simplify the logic around parallel snapshot creation.
- [#1809](https://github.com/FuelLabs/fuel-core/pull/1809): Fetch `ConsensusParameters` from the database
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion crates/services/upgradable-executor/src/executor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -386,7 +386,11 @@ where
.add_relayer(relayer)?
.add_input_data(block, options)?;

instance.run(module)
let output = instance.run(module)?;

match output {
fuel_core_wasm_executor::utils::ReturnType::V1(result) => result,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that the method signature defines the return type as ExecutorResult<..>, and not a generic/dyn, it appears that you will aways want the version variant of output to wrap ExecutorResult or implement Into<ExecutorResult>.

Assuming the signature is correct, then you can probably just move the match into a function (e.g., fn result(..)) on the implementation of ResultType, and call output.result().into() here at the call site.

But if the return type in the method signature may change in the future, I think it would make sense to reflect that change in this PR, to communicate that the type depends on the ReturnType version.

}

fn native_execute_inner<TxSource>(
Expand Down
81 changes: 17 additions & 64 deletions crates/services/upgradable-executor/src/instance.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ use fuel_core_types::{
use fuel_core_wasm_executor::utils::{
pack_exists_size_result,
unpack_ptr_and_len,
InputType,
ReturnType,
};
use std::{
Expand Down Expand Up @@ -424,7 +425,6 @@ impl Instance<Storage> {
/// This stage adds host functions - getter for the input data like `Block` and `ExecutionOptions`.
pub struct InputData {
input_component_size: u32,
input_options_size: u32,
}

impl Instance<Relayer> {
Expand All @@ -434,85 +434,44 @@ impl Instance<Relayer> {
block: ExecutionBlockWithSource<()>,
options: ExecutionOptions,
) -> ExecutorResult<Instance<InputData>> {
let encoded_block = postcard::to_allocvec(&block).map_err(|e| {
let input = InputType::V1 { block, options };
let encoded_input = postcard::to_allocvec(&input).map_err(|e| {
ExecutorError::Other(format!(
"Failed encoding of the block for `input` function: {}",
"Failed encoding of the input for `input` function: {}",
e
))
})?;
let encoded_block_size = u32::try_from(encoded_block.len()).map_err(|e| {
let encoded_input_size = u32::try_from(encoded_input.len()).map_err(|e| {
ExecutorError::Other(format!(
"The encoded block is more than `u32::MAX`. We support only wasm32: {}",
"The encoded input is more than `u32::MAX`. We support only wasm32: {}",
e
))
})?;

let input_component = self.input_component(encoded_block, encoded_block_size);
self.add_method("input_component", input_component)?;

let encoded_options = postcard::to_allocvec(&options).map_err(|e| {
ExecutorError::Other(format!(
"Failed encoding of the execution options for `input_options` function: {}",
e
))
})?;
let encoded_options_size = u32::try_from(encoded_options.len()).map_err(|e| {
ExecutorError::Other(format!(
"The encoded option is more than `u32::MAX`. We support only wasm32: {}",
e
))
})?;

let input = self.input_options(encoded_options, encoded_options_size);
self.add_method("input_options", input)?;
let input = self.input(encoded_input, encoded_input_size);
self.add_method("input", input)?;

Ok(Instance {
store: self.store,
linker: self.linker,
stage: InputData {
input_component_size: encoded_block_size,
input_options_size: encoded_options_size,
input_component_size: encoded_input_size,
},
})
}

fn input_component(
&mut self,
encoded_block: Vec<u8>,
encoded_block_size: u32,
) -> Func {
fn input(&mut self, encoded_input: Vec<u8>, encoded_input_size: u32) -> Func {
let closure = move |mut caller: Caller<'_, ExecutionState>,
out_ptr: u32,
out_len: u32|
-> anyhow::Result<()> {
if out_len != encoded_block_size {
if out_len != encoded_input_size {
return Err(anyhow::anyhow!(
"The provided buffer size is not equal to the encoded block size."
"The provided buffer size is not equal to the encoded input size."
));
}

caller.write(out_ptr, &encoded_block)
};

Func::wrap(&mut self.store, closure)
}

fn input_options(
&mut self,
encoded_options: Vec<u8>,
encoded_options_size: u32,
) -> Func {
let closure = move |mut caller: Caller<'_, ExecutionState>,
out_ptr: u32,
out_len: u32|
-> anyhow::Result<()> {
if out_len != encoded_options_size {
return Err(anyhow::anyhow!(
"The provided buffer size is not equal to the encoded options size."
));
}

caller.write(out_ptr, &encoded_options)
caller.write(out_ptr, &encoded_input)
};

Func::wrap(&mut self.store, closure)
Expand All @@ -521,10 +480,10 @@ impl Instance<Relayer> {

impl Instance<InputData> {
/// Runs the WASM instance from the compiled `Module`.
pub fn run(self, module: &Module) -> ReturnType {
pub fn run(self, module: &Module) -> ExecutorResult<ReturnType> {
self.internal_run(module).map_err(|e| {
ExecutorError::Other(format!("Error with WASM initialization: {}", e))
})?
})
}

fn internal_run(mut self, module: &Module) -> anyhow::Result<ReturnType> {
Expand All @@ -548,17 +507,11 @@ impl Instance<InputData> {
self.store.data_mut().memory = Some(memory);

let run = instance
.get_typed_func::<(u32, u32), u64>(&mut self.store, "execute")
.get_typed_func::<u32, u64>(&mut self.store, "execute")
.map_err(|e| {
anyhow::anyhow!("Failed to get the `execute` function: {}", e.to_string())
})?;
let result = run.call(
&mut self.store,
(
self.stage.input_component_size,
self.stage.input_options_size,
),
)?;
let result = run.call(&mut self.store, self.stage.input_component_size)?;

let (ptr, len) = unpack_ptr_and_len(result);
let (ptr, len) = (ptr as usize, len as usize);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ fuel-core-executor = { workspace = true, default-features = false }
fuel-core-storage = { workspace = true, default-features = false }
fuel-core-types = { workspace = true, default-features = false }
postcard = { workspace = true }
serde = { workspace = true }

[dev-dependencies]
proptest = { workspace = true }
Expand Down
34 changes: 5 additions & 29 deletions crates/services/upgradable-executor/wasm-executor/src/ext.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,7 @@ use crate::utils::{
InputType,
};
use core::marker::PhantomData;
use fuel_core_executor::{
executor::ExecutionOptions,
ports::MaybeCheckedTransaction,
};
use fuel_core_executor::ports::MaybeCheckedTransaction;
use fuel_core_types::{
blockchain::primitives::DaBlockHeight,
fuel_tx::Transaction,
Expand Down Expand Up @@ -82,11 +79,8 @@ mod host {
extern "C" {
// Initialization API

/// Returns the encoded component as an input to the executor.
pub(crate) fn input_component(output_ptr: Ptr32Mut<[u8]>, output_size: u32);

/// Returns the encoded execution options as an input to the executor.
pub(crate) fn input_options(output_ptr: Ptr32Mut<[u8]>, output_size: u32);
/// Returns the encoded input for the executor.
pub(crate) fn input(output_ptr: Ptr32Mut<[u8]>, output_size: u32);

// TxSource API

Expand Down Expand Up @@ -137,11 +131,11 @@ mod host {
pub struct ReturnResult(u16);

/// Gets the `InputType` by using the host function. The `size` is the size of the encoded input.
pub fn input_component(size: usize) -> anyhow::Result<InputType> {
pub fn input(size: usize) -> anyhow::Result<InputType> {
let mut encoded_block = vec![0u8; size];
let size = encoded_block.len();
unsafe {
host::input_component(
host::input(
Ptr32Mut::from_slice(encoded_block.as_mut_slice()),
u32::try_from(size).expect("We only support wasm32 target; qed"),
)
Expand All @@ -153,24 +147,6 @@ pub fn input_component(size: usize) -> anyhow::Result<InputType> {
Ok(input)
}

/// Gets the `ExecutionOptions` by using the host function. The `size` is the size of the encoded input.
pub fn input_options(size: usize) -> anyhow::Result<ExecutionOptions> {
let mut encoded_block = vec![0u8; size];
let size = encoded_block.len();
unsafe {
host::input_options(
Ptr32Mut::from_slice(encoded_block.as_mut_slice()),
u32::try_from(size).expect("We only support wasm32 target; qed"),
)
};

let input: ExecutionOptions = postcard::from_bytes(&encoded_block).map_err(|e| {
anyhow::anyhow!("Failed to decode the execution options: {:?}", e)
})?;

Ok(input)
}

/// Gets the next transactions by using the host function.
pub fn next_transactions(gas_limit: u64) -> anyhow::Result<Vec<MaybeCheckedTransaction>> {
let next_size = unsafe { host::peek_next_txs_size(gas_limit) };
Expand Down
76 changes: 43 additions & 33 deletions crates/services/upgradable-executor/wasm-executor/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,25 +13,28 @@
#![deny(unused_crate_dependencies)]
#![deny(warnings)]

use crate::{
relayer::WasmRelayer,
storage::WasmStorage,
tx_source::WasmTxSource,
utils::{
pack_ptr_and_len,
ReturnType,
},
};
use crate as fuel_core_wasm_executor;
use fuel_core_executor::executor::ExecutionInstance;
use fuel_core_storage::transactional::Changes;
use fuel_core_types::services::{
block_producer::Components,
executor::{
Error as ExecutorError,
ExecutionResult,
Result as ExecutorResult,
},
Uncommitted,
};
use fuel_core_wasm_executor as _;
use fuel_core_wasm_executor::{
relayer::WasmRelayer,
storage::WasmStorage,
tx_source::WasmTxSource,
utils::{
pack_ptr_and_len,
InputType,
ReturnType,
},
};

mod ext;
mod relayer;
Expand All @@ -40,44 +43,51 @@ mod tx_source;
pub mod utils;

#[no_mangle]
pub extern "C" fn execute(component_len: u32, options_len: u32) -> u64 {
let result = execute_without_commit(component_len, options_len);
let encoded = postcard::to_allocvec(&result).expect("Failed to encode the result");
pub extern "C" fn execute(input_len: u32) -> u64 {
let result = execute_without_commit(input_len);
let output = ReturnType::V1(result);
let encoded = postcard::to_allocvec(&output).expect("Failed to encode the output");
let static_slice = encoded.leak();
pack_ptr_and_len(
static_slice.as_ptr() as u32,
u32::try_from(static_slice.len()).expect("We only support wasm32 target; qed"),
)
}

pub fn execute_without_commit(component_len: u32, options_len: u32) -> ReturnType {
let block = ext::input_component(component_len as usize)
.map_err(|e| ExecutorError::Other(e.to_string()))?;
let options = ext::input_options(options_len as usize)
pub fn execute_without_commit(
input_len: u32,
) -> ExecutorResult<Uncommitted<ExecutionResult, Changes>> {
let input = ext::input(input_len as usize)
.map_err(|e| ExecutorError::Other(e.to_string()))?;

let (block, options) = match input {
InputType::V1 { block, options } => {
let block = block.map_p(|component| {
let Components {
header_to_produce,
gas_price,
coinbase_recipient,
..
} = component;

Components {
header_to_produce,
gas_price,
transactions_source: WasmTxSource::new(),
coinbase_recipient,
}
});

(block, options)
}
};

let instance = ExecutionInstance {
relayer: WasmRelayer {},
database: WasmStorage {},
options,
};

let block = block.map_p(|component| {
let Components {
header_to_produce,
gas_price,
coinbase_recipient,
..
} = component;

Components {
header_to_produce,
gas_price,
transactions_source: WasmTxSource::new(),
coinbase_recipient,
}
});

let (
ExecutionResult {
block,
Expand Down
22 changes: 19 additions & 3 deletions crates/services/upgradable-executor/wasm-executor/src/utils.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
use fuel_core_executor::executor::ExecutionBlockWithSource;
use fuel_core_executor::executor::{
ExecutionBlockWithSource,
ExecutionOptions,
};
use fuel_core_storage::transactional::Changes;
use fuel_core_types::services::{
executor::{
Expand Down Expand Up @@ -38,9 +41,22 @@ pub fn unpack_exists_size_result(val: u64) -> (bool, u32, u16) {
(exists, size, result)
}

pub type InputType = ExecutionBlockWithSource<()>;
/// The input type for the WASM executor. Enum allows handling different
/// versions of the input without introducing new host functions.
#[derive(Debug, serde::Serialize, serde::Deserialize)]
pub enum InputType {
V1 {
block: ExecutionBlockWithSource<()>,
options: ExecutionOptions,
},
}

pub type ReturnType = ExecutorResult<Uncommitted<ExecutionResult, Changes>>;
/// The return type for the WASM executor. Enum allows handling different
/// versions of the return without introducing new host functions.
#[derive(Debug, serde::Serialize, serde::Deserialize)]
pub enum ReturnType {
V1(ExecutorResult<Uncommitted<ExecutionResult, Changes>>),
}

#[cfg(test)]
mod tests {
Expand Down
Loading