Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a PolkaVM-based executor #3458

Merged
merged 9 commits into from
Mar 12, 2024
Merged

Conversation

koute
Copy link
Contributor

@koute koute commented Feb 23, 2024

This PR adds a new PolkaVM-based executor to Substrate.

  • The executor can now be used to actually run a PolkaVM-based runtime, and successfully produces blocks.
  • The executor is always compiled-in, but is disabled by default.
  • The SUBSTRATE_ENABLE_POLKAVM environment variable must be set to 1 to enable the executor, in which case the node will accept both WASM and PolkaVM program blobs (otherwise it'll default to WASM-only). This is deliberately undocumented and not explicitly exposed anywhere (e.g. in the command line arguments, or in the API) to disincentivize anyone from enabling it in production. If/when we'll move this into production usage I'll remove the environment variable and do it "properly".
  • I did not use our legacy runtime allocator for the PolkaVM executor, so currently every allocation inside of the runtime will leak guest memory until that particular instance is destroyed. The idea here is that I will work on the Remove unnecessary host functions allocator usage polkadot-fellows/RFCs#4 which will remove the need for the legacy allocator under WASM, and that will also allow us to use a proper non-leaking allocator under PolkaVM.
  • I also did some minor cleanups of the WASM executor and deleted some dead code.

No prdocs included since this is not intended to be an end-user feature, but an unofficial experiment, and shouldn't affect any current production user. Once this is production-ready a full Polkadot Fellowship RFC will be necessary anyway.

@koute koute added the R0-silent Changes should not be mentioned in any release notes label Feb 23, 2024
@koute koute requested a review from a team February 23, 2024 12:27
@michalkucharczyk michalkucharczyk requested a review from a team March 8, 2024 13:48
Copy link
Member

@davxy davxy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I suppose that at some point there will be a follow up to rename WasmError type, WasmInstance trait, ...

@koute koute added this pull request to the merge queue Mar 12, 2024
Merged via the queue into paritytech:master with commit b0f34e4 Mar 12, 2024
129 of 130 checks passed
@koute koute deleted the master_polkavm_executor branch March 12, 2024 06:01
ordian added a commit that referenced this pull request Mar 16, 2024
* master: (65 commits)
  collator protocol changes for elastic scaling (validator side) (#3302)
  Contracts use polkavm workspace deps (#3715)
  Add elastic scaling support in ParaInherent BenchBuilder  (#3690)
  Removes `as [disambiguation_path]` from `derive_impl` usage (#3652)
  fix(paseo-spec): New Paseo Bootnodes (#3674)
  Improve Penpal runtime + emulated tests (#3543)
  Staking ledger bonding fixes (#3639)
  DescribeAllTerminal for HashedDescription (#3349)
  Increase timeout for assertions (#3680)
  Add subsystems regression tests to CI (#3527)
  Always print connectivity report (#3677)
  Revert "FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280)" (#3665)
  authority-discovery: Add log for debugging DHT authority records (#3668)
  Construct Runtime v2  (#1378)
  Support for `keyring` in runtimes (#2044)
  Add api-name in `cannot query the runtime API version` warning (#3653)
  Add a PolkaVM-based executor (#3458)
  Adds default config for assets pallet (#3637)
  Bump handlebars from 4.3.7 to 5.1.0 (#3248)
  [Collator Selection] Fix weight refund for `set_candidacy_bond` (#3643)
  ...
dharjeezy pushed a commit to dharjeezy/polkadot-sdk that referenced this pull request Mar 24, 2024
This PR adds a new PolkaVM-based executor to Substrate.

- The executor can now be used to actually run a PolkaVM-based runtime,
and successfully produces blocks.
- The executor is always compiled-in, but is disabled by default.
- The `SUBSTRATE_ENABLE_POLKAVM` environment variable must be set to `1`
to enable the executor, in which case the node will accept both WASM and
PolkaVM program blobs (otherwise it'll default to WASM-only). This is
deliberately undocumented and not explicitly exposed anywhere (e.g. in
the command line arguments, or in the API) to disincentivize anyone from
enabling it in production. If/when we'll move this into production usage
I'll remove the environment variable and do it "properly".
- I did not use our legacy runtime allocator for the PolkaVM executor,
so currently every allocation inside of the runtime will leak guest
memory until that particular instance is destroyed. The idea here is
that I will work on the polkadot-fellows/RFCs#4
which will remove the need for the legacy allocator under WASM, and that
will also allow us to use a proper non-leaking allocator under PolkaVM.
- I also did some minor cleanups of the WASM executor and deleted some
dead code.

No prdocs included since this is not intended to be an end-user feature,
but an unofficial experiment, and shouldn't affect any current
production user. Once this is production-ready a full Polkadot
Fellowship RFC will be necessary anyway.
bgallois pushed a commit to duniter/duniter-polkadot-sdk that referenced this pull request Mar 25, 2024
This PR adds a new PolkaVM-based executor to Substrate.

- The executor can now be used to actually run a PolkaVM-based runtime,
and successfully produces blocks.
- The executor is always compiled-in, but is disabled by default.
- The `SUBSTRATE_ENABLE_POLKAVM` environment variable must be set to `1`
to enable the executor, in which case the node will accept both WASM and
PolkaVM program blobs (otherwise it'll default to WASM-only). This is
deliberately undocumented and not explicitly exposed anywhere (e.g. in
the command line arguments, or in the API) to disincentivize anyone from
enabling it in production. If/when we'll move this into production usage
I'll remove the environment variable and do it "properly".
- I did not use our legacy runtime allocator for the PolkaVM executor,
so currently every allocation inside of the runtime will leak guest
memory until that particular instance is destroyed. The idea here is
that I will work on the polkadot-fellows/RFCs#4
which will remove the need for the legacy allocator under WASM, and that
will also allow us to use a proper non-leaking allocator under PolkaVM.
- I also did some minor cleanups of the WASM executor and deleted some
dead code.

No prdocs included since this is not intended to be an end-user feature,
but an unofficial experiment, and shouldn't affect any current
production user. Once this is production-ready a full Polkadot
Fellowship RFC will be necessary anyway.
@athei athei mentioned this pull request Apr 8, 2024
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
R0-silent Changes should not be mentioned in any release notes
Projects
Status: done
Development

Successfully merging this pull request may close these issues.

4 participants