Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wasmtime: add config option for parallel compilation #1903

Merged
merged 1 commit into from
Jul 6, 2020

Conversation

ueno
Copy link
Contributor

@ueno ueno commented Jun 19, 2020

When running in embedded environments, threads creation is sometimes undesirable. This adds a means to disable wasmtime's internal thread creation for parallel compilation and caches, by reusing the cache configuration as the indication (i.e., if cache is disabled, parallel compilation is also disabled).

@bjorn3
Copy link
Contributor

bjorn3 commented Jun 19, 2020

I think it should be a separate option. During development a cache doesn't help much, as the wasm file changes continuously, but parallel compilation does help.

@tschneidereit
Copy link
Member

I agree with @bjorn3, this should ideally not be tied to whether caching is enabled.

Looking at the code a bit, it's not exactly straight-forward to pipe through configuration for this, unfortunately. I wonder if it'd make sense to introduce a higher-level config struct, which would include the Strategy, and other configuration such as for caching and parallelization.

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this could be configured via the Tunables structure? I agree with others that this should be a separate configuration option from caching, although if threading is disabled it should probably also disable caching.

crates/environ/src/cranelift.rs Show resolved Hide resolved
@ueno ueno changed the title wasmtime: Enable parallel compilation only when cache is enabled wasmtime: add config option for parallel compilation Jun 24, 2020
@ueno
Copy link
Contributor Author

ueno commented Jun 24, 2020

Thank you for the suggestions; I've added a separate config option in Tunables.

@github-actions github-actions bot added the wasmtime:api Related to the API of the `wasmtime` crate itself label Jun 24, 2020
@github-actions
Copy link

Subscribe to Label Action

cc @peterhuene

This issue or pull request has been labeled: "wasmtime:api"

Thus the following users have been cc'd because of the following labels:

  • peterhuene: wasmtime:api

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

@alexcrichton
Copy link
Member

This is looks pretty good to me, thanks!

Before merging though I wanted to confirm something. Does this work being a runtime configuration option? Is the purpose here to control memory usage? It seems like systems likely either support threads or they don't, and for systems that don't support threads it may make more sense for this to be a build-time configuration option. If the purpose is something like controlling memory usage though then that makes sense because it's specific to the runtime in question.

@ueno
Copy link
Contributor Author

ueno commented Jun 25, 2020

The primary motivation of this is to run wasmtime on an environment which doesn't (yet) support threads, so disabling it at build-time would indeed make sense. I'll try to turn the option into a build-time configuration.

@alexcrichton
Copy link
Member

Ok makes sense! To be clear though I'm not saying this must change, I mostly just wanted to confirm if it worked for you. While a runtime configuration is possible I'm not sure if the target in question doesn't have std::thread at all or if it just returns errors. It's probably easiest to make this a runtime configuration option anyway.

@alexcrichton alexcrichton changed the base branch from master to main June 25, 2020 18:48
@npmccallum
Copy link
Member

@alexcrichton @ueno I agree this probably makes the most sense as a compile-time feature flag.

@ueno
Copy link
Contributor Author

ueno commented Jul 2, 2020

Sorry for the delay; I've updated the PR to use feature flag.

When running in embedded environments, threads creation is sometimes
undesirable. This adds a feature to toggle wasmtime's internal thread
creation for parallel compilation.
Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@github-actions github-actions bot added the wasi Issues pertaining to WASI label Jul 6, 2020
@github-actions
Copy link

github-actions bot commented Jul 6, 2020

Subscribe to Label Action

cc @kubkon

This issue or pull request has been labeled: "wasi"

Thus the following users have been cc'd because of the following labels:

  • kubkon: wasi

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

@alexcrichton alexcrichton merged commit 2ce2dd0 into bytecodealliance:main Jul 6, 2020
alexcrichton added a commit to alexcrichton/wasmtime that referenced this pull request Jun 14, 2022
When parallel compilation was moved behind a compile-time feature in the
`wasmtime` crate we forgot to add the corresponding feature to the C API
which means that the C API hasn't been using parallel compilation since bytecodealliance#1903
(oh dear!)
alexcrichton added a commit that referenced this pull request Jun 14, 2022
When parallel compilation was moved behind a compile-time feature in the
`wasmtime` crate we forgot to add the corresponding feature to the C API
which means that the C API hasn't been using parallel compilation since #1903
(oh dear!)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wasi Issues pertaining to WASI wasmtime:api Related to the API of the `wasmtime` crate itself
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants