-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve robustness of cache loading/storing #974
Merged
alexcrichton
merged 9 commits into
bytecodealliance:master
from
alexcrichton:cache-opt-level
Feb 26, 2020
Merged
Improve robustness of cache loading/storing #974
alexcrichton
merged 9 commits into
bytecodealliance:master
from
alexcrichton:cache-opt-level
Feb 26, 2020
Commits on Feb 24, 2020
-
Improve robustness of cache loading/storing
Today wasmtime incorrectly loads compiled compiled modules from the global cache when toggling settings such as optimizations. For example if you execute `wasmtime foo.wasm` that will cache globally an unoptimized version of the wasm module. If you then execute `wasmtime -O foo.wasm` it would then reload the unoptimized version from cache, not realizing the compilation settings were different, and use that instead. This can lead to very surprising behavior naturally! This commit updates how the cache is managed in an attempt to make it much more robust against these sorts of issues. This takes a leaf out of rustc's playbook and models the cache with a function that looks like: fn load<T: Hash>( &self, data: T, compute: fn(T) -> CacheEntry, ) -> CacheEntry; The goal here is that it guarantees that all the `data` necessary to `compute` the result of the cache entry is hashable and stored into the hash key entry. This was previously open-coded and manually managed where items were hashed explicitly, but this construction guarantees that everything reasonable `compute` could use to compile the module is stored in `data`, which is itself hashable. This refactoring then resulted in a few workarounds and a few fixes, including the original issue: * The `Module` type was split into `Module` and `ModuleLocal` where only the latter is hashed. The previous hash function for a `Module` left out items like the `start_func` and didn't hash items like the imports of the module. Omitting the `start_func` was fine since compilation didn't actually use it, but omitting imports seemed uncomfortable because while compilation didn't use the import values it did use the *number* of imports, which seems like it should then be put into the cache key. The `ModuleLocal` type now derives `Hash` to guarantee that all of its contents affect the hash key. * The `ModuleTranslationState` from `cranelift-wasm` doesn't implement `Hash` which means that we have a manual wrapper to work around that. This will be fixed with an upstream implementation, since this state affects the generated wasm code. Currently this is just a map of signatures, which is present in `Module` anyway, so we should be good for the time being. * Hashing `dyn TargetIsa` was also added, where previously it was not fully hashed. Previously only the target name was used as part of the cache key, but crucially the flags of compilation were omitted (for example the optimization flags). Unfortunately the trait object itself is not hashable so we still have to manually write a wrapper to hash it, but we likely want to add upstream some utilities to hash isa objects into cranelift itself. For now though we can continue to add hashed fields as necessary. Overall the goal here was to use the compiler to expose what we're not hashing, and then make sure we organize data and write the right code to ensure everything is hashed, and nothing more.
Configuration menu - View commit details
-
Copy full SHA for e2e3a3d - Browse repository at this point
Copy the full SHA e2e3a3dView commit details -
Update crates/environ/src/module.rs
Co-Authored-By: Peter Huene <peterhuene@protonmail.com>
Configuration menu - View commit details
-
Copy full SHA for 502c36a - Browse repository at this point
Copy the full SHA 502c36aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 29da88d - Browse repository at this point
Copy the full SHA 29da88dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 08c1220 - Browse repository at this point
Copy the full SHA 08c1220View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2b53fee - Browse repository at this point
Copy the full SHA 2b53feeView commit details
Commits on Feb 25, 2020
-
Revert "Update the expected structure of the cache"
This reverts commit 2b53fee.
Configuration menu - View commit details
-
Copy full SHA for c544f20 - Browse repository at this point
Copy the full SHA c544f20View commit details -
Configuration menu - View commit details
-
Copy full SHA for f84efc2 - Browse repository at this point
Copy the full SHA f84efc2View commit details
Commits on Feb 26, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 56e3897 - Browse repository at this point
Copy the full SHA 56e3897View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5e2264b - Browse repository at this point
Copy the full SHA 5e2264bView commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.