[WebGPU] Fall back to 256MB for maxBufferSize if needed #17150
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Tested end-to-end on WebLLM with iPhone iOS 18 Beta Safari. |
tqchen
approved these changes
Jul 10, 2024
MasterJH5574
approved these changes
Jul 11, 2024
CharlieFRuan
added a commit
to mlc-ai/web-llm
that referenced
this pull request
Jul 11, 2024
Exactly the same as 0.2.46 except we update TVMjs to apache/tvm@32e9a48. The main change is that, when requesting a WebGPU device, we may fall back `maxBufferSize` to the default 256MB when 1GB is too much for the device. This is motivated by iOS support on Safari, tested on iOS 18 Developer Beta. Fore more, see: - apache/tvm#17150 ### Model WASM Version No change, still v0.2.43. ### TVMjs Updated, compiled at apache/tvm@32e9a48
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
By default, we request 1GB of
maxStorageBufferBindingSizeandmaxBufferSizewhen detecting a WebGPU device. However, low-resource devices such as iOS and Android may not be able to support 1GB.A previous PR falls back
maxStorageBufferBindingSizeto 128MB, the default values stated in WGSL doc, motivated by Android Chrome. This PR falls backmaxBufferSizeto 256MB, the default value, motivated by iOS Safari.