Getting Local models/wasm working with Create React App

[stinoga]

Question

I realize there's been a lot of talk about this in other issues, but I'm trying to gather if getting local-only model and wasm files will work with Create React App. I'm using WhisperForConditionalGeneration from @huggingface/transformers version 3.0.0-alpha.9.

My setup:

env.allowRemoteModels = false;
env.allowLocalModels = true;
env.backends.onnx.wasm.wasmPaths = process.env.PUBLIC_URL + "/dictation/";
env.localModelPath = process.env.PUBLIC_URL + "/dictation/models/";

... and in my {packagename}/public/models folder I've got:

ort-wasm-simd-threaded.jsep.wasm
models/config.json
models/generation_config.json
models/preprocessor_config.json
models/tokenizer_config.json
models/tokenizer.json
models/onnx/decoder_model_merged_q4.onnx
models/onnx/encoder_model.onnx

This returns the SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON error that has been discussed in other issues. If I set env.allowRemoteModels = true; and
env.allowLocalModels = false;, and clear my application cache, this works fine. My questions on that:

1. How can I get the wasm file to load locally only? It caches fine and calls locally (
   http://localhost:3000/dictation/ort-wasm-simd-threaded.jsep.wasm) after the initial CDN call, but I don't want to rely on an external CDN.
2. How can I get the model files to only call locally? (we will need to further train our own models). I have yet to get this working, but I assume the above error is to blame.
3. The main question: is this a limitation with CRA? I noticed that if I load the wasm file from the CDN first, it caches fine locally. It's just that initial call to the wasm local file (if not cached from the CDN) that fails, which people have said may be a CRA issue.

Thanks! Sorry for the long-winded question. Happy to provide any more code if needed.

[stinoga]

Found the solve for this! Including the fix in case anyone else runs into it. Needed to match the directory structure to the model_id value:

this.model_id = "onnx-community/whisper-base";

models/onnx-community/whisper-base/config.json
models/onnx-community/whisper-base/generation_config.json
models/onnx-community/whisper-base/preprocessor_config.json
models/onnx-community/whisper-base/tokenizer_config.json
models/onnx-community/whisper-base/tokenizer.json
models/onnx-community/whisper-base/onnx/decoder_model_merged_q4.onnx
models/onnx-community/whisper-base/onnx/encoder_model.onnx