Failing to load vscode-oniguruma wasm (and other files) since 1.80

[jhk-mjolner]

I've tried upgrading from monaco-languageclient 6.0.3 to the latest 6.4.6, and got several errors in the browser console related to not being allowed to load local resources.
I've tried different versions, and concluded that the error first appears in version 6.3.0 which uses 1.80 of monaco-vscode-api.

Error examples from browser console:
Not allowed to load local resource: file:///C:/<PATH_TO_MY_PROJECT>/node_modules/vscode/dist/service-override/vscode-oniguruma/release/onig.wasm
Not allowed to load local resource: file:///C:/<PATH_TO_MY_PROJECT>/frontend/node_modules/vscode/dist/default-extensions/theme-defaults/light_modern.json (there's several of this kind of error, about trying to load various theme .json files)
And those files rightly aren't there, so webpack can't bundle them and serve them as resources.
The error is thrown from node_modules\vscode\dist\vscode\vs\workbench\services\textMate\browser\textMateTokenizationFeatureImpl.js in async _loadVSCodeOnigurumaWASM() where it tries to load the path 'vscode-oniguruma/../onig.wasm'

I've tried looking at some source code and believe this change might be the culprit: e641ac3 as it was introduced in 1.80, remains in 1.81.7 (latest), and changes how the path to the wasm file is resolved.
Perhaps another similar change in another commit, also introduced in 1.80, might explain the same error for the theme .json file(s)?

I'm using Angular and an extra-webpack.config.js file to add webpack configuration and have this part for loading wasm, that worked fine before version 1.80:

      {
        test: /\.(mp3|wasm|ttf)$/i,
        type: 'asset/resource',
      },
      {
        test: /\.wasm$/,
        type: 'asset/inline',
      },

Is there some configuration I can use, either for monaco-languageclient, monaco-vscode-api or webpack, that can resolve this issue?
Or does it require code changes from your side?

[CGNonofr]

I see you are on windows and I've never tested it on that platform, maybe the C:/ part breaks it? (at least for the extension files part: light_modern.json for instance`)

Regarding the wasm loading issue, did you try to add an alias from vscode-oniguruma/release/onig.wasm to the absolute file path in the webpack config?

[jhk-mjolner]

I don't expect the C:/ part to be a problem, I think the problem is that the wasm is located at
C:\<PATH_TO_MY_PROEJCT>\node_modules\vscode-oniguruma\release\onig.wasm not C:/<PATH_TO_MY_PROJECT>/node_modules/vscode/dist/service-override/vscode-oniguruma/release/onig.wasm. It's the parent path relative within node_modules that has changed, I believe.
I don't know where the /vscode/dist/service-override/ comes from.
The path before the change, in the linked commit, was explicitly in node_modules. When that was removed and left to something else to resolve, I guess it gets resolved to node_modules/vscode/dist/service-override/ for some reason.

Regarding the webpack alias, I'm not sure I'm using the feature correctly, but none of these additions helped:

    alias: {
      // 'vscode-oniguruma/../onig.wasm': resolve(__dirname, './node_modules/vscode-oniguruma/release/onig.wasm'),
      // 'vscode-oniguruma/release/onig.wasm': resolve(__dirname, './node_modules/vscode-oniguruma/release/onig.wasm'),
      // 'vscode-oniguruma/../onig.wasm': './node_modules/vscode-oniguruma/release/onig.wasm',
      // 'vscode-oniguruma/release/onig.wasm': './node_modules/vscode-oniguruma/release/onig.wasm',
      // 'vscode-oniguruma/../onig.wasm': 'C:\\<PATH_TO_MY_PROJECT>\\node_modules\\vscode-oniguruma\\release\\onig.wasm',
      // 'vscode-oniguruma/release/onig.wasm': 'C:\\<PATH_TO_MY_PROJECT>\\node_modules\\vscode-oniguruma\\release\\onig.wasm',
      // 'C:\\<PATH_TO_MY_PROJECT>\\node_modules\\vscode-oniguruma\\release\\onig.wasm': 'vscode-oniguruma/../onig.wasm',
      // 'C:\\<PATH_TO_MY_PROJECT>\\node_modules\\vscode-oniguruma\\release\\onig.wasm': 'vscode-oniguruma/release/onig.wasm',
    },

[CGNonofr]

It seems that webpack fails to resolve vscode-oniguruma\release\onig.wasm and consider it as a relative path

I'm not sure how the new URL(..., import.meta.url) syntax is handled in webpack, that's the only absolute import using this syntax in the code

[jhk-mjolner]

Yeah, it seems like that's the issue.
I'm guessing the change can't be rolled back as it fixed something else?
I've tried looking at how webpack handles new URL(), but I'm not finding anything relevant for this case.

I don't know how much experience you have with webpack, but do you have any other ideas I could try?

[CGNonofr]

isn't is related to https://stackoverflow.com/questions/74038161/angular-cli-and-loading-assets-with-import-meta-url-causing-not-allowed-to-load ?

[jhk-mjolner]

Looking at the bundled code in my browser debugger source tab, I can see the line is transformed into

const _onigWasm = new URL('vscode-oniguruma/release/onig.wasm', "file:///C:/<PATH_TO_MY_PROJECT>/node_modules/vscode/dist/service-override/textmate.js").href;

So maybe the problem somehow lies with the import.meta.url pointing at the textmate.js file, and not node_modules.

[jhk-mjolner]

isn't is related to https://stackoverflow.com/questions/74038161/angular-cli-and-loading-assets-with-import-meta-url-causing-not-allowed-to-load ?

Yes!

Adding

module.exports = {
  module: {
    parser: {
      javascript: {
        url: true,
      },
    },
  },
};

and deleting the .angular folder as suggested in https://stackoverflow.com/a/75252098 seems to have resolved the issue for me.

[CGNonofr]

both issues?

[jhk-mjolner]

Yes, also the .json file loading issue. Thanks!

In the issue linked in the answer you linked, they also suggest migrating to esbuild which Angular experimentally supports now. Do you know whether using esbuild is more or less supported by monaco-languageclient and monaco-vscode-api?

[jhk-mjolner]

Taking another look at https://github.com/TypeFox/monaco-languageclient-ng-example/blob/main/custom-webpack.config.js I can also see it already includes these lines, but only mentions a 'ttf url loading issue' which I haven't had, so I haven't used those lines.
I'll suggest to them that they add that it's now necessary to load the wasm too.

[kaisalmen]

@jhk-mjolner sorry I am late to the discussion. I was only aware of the ttf issue being solved by this setting so far.
monaco-languageclient-ng-example needs to be updated. It always falls off the table. Do you want to update it to the latest version / create a PR? Do you have time for that? 😊

[CGNonofr]

I'm not sure. esbuild is used by vite in the demo of this project, we need a small hack to make it work in dev mode:

monaco-vscode-api/demo/vite.config.ts

Lines 62 to 90 in 40ba45c

	esbuildOptions: {
	plugins: [{
	name: 'import.meta.url',
	setup ({ onLoad }) {
	// Help vite that bundles/move files in dev mode without touching `import.meta.url` which breaks asset urls
	onLoad({ filter: /.*\.js/, namespace: 'file' }, async args => {
	const code = fs.readFileSync(args.path, 'utf8')
	const assetImportMetaUrlRE = /\bnew\s+URL\s*\(\s*('[^']+'|"[^"]+"|`[^`]+`)\s*,\s*import\.meta\.url\s*(?:,\s*)?\)/g
	let i = 0
	let newCode = ''
	for (let match = assetImportMetaUrlRE.exec(code); match != null; match = assetImportMetaUrlRE.exec(code)) {
	newCode += code.slice(i, match.index)
	const path = match[1].slice(1, -1)
	const resolved = await import.meta.resolve!(path, url.pathToFileURL(args.path))
	newCode += `new URL(${JSON.stringify(url.fileURLToPath(resolved))}, import.meta.url)`
	i = assetImportMetaUrlRE.lastIndex
	}
	newCode += code.slice(i)
	return { contents: newCode }
	})
	}
	}]
	}
	},

[jhk-mjolner]

I can also see that the change to new URL() syntax and the related issues are mentioned in the readme for this repo https://github.com/CodinGame/monaco-vscode-api#troubleshooting
Maybe it could be worth mentioning the issues webpack has with the syntax and how to fix it. Maybe link https://github.com/TypeFox/monaco-languageclient-ng-example/blob/main/custom-webpack.config.js as well.

[CGNonofr]

Is it related to webpack of to the angular cli though?

[jhk-mjolner]

Ah, right, it was mostly an Angular specific thing, and if you don't want to include troubleshooting for specific web component frameworks, I guess it shouldn't be added.

[CGNonofr]

It can still be added in a troubleshooting section, feel free to open an MR ;)

[jhk-mjolner]

I thought about what would make sense to write. I could write something only about using Angular and Webpack, but I guess more than parser.javascript.url: true is needed from the monaco-languageclient-ng-example webpack configuration. I just don't know how much.

In general, I could boil the guidance down to "look at the examples from monaco-languageclient". But I'm not sure how much of the configuration in those examples is needed for this project alone, or even how many people will be using this project without monaco-languageclient. And maybe being dependent on another project/repo for guidance is less than ideal?

[CGNonofr]

I guess it can be very pragmatic: if you encounter this specific error, here's something you can try (maybe with some hints on the reasons that can cause this: angular-cli)

[fumeboy]

i meet something wrong with the same file onig.wasm, I publish my vite project as a browser library which using @codingame/services , and have an example page to import my published lib, and i have the problem below:

the bad query of that long URL is to fetch the onig.wasm, and failed because URL is too long to got an 413 error.

[CGNonofr]

There is something wrong with your vite configuration, the library just includes a new URL('[...]/onig.wasm', import.meta.url). If it doesn't work, it's a matter of bundlers