Worker Thread using VM to run code fails

[d3x0r]

Version

23.5.0

Platform

Windows 11 Pro

Subsystem

No response

What steps will reproduce the bug?

I am working on using the VM module in worker_threads, and ran into an issue... I experimented a while to find a minimal case that would show me what I was missing... https://gist.github.com/d3x0r/c88e4cbadb37a3f4797ca40941ac3438 This is all of the approaches that I tried, and in the end I found

import { Script, constants, createContext, runInContext } from 'node:vm';
import vm from "vm";
import wt from "worker_threads";


console.log( "Starting worker 4" );
const w4 = new wt.Worker( 
				'const a = await import("node:fs").then(({readFile}) => readFile instanceof Function); \
		import {parentPort} from "worker_threads";   \
		import vm from "vm"; \
		const context = vm.createContext( {data:"yes"}, { importModuleDynamically: vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER } ); \
          \
		const b = vm.runInContext(  `import("node:fs").then(({readFile}) => readFile instanceof Function);`  \
				, context, { importModuleDynamically: vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER } );        \
          \
			parentPort.postMessage( {op:"log", f:["textScript 2", a, b] });'
			, {
				eval:true,
				execArgv : ['--input-type=module'],
				stderr:true,
				stdin:true,
				stdout:true,
			})
w4.on( "message", (msg)=>{
	console.log( "message:", msg );
} );

I tested all of the components of the above, the vm.runInContext works, the wt.Worker works, but using vm.runInContext in wt.Worker fails...

what I'm getting for an error in the application I'm developing this for I'm getting

WOrker started?
Error from thread: sandboxPrerun.js:17
import util from "util";
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at new Script (node:vm:117:7)
    at createScript (node:vm:269:10)
    at Object.runInContext (node:vm:300:10)
    at file:///M:/javascript/dekcore4.os/startup/[eval1]:110:5
    at ModuleJob.run (node:internal/modules/esm/module_job:272:25)
    at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:223:26)
    at async ModuleLoader.eval (node:internal/modules/esm/loader:219:20)
    at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:98:5)
uv loop at [000000EB105FF748] has open handles:
[000002C9F97725D0] async (active)
	Close callback: 0000000000000000 
	Data: 000002C9F97725C0 
	(First field): 000002C9F9710890 
[000002C9F9772D40] async (active)
	Close callback: 0000000000000000 
	Data: 000002C9F9772D30 
	(First field): 000002C9F97108D0 
uv loop at [000000EB105FF748] has 2 open handles in total

  #  go.2.bat >err 2>&1 - Far 3.0.6161.0 x64[60876]: void __cdecl node::CheckedUvLoopClose(struct uv_loop_s *) at c:\ws\src\debug_utils.cc:352
  #  Assertion failed: "Unreachable code reached" __VA_OPT__(": ") "uv_loop_close() while having open handles"

----- Native stack trace -----

 1: 00007FF7EC0064BD void __cdecl node::SetCppgcReference(class v8::Isolate * __ptr64,class v8::Local<class v8::Object>,void * __ptr64)+16957
 2: 00007FF7EBF6D061 EVP_MD_meth_get_input_blocksize+90625
 3: 00007FF7EC00607B void __cdecl node::SetCppgcReference(class v8::Isolate * __ptr64,class v8::Local<class v8::Object>,void * __ptr64)+15867
 4: 00007FF7EBE31CED RSA_meth_get_flags+210749
 5: 00007FF7EBE25ED5 RSA_meth_get_flags+162085
 6: 00007FF7EC0668B3 uv_poll_stop+291
 7: 00007FF7ED95FE4E inflateValidate+168766
 8: 00007FFBFDAA259D BaseThreadInitThunk+29
 9: 00007FFBFF44AF38 RtlUserThreadStart+40

which is a bad crash, what I get from the above example script is

message: { op: 'log', f: [ 'textScript 2', true ] }
node:internal/event_target:1101
  process.nextTick(() => { throw err; });
                           ^

Error [ERR_UNHANDLED_ERROR]: Unhandled error. ({})
    at Worker.emit (node:events:502:17)
    at [kOnErrorMessage] (node:internal/worker:326:10)
    at [kOnMessage] (node:internal/worker:337:37)
    at MessagePort.<anonymous> (node:internal/worker:232:57)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:827:20)
    at MessagePort.<anonymous> (node:internal/per_context/messageport:23:28) {
  code: 'ERR_UNHANDLED_ERROR',
  context: {}
}

Node.js v23.5.0

it's possible there's a syntax error in the example, but I'm not seeing it... the quotes should be nested so it's a simple translation...

How often does it reproduce? Is there a required condition?

Always with the example script. I was also testing with Script() instead of VM and it was also failing.

What is the expected behavior? Why is that the expected behavior?

vm and worker_thread can each use script code that can use import and is a module... but somehow vm IN worker_thread can't?

What do you see instead?

(described above)

Additional information

No response

[joyeecheung]

vm.runInContext() only support Scripts, not Modules (the two are different parse targets for V8). The right APIs to use for compiling and running ESM is vm.SourceTextModule though it's still behind --experimental-vm-modules.

[d3x0r]

good thing I'm not using SourceTextModule... I'm using Source() or RunInContext, and both work with import just fine. That is they work fine when not run in a worker thread. The test case tests that the vm.(whatever) that I'm using actually worked with import in the first place... and

const script = new Script(
  'import("node:fs").then(({readFile}) => readFile instanceof Function)',
  { importModuleDynamically: constants.USE_MAIN_CONTEXT_DEFAULT_LOADER }
);

const context = createContext( {} );
onst a = await script.runInThisContext();
const b = await script.runInContext( context, { importModuleDynamically: constants.USE_MAIN_CONTEXT_DEFAULT_LOADER } );

Both of those work just fine. (probalby don't need the option object on the runincontext

[joyeecheung]

What you see as "import" was dynamic import() that is available to the Scripts. The errors in the OP indicates that you are parsing static import statements which are not supported in Scripts and can only be parsed in a SourceTextModule.

[d3x0r]

I see; the example still didn't use import from and it was import() that was failing. I assumed it was the same issue, a few regexes later and the cod only uses import() just like the example code that fails, and what the Error is now is unexpected token 'import'

(await import( ...)) doesn't give an error about await, but instead the unexpected token import... if await is outside of the parens the there's an error about 'await only available in async function and top level modules'...

Yes, does seem I can stay away from import ... and just use import() but then, scripts that are imported in the vm get a globalThis that isn't the context object I supplied; but that's an entirely different issue than any of this.

Thanks for the feedback @joyeecheung