feat!: support running tests using VM context

[sheremet-va]

aka bring all Jest (V8) bugs into Vitest

The idea is to have a separate pool that will spawn workers with tinypool, but they will not be isolated. Isolation is introduced by vm.createContext inside the worker. For this, we need to implement our own import handling. I think we should keep our __vitest_ssr_import__ because it is the only way to support both ESM/CJS in the same file in wrongly bundled dependencies.

This is an optional pool! You need to opt-in to use it:

• via CLI argument: --experimental-vm-threads (this will throw an error, if you use --threads or --browser at the same time)

• via config option:

  export default {
    test: {
      experimentalVmThreads: true,
    }
  }

• run only a subset of files using this pool with poolMatchGlob

  export default {
    test: {
      threads: true,
      // all test files will be running like they did before,
      // except for the tests in "fast-leaking-tests" directory 
      poolMatchGlob: [
        ['**/fast-leaking-tests/*.spec.ts', 'experimentalVmThreads'],
      ]
    }
  }

You also cannot disable isolation when using experimentalVm, because this doesn't make sense with this pool. If you don't need isolation, use threads or child_process directly.

vm has a notable memory leak problem, so we are also introducing experimentalVmWorkerMemoryLimit option (works like Jest's workerIdleMemoryLimit to destroy the worker before it reaches this point. It is set to max_heap/number_of_workers by default. You might have better performance if you tweak this value manually.

--experimental-vm-child-process (experimentalVmChildProcess) will be implemented in a separate PR.

Pros:

• Tests go brrrrr (fast)

Cons:

• Vm is not very good with ESM so this introduces a ton of complexity
• We will have this fan-favorite bug: Jest globals differ from Node globals jestjs/jest#2549
• And this one: Memory Leak on ridiculously simple repo jestjs/jest#7874
• Also this bug: Improve performance of accessing globals in scripts running in vm modules nodejs/node#31658
• Basically, all of these issues are now also our issues: Roadmap for stabilization of vm modules nodejs/node#37648

---

Breaking changes:

• the environment is now loaded via the native Node.js mechanism, it should be transformed before the file is passed down to Vitest
• all environments always require transformMode (web or ssr). If transform mode is not specified, it will throw an error
• "VitestExecutor" is moved from "vitest/node" to "vitest/execute" entry point

[stackblitz]

Run & review this pull request in StackBlitz Codeflow.

[ghiscoding]

will that help with --no-threads (child process) which is extremely slow on Windows? or is that unrelated?

[sheremet-va]

will that help with --no-threads (child process) which is extremely slow on Windows? or is that unrelated?

If you need speed, don't use no-threads

Current version of this PR uses only workers to run tests in parallel. Will probably add a way to run child_process also.

[ghiscoding]

If you need speed, don't use no-threads

Yeah I wish but in Lerna-Lite (a fork of Lerna), it uses a ton of process.chdir(cwd) and child process and it seems that --no-threads is the only way to go about it (as per #1436)

Current version of this PR uses only workers to run tests in parallel. Will probably add a way to run child_process also.

that would be really nice and I wonder if that would help with my opened issue #3129 that I still have and found no fix yet (apart from not running the test one at a time). I did migrate from Jest to Vitest regardless

[silverwind]

With this third runner environment option, I wonder if the config should be refactored to replace threads with something like runner: 'worker_threads' | 'child_process' | 'vm' as I it seems they are mutually exclusive, or can they be combined?

[sheremet-va]

With this third runner environment option, I wonder if the config should be refactored to replace threads with something like runner: 'worker_threads' | 'child_process' | 'vm' as I it seems they are mutually exclusive, or can they be combined?

Yes, I've been thinking about a similar approach. But technically it should also contain "browser". The list might be:

• threads
• child_process
• browser
• experimentalVmThreads
• experimentalVmChildProcess

We decided against it for now to keep the same API. The --experimental-vm-threads --threads will throw an error.

[silverwind]

What's a browser environment? Some integration with playwright? 😆

[sheremet-va]

What's a browser environment? Some integration with playwright? 😆

https://vitest.dev/guide/browser.html

[silverwind]

Thanks, that does sound awesome indeed to be finally able to stop fighting jsdom 👍

[sheremet-va]

I think we also need to implement child_process pool to reuse it for vm-child-process and child_process pools 🤔

[netlify]

✅ Deploy Preview for fastidious-cascaron-4ded94 canceled.

Name	Link
🔨 Latest commit	82ae1a6
🔍 Latest deploy log	https://app.netlify.com/sites/fastidious-cascaron-4ded94/deploys/64c7ee70227853000826f971