[Bug]: the jest tests performance dropped badly from node 14 to 19

[adcorduneanu]

Version

29.4.2

Steps to reproduce

I have a suite of 2158 tests distributed in 396 files - actually, we have more that we decided to skip in the CI because of the actual issue.

Each test requires some imports, and some of them are done globally, via setupFiles and some are imported via the tests.

The total import size per test file is around 20MB.

We were using node 14 in our ecosystem, but because of some other factors, we decided to upgrade to the latest node version (19). Doing so we quickly noticed that our CI time for the jest tests more than doubled.

Those being said we start measuring our suite against multiple node versions. All tests were done multiple times on each node version with --no-cache enabled, in order to get the most accurate readings.

Node 14 - 60s
Node 16 - 90s
Node 18 - 120s
Node 19 - 160s

For exact versions of node here is the detailed list.

 nvm ls
    19.5.0
  * 18.14.0 (Currently using 64-bit executable)
    16.16.0
    14.20.0

The npm command used for running the tests

jest --watchAll=true --config=\"./_tests/jest-tests.config.js\"

jest-test.config.js

const config = {
	verbose: true,
	moduleDirectories: [
		"node_modules",
		"<rootDir>/Dist/js/",
		"<rootDir>/Dist/js/static/"
	],
	testEnvironment: "jsdom",
	testMatch: [
		"**/_Tests/**/*spec.js",
		"**/_Tests/**/*Test.js",
		"**/_Tests/**/*Tests.js"
	],
	transformIgnorePatterns: [
		"/node_modules/",
		"<rootDir>/Dist/js/",
		"<rootDir>/Dist/js/static/"
	],
	setupFiles: [
		"<rootDir>/_Tests/Setup/Common.globals.js", // this is used for main-level mocks, and addressing some global variables
		"<rootDir>/_Tests/Setup/Common.setup.js" // used for importing global stuffs like jquery, fake-indexeddb, some bundled content 
	],
	rootDir: "..",
	roots: [
		"<rootDir>/_Tests",
		"<rootDir>/Scripts"
	],
	testResultsProcessor: "jest-junit",
	watchman: false
};

module.exports = config; 

Another important fact is that the jsdom was locked to 16.4.0 as this affects all the test suites.

Expected behavior

Performance to not decrease like that over upgrading the node version.

Actual behavior

The performance between node 19 and node 14 is almost 3 times worse.

Additional context

As this comes from a real project we can't offer additional context at the given time.

Environment

System:
    OS: Windows 10 10.0.19044
    CPU: (12) x64 Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
  Binaries:
    Node: 18.14.0 - C:\Program Files\nodejs\node.EXE
    npm: 9.3.1 - C:\Program Files\nodejs\npm.CMD
  npmPackages:
    jest: 29.4.2 => 29.4.2

[mike-dvorscak-enablon]

I'm currently in the process of upgrading a project to node 18 and jest 29 and noticed a similar drop in performance on newer node versions.

Our specs:
Test Files: 495
Tests: 4135

jest config:

module.exports = {
    testEnvironment: "jsdom",
    testEnvironmentOptions: {
        url: "http://localhost" 
    },
    setupFiles: ["core-js", "<rootDir>/tests/setup"],
    setupFilesAfterEnv: [
        "<rootDir>/tests/matchers.ts",
        "<rootDir>/tests/jsdomPatch.js",
        "<rootDir>/enzyme.config.js"
    ],
    testMatch: ["**/*.spec.(js|ts|jsx|tsx)"],
    moduleFileExtensions: ["js", "json", "ts", "jsx", "tsx"],
    moduleNameMapper: require("./aliases.config").jest,
    coverageDirectory: "<rootDir>/coverage",
    coverageReporters: ["html", "json", "cobertura"],
    reporters: [
        "default"
    ],
    transformIgnorePatterns: [
        "node_modules/(?!(react-grid-layout/css|react-resizable/css|lodash-es|nanoid)/)"
    ],
    testResultsProcessor: "jest-sonar-reporter"
};

Environment
System:
OS: Windows 10 10.0.19044
CPU: (8) x64 Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
Binaries:
Node: 14.18.1 - C:\Program Files\nodejs\node.EXE
Yarn: 1.22.17 - C:\Program Files\nodejs\yarn.CMD
npm: 6.14.8 - ~\AppData\Roaming\npm\npm.CMD
npmPackages:
jest: 29.4.3 => 29.4.3

Test Performance
Note: only node version was changed when collecting these run times. Cold runs always started with jest --clearCache running prior to the tests themselves

node 14.18.1
cold run: 239s
hot run: 196s

node 18.14.2
cold run: 276s
hot run: 240s

[mike-dvorscak-enablon]

Not sure if this is relevant, but the issue may be coming from node itself: nodejs/node#45662

[muhammadbilal87]

We are also facing the same issue while updating from node 12 to node 19

[BrevAlessio]

Same issue, updated node 12 -> 18 and Jest 26 -> 27. We have about 1000 test suites, time of testing on Jenkins CI went from 10 minutes to 23 minutes.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[mike-dvorscak-enablon]

I recently got a new dev machine and wanted to re-run jest + webpack on versions 14/16/18 of nodejs.

new machine specs:
System:
OS: Windows 10 10.0.19044
CPU: (16) x64 11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz
Binaries:
Node: 14.18.1 - C:\Program Files\nodejs\node.EXE
Yarn: 1.22.19 - C:\Program Files (x86)\Yarn\bin\yarn.CMD
npm: 6.14.15 - C:\Program Files\nodejs\npm.CMD

Below are the updated results:

command	mode	14.18.1	16.20.0	18.16.0
unit (jest 29.4.3)	cold	72.0s	128.66	103.5s
unit (jest 29.4.3)	hot	72.28s	90.71s	91.96s
build (webpack 5.70)	cold	64.05s	88.28s	54.66s
build (webpack 5.70)	hot	38.4s	46.16s	31.17s

Note: none of the source code or npm packages were changed between runs. On cold runs, I did delete cached webpack artefacts.

From node 14 to 18, our jest tests ran 27% slower while our webpack build was 18% faster. This leads me to believe that the issue is not solely with node itself but some interaction between node + jest.

Let me know if there is any other information that you need.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[adcorduneanu]

So, jest has a huge community, but nobody is interested in performance?!?

[lukeapage]

duplicate of #11956 I expect

[uladzimirdev]

jest 29.5.1, node 16.10 -> 18.16

at github actions runner with maxWorkers=2

16.10	18.16
21min	27min

[dmansomk]

Hi, same here. Any solution?

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[ottodranik]

Same thing. A huge drop of performance. I've used Angular 12 with jest 26 and CI and node 16 and tests run 5 minutes. Now I'm using Angular 15 with jest 29 and Node 18 and I'm getting 10-11 minutes with the same pack of tests.

[MorenoMdz]

From Node 16.4.2 to 16.14.0 or any newer version makes our current suit go from 20~ to 50+ seconds to run.

[MorenoMdz]

jest 29.5.1, node 16.10 -> 18.16

at github actions runner with maxWorkers=2

16.10 18.16
21min 27min

Ours went from 3min to 9min with 5 shards : /

Node 16

Node 20

Tested with node 18, same thing. Same workflow on both.

[MajorScruffy]

Same here. Upgraded from Node14 to Node18 and unit tests now take >250s when they used to take just 70s

[klaurtar]

Has anyone resolved this issue? I have 350~ passing tests in node 14 and switching to node 16 or 18 causes my tests to take forever and failures start happening

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[lukeapage]

This is a duplicate of #11956 follow that incident for updates - tldr light is at the end of the tunnel - there is a pr that fixed node js

[SimenB]

Most of the comments in this issue is indeed a duplicate of #11956. But this from the OP doesn't seem like the same thing:

Node 14 - 60s
Node 16 - 90s
Node 18 - 120s
Node 19 - 160s

[mahdibx]

I have a similar issue, migrating from node 18 to 20, tests are 2x slower, we have a lot of tests in our app with jsdom, I also tried jest 30.0.0 alpha1 in order to use jsdom 22 but doesn't seem to improve the situation. here are some benchmarks:

Node 18: 587s
Node 20: 1182s

On the CI:
Node 18 (80% workers): 6min5s - 7min46s
Node 20 (80% workers): 24min (and some tests failing occasionally with timeout default [5000ms])
Node 20 (60% workers): 11min40s

[SimenB]

Do you have a reproduction for this somehow?

And can you try with Node 21, that should have some good improvements (but not related to regression between 18 and 20)

[mahdibx]

Thank you @SimenB ! Node 21 works much better:

node21 80% workers: 436s
node21 100% workers: 349s
(testing done with jest 30 alpha1 and jsdom 22.1.0)

So even better than node 18, and it's scaling much better with the number of workers, which is great news!

For now we'll postpone the migration I think, as we want to stay on LTS versions only on our CI environments.
EDIT: If you have hints on what's different between v20 and v21 and if these improvement will come to v20 later, I'll be interested, otherwise I'll just wait for the next LTS. I found your answer to this in the memory leak issue, thanks again.

[kevin-donovan-zocdoc]

Just want to note that the fix from Node 21 made it into Node 20.10.0. My unit test runtime dropped by 50% with this fix.

[mike-dvorscak-enablon]

Just want to note that the fix from Node 21 made it into Node 20.10.0. My unit test runtime dropped by 50% with this fix.

I also wanted to note that this node version has restored performance back to the v14 levels, the updated benchmarks for my project:

node: 14.21.3
cold run: 116s
hot run: 103s

node: 20.10.0
cold run: 113s
hot run: 103s

[uladzimirdev]

18.16.1	20.10.0
230s	140s

🎉