perf issues with esbuild

[stafyniaksacha]

Describe the bug

I found that the vite:esbuild plugin is taking very long time when project has a lot of directories / content (I mean it can take minutes, tested on WSL2 & Win)

vite/packages/vite/src/node/plugins/esbuild.ts

Lines 235 to 237 in 7be0ba5

	async configResolved(config) {
	await initTSConfck(config)
	},

Can we not await it on resolveConfig hook ?
Why do we need to retrieve all tsconfig on the project?

Reproduction

https://stackblitz.com/edit/vitejs-vite-pruyxz?terminal=dev

Steps to reproduce

It's hard to reproduce, it needs a lot of directory / content.
I can share a private repository to reproduce or adapt https://github.com/stafyniaksacha/nuxt-compos-bench

System Info

System:
    OS: Linux 5.15 Ubuntu 22.04.2 LTS 22.04.2 LTS (Jammy Jellyfish)
    CPU: (16) x64 AMD Ryzen 9 5900HS with Radeon Graphics
    Memory: 6.31 GB / 15.33 GB
    Container: Yes
    Shell: 5.8.1 - /usr/bin/zsh
  Binaries:
    Node: 18.15.0 - ~/.nvm/versions/node/v18.15.0/bin/node
    Yarn: 1.22.19 - /mnt/c/Program Files/nodejs/yarn
    npm: 9.5.0 - ~/.nvm/versions/node/v18.15.0/bin/npm
  Browsers:
    Chrome: 110.0.5481.177
  npmPackages:
    vite: ^4.2.0 => 4.2.1

hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   46682 MB in  2.00 seconds = 23388.97 MB/sec
 Timing buffered disk reads: 362 MB in  0.55 seconds = 663.80 MB/sec

Used Package Manager

pnpm

Logs

No response

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to vuejs/core instead.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[stackblitz]

Start a new pull request in StackBlitz Codeflow.

• vitejs-vite-pruyxz (Open reproduction)

[dominikg]

Are you sure that it is actually tsconfck that is blocking?

you can run DEBUG=vite:esbuild vite dev and it is going to log how long it takes vite to initialize tsconfck cache.
Or you can run npx tsconfck@latest find-all . and tsconfck is going to print all tsconfig.json files in that directory.

For reference, tsconfck find-all on my /develop directory returned a list of 2.2k tsconfig.json files after 49s. out of 1.8 million files and directories.

That being said, find -type f -name tsconfig.json . only takes 3s for the same result, so there definetely is room for optimization here.

It would be great if you could share stats about the size ( number of directories, files, nesting depth) in your project.

[stafyniaksacha]

I got different result between running via CLI and in my app (which use nuxt 3) but the size is not that huge from your tests

> find . -type f -not -path '.*node_modules/*' | wc -l
7927

Adding console.time inside my node module like:

async function findAll(dir, options) {
  const files = [];
  console.time('findAll')
  for await (const tsconfigFile of findTSConfig(require$$0$4.resolve(dir), options)) {
    files.push(tsconfigFile);
  }
  console.timeEnd('findAll')
  return files;
}

gives: findAll: 1:03.093 (m:ss.mmm)

But using CLI (without having anything in the stack) has much better results:

$ time npx tsconfck@latest find-all .
npx tsconfck@latest find-all .  5.57s user 1.51s system 102% cpu 6.898 total

$ time find . -type f -name tsconfig.json
find . -type f -name tsconfig.json  0.03s user 0.06s system 99% cpu 0.096 total

I've found this stackoverflow topic that speak about libuv thread pool that might be stuck waiting for other read: https://stackoverflow.com/a/59594320/5303877

-- I'll try to post more info when I found some

[stafyniaksacha]

Awesome @dominikg thanks!

Also, is it necessary to block configResolved hook while initTSConfck hasn't finished?

[dominikg]

That is is going to be updated in a different PR. We have to block on it before first transpile though so i'm not sure how much that is going to help.