CommonJS dependencies crash SSR server

[rtsao]

Describe the bug

CommonJS dependencies encountered during SSR can crash the server (e.g. ReferenceError: module is not defined or ReferenceError: require is not defined, depending on which is first) if workspaces are used in some capacity.

I have identified two specific scenarios where this will occur:

1. The CJS dependency is a workspace (i.e. app → cjs-workspace)
2. The CJS dependency is a sub-dependency of a pnpm or Yarn PnP workspace dependency (i.e. app → other-workspace → cjs-npm-dependency)

While possibly confusing, the first scenario is arguably intentional behavior because workspaces are typically implemented with symlinks and this logic indicates that symlinked packages should never be externalized:

vite/packages/vite/src/node/plugins/resolve.ts

Lines 656 to 659 in c6870f3

	// don't external symlink packages
	if (!allowLinkedExternal && !resolved.id.includes('node_modules')) {
	return resolved
	}

However, the second scenario is almost certainly a bug.

Reproduction

https://github.com/rtsao/vite-bug-repro-cjs-workspaces (for scenario 1)
https://github.com/rtsao/vite-bug-repro-ssr-pnp and https://github.com/rtsao/vite-bug-repro-ssr-pnpm (for scenario 2)

System Info

System:
    OS: macOS 12.5
    CPU: (10) arm64 Apple M1 Max
    Memory: 36.02 GB / 64.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 16.16.0 - ~/Library/Caches/fnm_multishells/1157_1660588473591/bin/node
    Yarn: 3.2.2 - ~/Library/Caches/fnm_multishells/1157_1660588473591/bin/yarn
    npm: 8.11.0 - ~/Library/Caches/fnm_multishells/1157_1660588473591/bin/npm
  Browsers:
    Chrome: 104.0.5112.101
    Safari: 15.6

Used Package Manager

yarn

Logs

Click to expand!

vite-bug-repro-ssr-pnp main
❯ corepack enable
yarn install
yarn node app/server.mjs
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0013: │ vite@npm:3.0.8 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ which@npm:2.0.2 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ wide-align@npm:1.1.5 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ wrappy@npm:1.0.2 can't be found in the cache and will be fetched from the remote registry
➤ YN0013: │ yallist@npm:4.0.0 can't be found in the cache and will be fetched from the remote registry
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
➤ YN0007: │ esbuild@npm:0.14.54 must be built because it never has been before or the last one failed
➤ YN0000: └ Completed in 0s 616ms
➤ YN0000: Done with warnings in 0s 788ms
http://localhost:5173
ReferenceError: module is not defined
    at /@fs/Users/rtsao/code/vite-bug-repro-ssr-pnp/.yarn/cache/locale-npm-0.1.0-6c7fdc5891-42af4432bc.zip/node_modules/locale/lib/index.js:159:3
    at /@fs/Users/rtsao/code/vite-bug-repro-ssr-pnp/.yarn/cache/locale-npm-0.1.0-6c7fdc5891-42af4432bc.zip/node_modules/locale/lib/index.js:161:4
    at instantiateModule (file:///Users/rtsao/code/vite-bug-repro-ssr-pnp/.yarn/__virtual__/vite-virtual-f00fbd239e/0/cache/vite-npm-3.0.8-cf480b5a77-ec3f57d52f.zip/node_modules/vite/dist/node/chunks/dep-74663fff.js:50512:15)
ReferenceError: module is not defined
    at /@fs/Users/rtsao/code/vite-bug-repro-ssr-pnp/.yarn/cache/locale-npm-0.1.0-6c7fdc5891-42af4432bc.zip/node_modules/locale/lib/index.js:157:3
    at /@fs/Users/rtsao/code/vite-bug-repro-ssr-pnp/.yarn/cache/locale-npm-0.1.0-6c7fdc5891-42af4432bc.zip/node_modules/locale/lib/index.js:159:4
    at instantiateModule (file:///Users/rtsao/code/vite-bug-repro-ssr-pnp/.yarn/__virtual__/vite-virtual-f00fbd239e/0/cache/vite-npm-3.0.8-cf480b5a77-ec3f57d52f.zip/node_modules/vite/dist/node/chunks/dep-74663fff.js:50512:15)

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to vuejs/core instead.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[sun0day]

It seems like Vite has not supported ssr pre bundle yet. See https://github.com/vitejs/vite/blob/main/packages/vite/src/node/optimizer/optimizer.ts#L724

[patak-dev]

There is now support for prebundling SSR deps during dev, which should give a way out of these issues. You need to manually enable it, and add these dependencies to the include list:

ssr: {
  optimizeDeps: {
    disabled: 'build'
    include: [ 'dep' ]
  }
}

[sun0day]

It didn't work, did I miss something? @patak-dev

There is now support for prebundling SSR deps during dev, which should give a way out of these issues. You need to manually enable it, and add these dependencies to the include list:

ssr: {
  optimizeDeps: {
    disabled: 'build'
    include: [ 'dep' ]
  }
}

[patak-dev]

Just in case @sun0day, 'dep' was a placeholder, did you replace that with the dependency you want to optimize? Please share a minimal reproduction of your setup if that is the case

[sun0day]

Just in case @sun0day, 'dep' was a placeholder, did you replace that with the dependency you want to optimize? Please share a minimal reproduction of your setup if that is the case

I just clone https://github.com/rtsao/vite-bug-repro-ssr-pnp mentioned above, and add

ssr: {
    optimizeDeps: {
      disabled: 'build',
      include: [ 'pkg-a' ]
    }
  }

to app/vite.config.js. Then follow the reproduction steps as README.md says.

[rtsao]

I'm aware of a workaround to forcibly externalize these deps (as they should be): {ssr: {external: ["the-dep"]}}) but this is rather cumbersome for large projects where there could be hundreds of CommonJS sub-dependencies. Very few npm packages are ESM.

I think the heuristics for whether dependencies should be externalized is broken somehow.

There is now support for prebundling SSR deps during dev

Why bundle server dependencies? Isn't this just overhead?

[rtsao]

Put together a fix: #9763

TL;DR: the current externalization method of attempting to resolve all nested deps from root is broken with pnpm and Yarn PnP

[arthurgailes]

@rtsao Thank you. This also appears to affect nx workspaces, in my case with the package d3.