Cannot find module '/app/server/node_modules/@bufbuild/protobuf/dist/proxy/index.js' imported from /app/server/chunks/app/server.mjs

[daviddomkar]

Hi. I am using Nuxt 3 for my application where I use this package. The problem is that after version 1.3.3. I started getting this error in production. From what I found, the proxy directory is missing from the build. I think this change is the cause: #566

Do you have an idea how to resolve this?

[timostamm]

Hey David, the directory is there in the published npm package. I assume that the error is raised by Node.js?

If you are using an older version of Node.js, it is possible that this is a bug in the module resolution that has been fixed in a later version. So I'd try updating Node.js to a latest v18 or v20.

If you set the environment variable NODE_DEBUG=module, Node will print debugging information for the module resolution. For a complex project, the debugging output alone might not show the smoking gun. If you can isolate the problem, we're happy to take a look if you can push it up 🙂

[daviddomkar]

Hi. Thank you for looking into this. I made a small reproduction repo showcasing my problem.

https://github.com/DavidDomkar/bufbuild-protobuf-error-reproduction

If you clone and do yarn install && yarn build && yarn preview, you can see the webpage with the error on localhost:3000. The problem is, despite the proxy folder being in node_modules, when running yarn build, the build process creates another node_modules directory in .output/server/node_modules and there the directory is missing. I am using node 18. I think this is something related to how vite does treeshaking when compiling the server. But I do not know how to fix it properly.

[timostamm]

Interesting, I see the same issue. I'm not familiar with Nuxt, but it looks like it is using Nitro for the server.

Nitro (or some dependency it uses) sees the "module" export we set, and deletes all other exports from our package. Node.js however will prefer the "import" export over "module", which is also set by our package. As a result, our package - as modified by Nitro - is broken on Node.js.

It looks like Nitro is tracking the issue in nitrojs/nitro#1558. I think it would be helpful to provide them with your reproducible example.

[daviddomkar]

Thank you for looking into the details, this was very helpful. I left the comment in the linked issue, hopefully, they will look into it as it is a Nitro issue.

[pi0]

Hi dear @timostamm and @daviddomkar. Checking this repo/packages, it seems there is an issue with ordering of exports field conditions. Rollup prefers the module export condition where as you have to consider it is a nonstandard one but when a library is externalized, native ESM resolution in Node.js, prefers import condition which is a different one. Consider the ordering of fields matters if you want to prefer import, it should be the first.

The fix is pretty straight forward you can see sanity-io/client#318 in sentry package that finally removed this non standard condition also sanity-io/client#309 for more context and alternative. (made #611 draft)

BTW just saw this lib, seems pretty awesome ❤️

[timostamm]

We can't really prefer "import" over "module", as described in #611 (comment).

We might be able to modify our exports as follows:

  "type": "module",
  "exports": {
    ".": {
      "node": {
        "import": "./dist/proxy/index.js" // ESM wrapper specific to Node.js, re-exports CJS
      },
      "require": "./dist/cjs/index.js",
      "default": "./dist/esm/index.js" // pure ESM
    }

We will have to run more extensive tests before we can make this change (connectrpc/examples-es#1002 for context), but it does look reasonable.

@daviddomkar, to work around the issue, you can modify your package.json scripts as follows:

    "build": "nuxt build && yarn run postbuild",
    "postbuild": "cp -r node_modules/@bufbuild/protobuf/dist .output/server/node_modules/@bufbuild/protobuf",

This will restore the build artifacts of the package, until Nitro disregards the "module" condition for the node-server preset, or protobuf-es makes the exports change above (which will take some time to thoroughly test).

[pi0]

We can't really prefer "import" over "module",

I hope you revise this decision in later versions because, module export condition was never a standard and causes problems with ESM-native runtimes. While i fully understand that it is not an easy change for this package specially since it can potentially be breaking change, IMO going toward standards might be better choice.

We might be able to modify our exports as follows:

This would also a good idea to at least support node.import export condition i suppose it will fix the bug. Only consider there are more ESM-native runtimes that support externalized native ESM dependencies. bun and deno. You might consider runtime-keys for the future. (anyway updated #611 with this suggestion as quick fix 👍🏼 )

until Nitro disregards the "module" condition for the node-server preset

Nitro does not disregards module condition. Node.js natively never regards it because standard condition is import for ESM. When trace used files. It is fooled with export condition as rollup resolver (for i guess traditional reasons) still support non-standard module condition and prefers it because this package says to prefer it. It will be extracted but in runtime Node.js looks for import condition which well does not works as file was not traced properly.

There are of course several but I'm afraid nothing is to fix particularly in Nitro other than patching incompatible/nonstandard packages or rollup behavior which I'm afraid is not a safe choice until rollup itself drops the package. (Thinking to add a safety check to automatically deoptimize packages with this situation of having module condition)

[daviddomkar]

We can't really prefer "import" over "module", as described in #611 (comment).

We might be able to modify our exports as follows:

  "type": "module",
  "exports": {
    ".": {
      "node": {
        "import": "./dist/proxy/index.js" // ESM wrapper specific to Node.js, re-exports CJS
      },
      "require": "./dist/cjs/index.js",
      "default": "./dist/esm/index.js" // pure ESM
    }

We will have to run more extensive tests before we can make this change (connectrpc/examples-es#1002 for context), but it does look reasonable.

@daviddomkar, to work around the issue, you can modify your package.json scripts as follows:

    "build": "nuxt build && yarn run postbuild",
    "postbuild": "cp -r node_modules/@bufbuild/protobuf/dist .output/server/node_modules/@bufbuild/protobuf",

This will restore the build artifacts of the package, until Nitro disregards the "module" condition for the node-server preset, or protobuf-es makes the exports change above (which will take some time to thoroughly test).

Thank you for providing the workaround, I will use it until this is resolved another way. I can confirm that the quick fix with the node export condition works for me when I patch the package in node modules.