[Bug?]: babel runtime, cannot import ESM module

[JonFranchi]

Self-service

• I'd be willing to implement a fix

Describe the bug

I have an older service using babel/register and babel/runtime (for the time being), and it transpiles to commonjs. Some modules are now ESM only and importing them became a problem in this particular project.

I run containerized and am required to have entrypoint to be node to run on PID 1, so I can't run yarn. It works fine locally running yarn start.

So we have for a while used node -r .pnp.cjs start.js in the container and that was no problem.

After we upgraded a particular package to an ESM only version, we started getting this error:

Error loading ESM Module: Error: Cannot find package 'oidc-provider' imported from /home/<redacted>/code/yarn-esm-cjs-reproduction/dynamic-import.js
Did you mean to import "oidc-provider/lib/index.js"?
    at packageResolve (node:internal/modules/esm/resolve:873:9)
    at moduleResolve (node:internal/modules/esm/resolve:946:18)
    at defaultResolve (node:internal/modules/esm/resolve:1188:11)
    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:542:12)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:510:25)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:239:38)
    at ModuleLoader.import (node:internal/modules/esm/loader:472:34)
    at defaultImportModuleDynamicallyForScript (node:internal/modules/esm/utils:227:31)
    at importModuleDynamicallyCallback (node:internal/modules/esm/utils:249:12)
    at _callee$ (/home/<redacted>/code/yarn-esm-cjs-reproduction/dynamic-import.js:1:25)
    at tryCatch (/home/<redacted>/code/yarn-esm-cjs-reproduction/.yarn/cache/@babel-runtime-npm-7.26.9-8369ff039f-e851713111.zip/node_modules/@babel/runtime/helpers/regeneratorRuntime.js:45:16)

Digging through every thread and bit of documentation on this, I couldn't find a comparable example. It had been on 3.x and upgraded to 4.6 which didn't help.

Eventually I pieced together the idea to try both the cjs and mjs loader and that worked.
node --experimental-loader ./.pnp.loader.mjs -r ./.pnp.cjs bootstrap.js

The problem is, I'm not exactly sure why I was able to get it to work this way as I really found no documentation or issues that suggested this should be done, so I'm hesitant to believe this is a good solution. Eventually we do want to rip out babel/runtime as well as upgrade from node 18, but in the meantime, should I expect a performance hit? Is this a workable solution on higher existing node upgrades? And is this forward compatible with how you intend yarn to function in future versions?

To reproduce

I created a minimal reproduction repo:
https://github.com/JonFranchi/yarn-esm-cjs-reproduction

1. run yarn install
2. To run the app the normal way without issue, run yarn start
3. To see the ESM module fail, run node -r .pnp.cjs bootstrap.js
4. To see the ESM module work, run node --experimental-loader ./.pnp.loader.mjs -r ./.pnp.cjs bootstrap.js

Environment

Node 18
yarn 4.6
@babel/register@7.x
@babel/runtime@7.x

Additional context

No response