Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(builtin): linker incorrectly resolves workspace node_modules for windows #2659

Conversation

devversion
Copy link
Contributor

@devversion devversion commented May 6, 2021

Depending on whether there are generated files being stored in
bazel-out/bin/external/npm or not, the linker might resolve the
workspace node modules incorrectly for Windows. e.g.

consider the manifest being the followed:

npm/karma/bin/_karma.module_mappings.json C:/users/paul/<..>/execroot/angular_material/bazel-out/<..>/bin/external/npm/<..>`
npm/node_modules/@angular/core/index.js C:/users/paul/projects/material2/node_modules/@angular/core/index.js

The linker currently on Windows will resolve the NPM workspace to the
bazel-out directory. This is incorrect as on Windows, the node modules
are not symlinked as with linux/macOS. Hence the node modules are not
symlinked to the execroot properly and tests/builds fail due to NodeJS
imports being "faulty".

The fix is to simply use the runfile helpers to resolve the workspace
node modules. There is no need in resolving the workspace root if we
can just directly narrow the lookup to the workspace node modules.

This is needed as the runfile resolution with manifest naively looks
for the first entry starting with npm in the manifest. This breaks
the assumption that the resolved workspace path refers to a directory
that also contains the node_modules folder (as seen in the example
above).

@google-cla google-cla bot added the cla: yes label May 6, 2021
@devversion devversion force-pushed the fix/windows-linker-node-modules-not-linked-properly branch from ab1895a to f7e7579 Compare May 7, 2021 15:20
…r windows

Depending on whether there are generated files being stored in
`bazel-out/bin/external/npm` or not, the linker might resolve the
workspace node modules incorrectly for Windows. e.g.

consider the manifest being the followed:

``
npm/karma/bin/_karma.module_mappings.json C:/users/paul/<..>/execroot/angular_material/bazel-out/<..>/bin/external/npm/<..>`
npm/node_modules/@angular/core/index.js C:/users/paul/projects/material2/node_modules/@angular/core/index.js
```

The linker currently on Windows will resolve the NPM workspace to the
`bazel-out` directory. This is incorrect as on Windows, the node modules
are not symlinked as with linux/macOS. Hence the node modules are not
symlinked to the execroot properly and tests/builds fail due to NodeJS
imports being "faulty".

The fix is to simply use the runfile helpers to resolve the workspace
node modules. There is no need in resolving the workspace root if we
can just directly narrow the lookup to the workspace node modules.

This is needed as the runfile resolution with manifest naively looks
for the first entry starting with `npm` in the manifest. This breaks
the assumption that the resolved workspace path refers to a directory
that also contains the `node_modules` folder (as seen in the example
above).
@devversion devversion force-pushed the fix/windows-linker-node-modules-not-linked-properly branch from f7e7579 to ab8cc76 Compare May 7, 2021 15:24
@devversion devversion marked this pull request as ready for review May 7, 2021 15:54
@@ -309,7 +309,7 @@ function main(args, runfiles) {
}
else {
log_verbose(`no npm workspace node_modules folder under ${packagePath} to link to; creating node_modules directories in ${process.cwd()} for ${packagePath} 1p deps`);
yield mkdirp('${packagePath}/node_modules');
yield mkdirp(`${packagePath}/node_modules`);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😱

Copy link
Collaborator

@gregmagolan gregmagolan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fix. Thank you.
Windows CI is currently very flaky. I'll hit retry on it.

@alexeagle
Copy link
Collaborator

Thanks Paul, awesome fix 😎

@alexeagle alexeagle merged commit 7cf7d73 into bazel-contrib:stable May 7, 2021
twheys pushed a commit to twheys/rules_nodejs that referenced this pull request Jan 13, 2022
…r windows (bazel-contrib#2659)

Depending on whether there are generated files being stored in
`bazel-out/bin/external/npm` or not, the linker might resolve the
workspace node modules incorrectly for Windows. e.g.

consider the manifest being the followed:

``
npm/karma/bin/_karma.module_mappings.json C:/users/paul/<..>/execroot/angular_material/bazel-out/<..>/bin/external/npm/<..>`
npm/node_modules/@angular/core/index.js C:/users/paul/projects/material2/node_modules/@angular/core/index.js
```

The linker currently on Windows will resolve the NPM workspace to the
`bazel-out` directory. This is incorrect as on Windows, the node modules
are not symlinked as with linux/macOS. Hence the node modules are not
symlinked to the execroot properly and tests/builds fail due to NodeJS
imports being "faulty".

The fix is to simply use the runfile helpers to resolve the workspace
node modules. There is no need in resolving the workspace root if we
can just directly narrow the lookup to the workspace node modules.

This is needed as the runfile resolution with manifest naively looks
for the first entry starting with `npm` in the manifest. This breaks
the assumption that the resolved workspace path refers to a directory
that also contains the `node_modules` folder (as seen in the example
above).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants