Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix install with local modules under multiple paths #217

Closed
wants to merge 2 commits into from
Closed

Fix install with local modules under multiple paths #217

wants to merge 2 commits into from

Conversation

salomvary
Copy link
Contributor

I found this issue a few days ago, added a test case, git bisected and found 2f0c883 to be the cause of the problem.

Reverting said commit seems to resolve the problem. Verified it by running npm run tap test/tap/install-from-local-multipath.js before and after the revert on latest.

I tried running the entire test suite and seems like there no failing tests caused by the revert.

Note: I have no idea what I am doing and the author made no changes to tests nor left any notes on the reason for the changes in 2f0c883. Someone with more knowledge should have a careful look what the proper solution should be.

@salomvary salomvary requested a review from a team as a code owner July 22, 2019 10:24
@isaacs
Copy link
Contributor

isaacs commented Jul 22, 2019

Yeah, I think you've found a bug, but I'm going to have to do a bit of work to understand the solution before believing that it won't cause other problems.

Kicked out to 6.10.3 (at least).

@salomvary
Copy link
Contributor Author

@isaacs looks like a package-lock.json created before the "fix" is partially rewritten by an npm install after the fix. Might be an unwanted effect, shall I post some examples?

@isaacs
Copy link
Contributor

isaacs commented Jul 23, 2019

@salomvary yeah, reproduction examples are always great, thanks :)

@salomvary
Copy link
Contributor Author

@isaacs given an existing package-lock.json and an empty node_modules folder, the patched version does rewrites like this:

     "my-shared-project": {
-      "version": "file:../parent-folder-of-shared-projects/my-shared-project",
-      "requires": {
-        "ajv": "^6.5.0",
-        "ajv-keywords": "^3.2.0",
-        "ajv-merge-patch": "^3.0.0",
-        "clone": "^2.1.2",
-        "dot-prop": "^4.2.0",
-        "json-schema-ref-parser": "^4.1.1"
-      }
+      "version": "file:../parent-folder-of-shared-projects/my-shared-project"
     },

isaacs added a commit that referenced this pull request Aug 6, 2019
@isaacs
install: do not descend into directory deps' child modules
1ad53e5 
This builds on the work of #217, bringing
back the logic in 2f0c883 for all deps
other than 'directory' symlinks where it causes problems.

This also causes the installer to *fix* shrinkwraps that inappropriately
list the dependencies of directory symlink packages, which goes further
to address the problems highlighted in
https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863
and
https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327,
even if a prior npm install created a broken package-lock.json file.

Related: #217
Credit: @isaacs
Fix: https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327
Fix: https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863
isaacs added a commit that referenced this pull request Aug 6, 2019
@isaacs
install: do not descend into directory deps' child modules
ff7abd9 
This builds on the work of #217, bringing
back the logic in 2f0c883 for all deps
other than 'directory' symlinks where it causes problems.

This also causes the installer to *fix* shrinkwraps that inappropriately
list the dependencies of directory symlink packages, which goes further
to address the problems highlighted in
https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863
and
https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327,
even if a prior npm install created a broken package-lock.json file.

Related: #217
Credit: @isaacs
Fix: https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327
Fix: https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863
isaacs added a commit that referenced this pull request Aug 6, 2019
@isaacs
install: do not descend into directory deps' child modules
45772af 
This builds on the work of #217, bringing
back the logic in 2f0c883 for all deps
other than 'directory' symlinks where it causes problems.

This also causes the installer to *fix* shrinkwraps that inappropriately
list the dependencies of directory symlink packages, which goes further
to address the problems highlighted in
https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863
and
https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327,
even if a prior npm install created a broken package-lock.json file.

Related: #217
Credit: @isaacs
Fix: https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327
Fix: https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863
@isaacs
Copy link
Contributor

isaacs commented Aug 6, 2019

Ok, got a chance to review this. Turns out, yes, this fix does have some other side effects, and doesn't address the problem for all cases. The rest of the fix is on 45772af.

It will be in the next release.

@isaacs isaacs added semver:patch semver patch level for changes and removed needs-discussion labels Aug 12, 2019
@isaacs isaacs closed this in 24acc9f Aug 20, 2019
@isaacs isaacs mentioned this pull request Aug 20, 2019
Closed
Trott pushed a commit to npm/node that referenced this pull request Aug 20, 2019
@isaacs @Trott
deps: update npm to 6.11.0
5fb908e 
Full release notes:

A few meaty bugfixes, and introducing `peerDependenciesMeta`.

FEATURES

* [`a12341088`](npm/cli@a123410)
  [nodejs#224](npm/cli#224) Implements
  peerDependenciesMeta ([@arcanis](https://github.com/arcanis))
* [`2f3b79bba`](npm/cli@2f3b79b)
  [nodejs#234](npm/cli#234) add new forbidden 403
  error code ([@claudiahdz](https://github.com/claudiahdz))

BUGFIXES

* [`24acc9fc8`](npm/cli@24acc9f)
  and
  [`45772af0d`](npm/cli@45772af)
  [nodejs#217](npm/cli#217)
  [npm.community#8863](https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863)
  [npm.community#9327](https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327,)
  do not descend into directory deps' child modules, fix shrinkwrap
  files that inappropriately list child nodes of symlink packages
  ([@isaacs](https://github.com/isaacs) and
  [@salomvary](https://github.com/salomvary))
* [`50cfe113d`](npm/cli@50cfe11)
  [nodejs#229](npm/cli#229) fixed typo in semver doc
  ([@gall0ws](https://github.com/gall0ws))
* [`e8fb2a1bd`](npm/cli@e8fb2a1)
  [nodejs#231](npm/cli#231) Fix spelling mistakes in
  CHANGELOG-3.md ([@XhmikosR](https://github.com/XhmikosR))
* [`769d2e057`](npm/cli@769d2e0)
  [npm/uid-number#7](npm/uid-number#7) Better
  error on invalid `--user`/`--group` configs. This addresses the issue
  when people fail to install binary packages on Docker and other
  environments where there is no 'nobody' user.
  ([@isaacs](https://github.com/isaacs))
* [`8b43c9624`](npm/cli@8b43c96)
  [nodejs#28987](nodejs#28987)
  [npm.community#6032](https://npm.community/t/npm-ci-doesnt-respect-npmrc-variables/6032)
  [npm.community#6658](https://npm.community/t/npm-ci-doesnt-fill-anymore-the-process-env-npm-config-cache-variable-on-post-install-scripts/6658)
  [npm.community#6069](https://npm.community/t/npm-ci-does-not-compile-native-dependencies-according-to-npmrc-configuration/6069)
  [npm.community#9323](https://npm.community/t/npm-6-9-x-not-passing-environment-to-node-gyp-regression-from-6-4-x/9323/2)
  Fix the regression where random config values in a .npmrc file are not
  passed to lifecycle scripts, breaking build processes which rely on
  them.  ([@isaacs](https://github.com/isaacs))
* [`8b85eaa47`](npm/cli@8b85eaa)
  save files with inferred ownership rather than relying on `SUDO_UID`
  and `SUDO_GID`. ([@isaacs](https://github.com/isaacs))
* [`b7f6e5f02`](npm/cli@b7f6e5f)
  Infer ownership of shrinkwrap files
  ([@isaacs](https://github.com/isaacs))
* [`54b095d77`](npm/cli@54b095d)
  [nodejs#235](npm/cli#235) Add spec to dist-tag
  remove function ([@theberbie](https://github.com/theberbie))

DEPENDENCIES

* [`dc8f9e52f`](npm/cli@dc8f9e5)
  `pacote@9.5.7`: Infer the ownership of all unpacked files in
  `node_modules`, so that we never have user-owned files in root-owned
  folders, or root-owned files in user-owned folders.
  ([@isaacs](https://github.com/isaacs))
* [`bb33940c3`](npm/cli@bb33940)
  `cmd-shim@3.0.0`:
  * [`9c93ac3`](npm/cmd-shim@9c93ac3)
    [#2](npm/cmd-shim#2)
    [npm#3380](npm/npm#3380) Handle
    environment variables properly
    ([@basbossink](https://github.com/basbossink))
  * [`2d277f8`](npm/cmd-shim@2d277f8)
    [nodejs#25](npm/cmd-shim#25)
    [nodejs#36](npm/cmd-shim#36)
    [nodejs#35](npm/cmd-shim#35) Fix 'no shebang' case
    by always providing `$basedir` in shell script
    ([@igorklopov](https://github.com/igorklopov))
  * [`adaf20b`](npm/cmd-shim@adaf20b)
    [nodejs#26](npm/cmd-shim#26) Fix `$*` causing an
    error when arguments contain parentheses
    ([@satazor](https://github.com/satazor))
  * [`49f0c13`](npm/cmd-shim@49f0c13)
    [nodejs#30](npm/cmd-shim#30) Fix paths for
    MSYS/MINGW bash ([@dscho](https://github.com/dscho))
  * [`51a8af3`](npm/cmd-shim@51a8af3)
    [nodejs#34](npm/cmd-shim#34) Add proper support
    for PowerShell ([@ExE-Boss](https://github.com/ExE-Boss))
  * [`4c37e04`](npm/cmd-shim@4c37e04)
    [#10](npm/cmd-shim#10) Work around quoted
    batch file names ([@isaacs](https://github.com/isaacs))
* [`a4e279544`](npm/cli@a4e2795)
  `npm-lifecycle@3.1.3` ([@isaacs](https://github.com/isaacs)):
    * fail properly if `uid-number` raises an error
    * [`7086a1809`](npm/cli@7086a18)
  `libcipm@4.0.3` ([@isaacs](https://github.com/isaacs))
* [`8845141f9`](npm/cli@8845141)
  `read-package-json@2.1.0` ([@isaacs](https://github.com/isaacs))
* [`51c028215`](npm/cli@51c0282)
  `bin-links@1.1.3` ([@isaacs](https://github.com/isaacs))
* [`534a5548c`](npm/cli@534a554)
  `read-cmd-shim@1.0.3` ([@isaacs](https://github.com/isaacs))
* [`3038f2fd5`](npm/cli@3038f2f)
  `gentle-fs@2.2.1` ([@isaacs](https://github.com/isaacs))
* [`a609a1648`](npm/cli@a609a16)
  `graceful-fs@4.2.2` ([@isaacs](https://github.com/isaacs))
* [`f0346f754`](npm/cli@f0346f7)
  `cacache@12.0.3` ([@isaacs](https://github.com/isaacs))
* [`ca9c615c8`](npm/cli@ca9c615)
  `npm-pick-manifest@3.0.0` ([@isaacs](https://github.com/isaacs))
* [`b417affbf`](npm/cli@b417aff)
  `pacote@9.5.8` ([@isaacs](https://github.com/isaacs))

TESTS

* [`b6df0913c`](npm/cli@b6df091)
  [nodejs#228](npm/cli#228) Proper handing of
  /usr/bin/node lifecycle-path test
  ([@olivr70](https://github.com/olivr70))
* [`aaf98e88c`](npm/cli@aaf98e8)
  `npm-registry-mock@1.3.0` ([@isaacs](https://github.com/isaacs))
isaacs added a commit to npm/node that referenced this pull request Aug 21, 2019
@isaacs
deps: update npm to 6.11.1
ee11376 
Full changelog:

6.11.1 (2019-08-20):

Fix a regression for windows command shim syntax.

* [`37db29647`](npm/cli@37db296)
  `cmd-shim@3.0.2` ([@isaacs](https://github.com/isaacs))

v6.11.0 (2019-08-20):

A few meaty bugfixes, and introducing `peerDependenciesMeta`.

FEATURES

* [`a12341088`](npm/cli@a123410)
  [nodejs#224](npm/cli#224) Implements
  peerDependenciesMeta ([@arcanis](https://github.com/arcanis))
* [`2f3b79bba`](npm/cli@2f3b79b)
  [nodejs#234](npm/cli#234) add new forbidden 403 error
  code ([@claudiahdz](https://github.com/claudiahdz))

BUGFIXES

* [`24acc9fc8`](npm/cli@24acc9f)
  and
  [`45772af0d`](npm/cli@45772af)
  [nodejs#217](npm/cli#217)
  [npm.community#8863](https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863)
  [npm.community#9327](https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327,)
  do not descend into directory deps' child modules, fix shrinkwrap files
  that inappropriately list child nodes of symlink packages
  ([@isaacs](https://github.com/isaacs) and
  [@salomvary](https://github.com/salomvary))
* [`50cfe113d`](npm/cli@50cfe11)
  [nodejs#229](npm/cli#229) fixed typo in semver doc
  ([@gall0ws](https://github.com/gall0ws))
* [`e8fb2a1bd`](npm/cli@e8fb2a1)
  [nodejs#231](npm/cli#231) Fix spelling mistakes in
  CHANGELOG-3.md ([@XhmikosR](https://github.com/XhmikosR))
* [`769d2e057`](npm/cli@769d2e0)
  [npm/uid-number#7](npm/uid-number#7) Better
  error on invalid `--user`/`--group` configs. This addresses the issue
  when people fail to install binary packages on Docker and other
  environments where there is no 'nobody' user.
  ([@isaacs](https://github.com/isaacs))
* [`8b43c9624`](npm/cli@8b43c96)
  [nodejs#28987](nodejs#28987)
  [npm.community#6032](https://npm.community/t/npm-ci-doesnt-respect-npmrc-variables/6032)
  [npm.community#6658](https://npm.community/t/npm-ci-doesnt-fill-anymore-the-process-env-npm-config-cache-variable-on-post-install-scripts/6658)
  [npm.community#6069](https://npm.community/t/npm-ci-does-not-compile-native-dependencies-according-to-npmrc-configuration/6069)
  [npm.community#9323](https://npm.community/t/npm-6-9-x-not-passing-environment-to-node-gyp-regression-from-6-4-x/9323/2)
  Fix the regression where random config values in a .npmrc file are not
  passed to lifecycle scripts, breaking build processes which rely on them.
  ([@isaacs](https://github.com/isaacs))
* [`8b85eaa47`](npm/cli@8b85eaa)
  save files with inferred ownership rather than relying on `SUDO_UID` and
  `SUDO_GID`. ([@isaacs](https://github.com/isaacs))
* [`b7f6e5f02`](npm/cli@b7f6e5f)
  Infer ownership of shrinkwrap files
  ([@isaacs](https://github.com/isaacs))
* [`54b095d77`](npm/cli@54b095d)
  [nodejs#235](npm/cli#235) Add spec to dist-tag remove
  function ([@theberbie](https://github.com/theberbie))

DEPENDENCIES

* [`dc8f9e52f`](npm/cli@dc8f9e5)
  `pacote@9.5.7`: Infer the ownership of all unpacked files in
  `node_modules`, so that we never have user-owned files in root-owned
  folders, or root-owned files in user-owned folders.
  ([@isaacs](https://github.com/isaacs))
* [`bb33940c3`](npm/cli@bb33940)
  `cmd-shim@3.0.0`:
  * [`9c93ac3`](npm/cmd-shim@9c93ac3)
    [#2](npm/cmd-shim#2)
    [npm#3380](npm/npm#3380) Handle environment
    variables properly ([@basbossink](https://github.com/basbossink))
  * [`2d277f8`](npm/cmd-shim@2d277f8)
    [nodejs#25](npm/cmd-shim#25)
    [nodejs#36](npm/cmd-shim#36)
    [nodejs#35](npm/cmd-shim#35) Fix 'no shebang' case by
    always providing `$basedir` in shell script
    ([@igorklopov](https://github.com/igorklopov))
  * [`adaf20b`](npm/cmd-shim@adaf20b)
    [nodejs#26](npm/cmd-shim#26) Fix `$*` causing an
    error when arguments contain parentheses
    ([@satazor](https://github.com/satazor))
  * [`49f0c13`](npm/cmd-shim@49f0c13)
    [nodejs#30](npm/cmd-shim#30) Fix paths for MSYS/MINGW
    bash ([@dscho](https://github.com/dscho))
  * [`51a8af3`](npm/cmd-shim@51a8af3)
    [nodejs#34](npm/cmd-shim#34) Add proper support for
    PowerShell ([@ExE-Boss](https://github.com/ExE-Boss))
  * [`4c37e04`](npm/cmd-shim@4c37e04)
    [#10](npm/cmd-shim#10) Work around quoted
    batch file names ([@isaacs](https://github.com/isaacs))
* [`a4e279544`](npm/cli@a4e2795)
  `npm-lifecycle@3.1.3` ([@isaacs](https://github.com/isaacs)):
    * fail properly if `uid-number` raises an error
* [`7086a1809`](npm/cli@7086a18)
  `libcipm@4.0.3` ([@isaacs](https://github.com/isaacs))
* [`8845141f9`](npm/cli@8845141)
  `read-package-json@2.1.0` ([@isaacs](https://github.com/isaacs))
* [`51c028215`](npm/cli@51c0282)
  `bin-links@1.1.3` ([@isaacs](https://github.com/isaacs))
* [`534a5548c`](npm/cli@534a554)
  `read-cmd-shim@1.0.3` ([@isaacs](https://github.com/isaacs))
* [`3038f2fd5`](npm/cli@3038f2f)
  `gentle-fs@2.2.1` ([@isaacs](https://github.com/isaacs))
* [`a609a1648`](npm/cli@a609a16)
  `graceful-fs@4.2.2` ([@isaacs](https://github.com/isaacs))
* [`f0346f754`](npm/cli@f0346f7)
  `cacache@12.0.3` ([@isaacs](https://github.com/isaacs))
* [`ca9c615c8`](npm/cli@ca9c615)
  `npm-pick-manifest@3.0.0` ([@isaacs](https://github.com/isaacs))
* [`b417affbf`](npm/cli@b417aff)
  `pacote@9.5.8` ([@isaacs](https://github.com/isaacs))

TESTS

* [`b6df0913c`](npm/cli@b6df091)
  [nodejs#228](npm/cli#228) Proper handing of
  /usr/bin/node lifecycle-path test
  ([@olivr70](https://github.com/olivr70))
* [`aaf98e88c`](npm/cli@aaf98e8)
  `npm-registry-mock@1.3.0` ([@isaacs](https://github.com/isaacs))
@isaacs isaacs mentioned this pull request Aug 21, 2019
Closed
This was referenced Aug 30, 2019
Closed
Open
Open
Merged
@snyk-bot snyk-bot mentioned this pull request Sep 19, 2019
Open
@snyk-bot snyk-bot mentioned this pull request Oct 1, 2019
Open
This was referenced Oct 26, 2019
Open
Open
This was referenced Nov 5, 2019
Open
Open
@snyk-bot snyk-bot mentioned this pull request Nov 14, 2019
Open
This was referenced Nov 25, 2019
Open
Open
This was referenced Dec 5, 2019
Open
Open
This was referenced Oct 22, 2024
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
semver:patch semver patch level for changes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@salomvary @isaacs