[BUG] npm ci can't resolve dependencies without --force, or --legacy-peer-deps

[Darkein]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

I've updated to latest node version which use npm 8.11.0, but the problem exist with versions >= 8.6.0 up to 8.12.1

when I do an npm install --legacy-peer-deps, all dependencies are correctly installed and I can start my app.

but when my CI try do make an npm ci, I have conflit errors

>npm ci
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: @nestjs/elasticsearch@8.1.0
npm ERR! Found: @elastic/elasticsearch@5.6.22
npm ERR! node_modules/@elastic/elasticsearch
npm ERR!   @elastic/elasticsearch@"~5.6.22" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer @elastic/elasticsearch@"^7.4.0 || ^8.0.0" from @nestjs/elasticsearch@8.1.0
npm ERR! node_modules/@nestjs/elasticsearch
npm ERR!   @nestjs/elasticsearch@"~8.1.0" from the root project
npm ERR!
npm ERR! Conflicting peer dependency: @elastic/elasticsearch@8.2.1
npm ERR! node_modules/@elastic/elasticsearch
npm ERR!   peer @elastic/elasticsearch@"^7.4.0 || ^8.0.0" from @nestjs/elasticsearch@8.1.0
npm ERR!   node_modules/@nestjs/elasticsearch
npm ERR!     @nestjs/elasticsearch@"~8.1.0" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

Expected Behavior

I should not have any errors when I do npm ci since it should use package-lock wich contains all conflict resolutions.
If I revert to npm@8.5.5, it works correctly without having to do a new npm install

Steps To Reproduce

npm init
npm install @nestjs/elasticsearch@8.1.0 --package-lock-only
npm install @elastic/elasticsearch@5.6.22 --package-lock-only --legacy-peer-deps
npm ci

you can also try without --package-lock-only but by removing node_modules folder

Environment

• npm: 8.11.0
• Node.js: 6.15.1
• OS Name: Alpine
• System Model Name: WSL
• npm config:

; "builtin" config from /usr/local/lib/node_modules/npm/npmrc

globalignorefile = "/etc/npmignore"
prefix = "/usr/local"
python = "/usr/bin/python3"

; "user" config from /home/pierre/.npmrc

save-prefix = "~"

; node bin location = /usr/bin/node
; cwd = ***
; HOME = /home/pierre
; Run `npm config ls -l` to show all defaults.

[ranyefet]

This is a breaking change without bumping major version. Any update on this issue?

[stormbard]

Not sure if this is related. I have a package-lock file generated using npm 8.5.0 when I try to do a npm ci using npm 8.11.0 I get the following

npm ERR! code EUSAGE
npm ERR!
npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
npm ERR!
npm ERR! Invalid: lock file's acorn@7.1.1 does not satisfy acorn@8.7.1
npm ERR! Missing: acorn@7.4.1 from lock file
npm ERR! Missing: acorn@7.4.1 from lock file
npm ERR!
npm ERR! Clean install a project
npm ERR!
npm ERR! Usage:
npm ERR! npm ci
npm ERR!
npm ERR! Options:
npm ERR! [--no-audit] [--foreground-scripts] [--ignore-scripts]
npm ERR! [--script-shell <script-shell>]
npm ERR!
npm ERR! aliases: clean-install, ic, install-clean, isntall-clean
npm ERR!
npm ERR! Run "npm help ci" for more info

npm ERR! A complete log of this run can be found in:

Adding --legacy-peer-deps to the npm ci command allows the clean install to work as I'd expect.

[kopylash]

Also, having this issue. When downgrading npm to 8.1 everything works fine.
Node 16.15.1 was released which includes npm@8.11 and our CI builds started failing because we use node:16 Docker image

[ctretyak]

The bug (feature maybe?, but very annoying) exists since npm@8.6 (included). On npm@8.5 everything good

[l-moveshelf]

I think this is the intended behaviour. The official doc states that:

NOTE: If you create your package-lock.json file by running npm install with flags that can affect the shape of your dependency tree, such as --legacy-peer-deps or --install-links, you must provide the same flags to npm ci or you are likely to encounter errors

[mlvzk]

If this is intended and breaks npm install (if 8.5 works and 8.6 doesn't) shouldn't it have been released in a major update, not minor?

[l-moveshelf]

If this is intended and breaks npm install (if 8.5 works and 8.6 doesn't) shouldn't it have been released in a major update, not minor?

This doesn't "break" npm install; my assumption is that it was previously working in our use case because of a bug, which has now been fixed (can't see the official documentation history, so I'm not sure if it was also stated before)

[ctretyak]

Before 8.6 I could run npm i --force once and after that just used npm i many times. Now I have to use --force every time. For me, it's convenient to allow to install dangerous code once than do mindless operation every time. May you can add any flags that work like "--force" before npm@8.6 (e.g. --force-permanent)?

[l-moveshelf]

I believe you can with the following:

An easy way to do this is to run, for example, npm config set legacy-peer-deps=true --location=project and commit the .npmrc file to your repo.

[stormbard]

This is where my case is slightly different. Generated the lock-file without any special parameters, but adding the --legacy-peer-deps allows it to work(?) with npm 8.11. Without it the process just errors.

[l-moveshelf]

This is where my case is slightly different. Generated the lock-file without any special parameters, but adding the --legacy-peer-deps allows it to work(?) with npm 8.11. Without it the process just errors.

Did you try removing the lock file as well as the node modules and retrying?

[funnylookinhat]

I've seen this behavior as well - but the puzzling part is that I'm seeing it fail in builds and then succeed with a build that is retried.

In spite of the documentation (and that providing either an .npmrc or --legacy-peer-deps fixing the problem), I would consider this a regression - as quite a few build processes have begun failing where they were previously working as intended. Docs are only as good as they adhere to the process. I don't believe including the phrase may as a clause for possible breakage is a fair contract for determining if a change is breaking.

[stormbard]

@luca-moveshelf yeah, I removed the lock-file and node modules and then did an npm install using npm 8.5, swapped to 8.11 and did an npm ci and got the error. If I do a npm install on only 8.11 or just regenerate the package-lock on 8.11 everything works fine without any special flags.

[ljharb]

If you need force or legacy peer deps, it’s because your dep graph is invalid - nothing should ever have worked, and it’s critically important to break things so you’re notified that your graph is invalid.

Your top priority should be fixing your dep graph so that npm install works without any flags.

[jcarrus]

@funnylookinhat I also was puzzled by the intermittency of the issue, but I've just realized that Github Actions's Setup Node action logs the cached version of node that it uses:

and I've been getting different values for failing and succeeding runs:

So that may be something to consider if you are using GH Actions.

[mofojed]

Similar issue between v8.3.1 and v8.3.2 - a package-lock.json that was created with v8.3.1 is no longer valid with v8.3.2.

Issue here is that in .nvmrc and in our GitHub actions we had lts/gallium specified as our version of node, but then builds started breaking because the previously generated package-lock.json is no longer valid.

If you need force or legacy peer deps, it’s because your dep graph is invalid - nothing should ever have worked, and it’s critically important to break things so you’re notified that your graph is invalid.

Your top priority should be fixing your dep graph so that npm install works without any flags.

@ljharb This may be the case - but the previous package-lock was generated from npm, and even if the dep graph is invalid and nothing should have worked, this still should have been a major version change since it broke compatibility.

[ljharb]

@mofojed the major version change was done in v7, where npm fails to install if your graph is unresolvable.

If it's failing now, I'd consider it a bug that it worked before, and that bug has been fixed.

[mofojed]

@ljharb I've created an example project demonstrating the issue: https://github.com/mofojed/npm-scratch

I agree there probably was an issue with previous versions of npm, and perhaps even how the package-lock.json was being generated. But shouldn't we deprecate those versions of npm in the registry then? As it is right now, there's definitely a change in behaviour.
In that project, with npm v8.5.5, npm install and npm ci work fine, making no changes to the package-lock.json.
However, if you switch to npm v8.11.0, doing an npm ci fails, while doing an npm install updates the package-lock.json.

Even if this is a fix for a bug, I would expect either the previous buggy version to be deprecated, or the new version to be a major version bump. As it is right now, these two versions are not compatible which can cause problems for developers and CI (developer could still be using v8.5.5, CI using v8.x and installs the latest v8.11, and then things break).

[ljharb]

Deprecation isn’t typically used merely for buggy versions; it’s rarely used at all.

[bassner]

Before 8.6 I could run npm i --force once and after that just used npm i many times. Now I have to use --force every time. For me, it's convenient to allow to install dangerous code once than do mindless operation every time. May you can add any flags that work like "--force" before npm@8.6 (e.g. --force-permanent)?

I always considered this to be a feature of npm. Decide once that you want to force whatever npm complains about; npm acknowledges that and stores the forced / desired state in the lock file. From then on, it just takes what's there. It sounds very strange to me that this is a "bug", and it's very bad that this was basically silently introduced to a lot of CI pipelines that stopped to work because they were bumped from Node.js 16.15.0 to 16.15.1 which is a minor patch for which no-one expected any breaking changes

[mofojed]

Deprecation isn’t typically used merely for buggy versions; it’s rarely used at all.

Right - I'm not implying it be used merely for a buggy version. The problem is usage of these versions is incompatible. A package-lock.json file that works/was created in an older version of 8.x does not work when using a newer version of 8.x. Any CI system specifying just v8.x (or Node 16.x, as v16.15.1 updated npm from 8.5.5 to 8.11.0) is at risk of breaking because of this incompatibility (even if that is a fix from a bug).

To avoid this CI must pin a specific version, but then they're at risk for not having the latest security updates. Minor version bumps should not break compatibility, even if it's fixing a previous bug.

[ljharb]

Every single bugfix risks breaking someone relying on the buggy behavior - treating every bug fix as breaking is untenable.

[kvetis]

If you need force or legacy peer deps, it’s because your dep graph is invalid - nothing should ever have worked, and it’s critically important to break things so you’re notified that your graph is invalid.

I consider that an overstatement. My app relies on several packages that are just overly strict when stating their (peer) dependencies - specifically Angular version. They work perfectly fine with newer versions of Angular. I don't have the time to fiddle with all new breaking changes in versions that on paper support newer Angular version (if these versions exists.)

I'm fine with adding a permanent legacy-peer-deps into .npmrc (no, I'm not able to "fix" my dep graph).

I understand treating a bugfix as breaking is untenable, so maybe we can close this linking to the docs and the actual bugfix PR?

[mofojed]

For reference, here was the fix that identified invalid nodes: bd96ae407 #4599 fix(arborist): identify and repair invalid nodes in the virtual tree

What was previously "working" with npm ci (without the --legacy-deps flag) now finds an invalid node and bails out. If you set up package-lock.json with npm install --legacy-deps, but never added the --legacy-deps flag to npm ci, then it's possible to run into this issue; however, it never should have worked in the first place. npm ci --legacy-deps works, or updating dependency graph works to resolve the issue.

In terms of semantic versioning, on the consumer side, it would have been smoother if npm ci just spit out a big warning about invalid nodes, recommending you update your dependency graph (or adding legacy-deps flag), then on the next major version bump make it an error. Or npm install could spit out a message saying it found invalid nodes and it's auto-repairing them.

I assume people are only hitting this now even though it's been in npm since v8.6.0 because they just update with Node or using nvm, and Node just last week updated to v16.15.1, which updated from v8.5.5 to v8.11.0.

In any case, I don't think much can be done about it now. Whoever hits this in their build ci just needs to update their package-lock.json (run npm install and commit it), or add the --legacy-deps flag to npm ci (which we should have had in the first place).

[ljharb]

@kvetis only the package author is the arbiter of "overly strict" - if the package declares a peer dep, then it is not compatible with anything outside of that range, even if it happens to work.

[paulius-valiunas]

@kvetis only the package author is the arbiter of "overly strict" - if the package declares a peer dep, then it is not compatible with anything outside of that range, even if it happens to work.

this would be true if npm resolved semver numbers correctly. But for some reason it thinks 1.2.0-rc.1 doesn't satisfy ^1.1.0.

https://semver.org/#spec-item-9

[ljharb]

That is correct; ranges aren’t in the semver spec (yet) so there’s no contradiction. Ranges on a non-prerelease don’t include prereleases, never have, and never will.

[phil-lgr]

I am just coming back to npm after 3 years on yarn v1 and this issue broke our CI today, we have a large mono repo that we install without --legacy-peer-deps, and the npm ci commands fails

only way to fix it was to run npm i --legacy-peer-deps and then npm ci --legacy-peer-deps in our CI

[mohd-akram]

Isn't like the entire point of npm ci to be deterministic behavior that does exactly what's in package-lock? You don't change the documentation to justify a behavior change and then call it a "bugfix".

[nhowell]

Isn't like the entire point of npm ci to be deterministic behavior that does exactly what's in package-lock? You don't change the documentation to justify a behavior change and then call it a "bugfix".

Completely, 100% agree! npm ci shouldn't care if the package-lock.json is kind of "invalid". It was working before, so at most this "bugfix" should have been a warning. It is entirely unacceptable to break a bunch of people's npm ci and call it a "bugfix". If NPM doesn't want to support the "invalid" package locks in future versions, then make the breaking change in a major version bump, for goodness sake.

[syamanashi]

For anyone stumbling into this thread looking for clear instructions for a work-around to repair your failed Github Actions build for an Angular project ASAP, this worked for me:

1. Add a new file to the root of your project named .npmrc
2. Add this single line to this file and save it: legacy-peer-deps=true
3. Run install command (this step is optional but useful to ensure nothing goes haywire): npm i
4. Commit this change into your branch and push or perform a PR merge as usual that kicks off your build process.

[ObieMunoz]

Rebuilding the package-lock using the updated versions of node and npm resolved the issue (for us).

[snebjorn]

What is the point of using the legacy-peer-deps flag when it have to be applied everywhere all the time? Doesn't that make it redundant?
Sure in the ideal world packages would always have updated peer dependency ranges. But in the real world packages use ranges like peerA@">=16.0.0 <18.0.0" and then peerA@18 is released and it works. But the author of the package is on hiatus so a new version won't come for some time.
Now my build fails because of this.

It used to be a decision made when installing/updating a peer dependency. Once a person had installed a package using --legacy-peer-deps then subsequent npm i / npm ci would work because the person that installed the package with outdated peer dependencies had checked that it worked.

That made sense to me. It was nice to get an error. It was nice to tell NPM "hey I checked it works and it's fine".
Now I have to put legacy-peer-deps=true in .npmrc and I won't get an error when I try to install something with outdated peer dependencies. I also won't know if other people checked if a package worked.

Can this please be reverted? It was working before.

[ouzkhanmete]

Same here, suddenly all the pipelines with Node 16 in it started to throw errors. If we are forced to put the --legacy-peer-deps in .npmrc for every service with Node 16+ - what is the point in this flag at all? Why there are no big announcements on that? Why it was bumped in minor version, not major?

My vote is to revert this change as it ruins peoples' time with no reason.

[ObieMunoz]

Same here, suddenly all the pipelines with Node 16 in it started to throw errors. If we are forced to put the --legacy-peer-deps in .npmrc for every service with Node 16+ - what is the point in this flag at all? Why there are no big announcements on that? Why it was bumped in minor version, not major? My vote is to revert this change as it ruins peoples' time with no reason.

Did you update your versions locally and rebuild your package-lock file?

[nhowell]

Did you update your versions locally and rebuild your package-lock file?

This is exactly what people should not have to do. Their existing package-lock.json file should continue to work. Any requirement otherwise is a breaking change to npm ci and should be reverted.

[mofojed]

To avoid this kind of issue in the future, I'll pin CI and dev to a specific version of node/npm (using .nvmrc file) instead of pinning to an lts release (previously we used 16.x in GitHub actions and lts/gallium in our .nvmrc).

[ouzkhanmete]

Same here, suddenly all the pipelines with Node 16 in it started to throw errors. If we are forced to put the --legacy-peer-deps in .npmrc for every service with Node 16+ - what is the point in this flag at all? Why there are no big announcements on that? Why it was bumped in minor version, not major? My vote is to revert this change as it ruins peoples' time with no reason.

Did you update your versions locally and rebuild your package-lock file?

No I did not. I have like 100+ dependencies broken, do you want my team to waste a month of work rebuilding the codebase over few hundred repositories? Are you joking?

We rolled back to Node v16.15.0, which has a working version of npm 8.5.5.

--legacy-peer-deps flag did not help, when rebuilding package-lock.json from scratch and doing npm ci with the very same flag afterwards.

[NotYevvie]

Are there any plans to deprecate / remove support for --legacy-peer-deps in the near future (0-2 years)?

[paulius-valiunas]

Same here, suddenly all the pipelines with Node 16 in it started to throw errors. If we are forced to put the --legacy-peer-deps in .npmrc for every service with Node 16+ - what is the point in this flag at all? Why there are no big announcements on that? Why it was bumped in minor version, not major? My vote is to revert this change as it ruins peoples' time with no reason.

Did you update your versions locally and rebuild your package-lock file?

No I did not. I have like 100+ dependencies broken, do you want my team to waste a month of work rebuilding the codebase over few hundred repositories? Are you joking?

We rolled back to Node v16.15.0, which has a working version of npm 8.5.5.

--legacy-peer-deps flag did not help, when rebuilding package-lock.json from scratch and doing npm ci with the very same flag afterwards.

All you have to do is install node.js 16.15.1 locally and run npm install. Sometimes that's enough to fix this problem.

[ObieMunoz]

Same here, suddenly all the pipelines with Node 16 in it started to throw errors. If we are forced to put the --legacy-peer-deps in .npmrc for every service with Node 16+ - what is the point in this flag at all? Why there are no big announcements on that? Why it was bumped in minor version, not major? My vote is to revert this change as it ruins peoples' time with no reason.

Did you update your versions locally and rebuild your package-lock file?

No I did not. I have like 100+ dependencies broken, do you want my team to waste a month of work rebuilding the codebase over few hundred repositories? Are you joking?

We rolled back to Node v16.15.0, which has a working version of npm 8.5.5.

--legacy-peer-deps flag did not help, when rebuilding package-lock.json from scratch and doing npm ci with the very same flag afterwards.

All you have to do is install node.js 16.15.1 locally and run npm install. Sometimes that's enough to fix this problem.

That's what I was saying but apparently that's not ok. It fixed all of our pipelines though.

[fritzy]

The consistency between your package-lock.json vs your package.json has security implications, among other things. Additionally npm can't infer the configuration (like --legacy-peer-deps) used in npm install when you later use npm ci, which is what this specific bug report was about.

It's unfortunate that this fix has caused so many npm ci failures, but if your package.json isn't properly represented in your package-lock.json, then you'll need to regenerate your package-lock.json with a new npm install. We'll take some lessons learned here about what we could do better in the future when addressing bugs that might disrupt workflows when fixed.

Again, this particular report has to do with using a consistent configuration between the command that generates the lock and the command the uses the lock. As such, I'm going to close this issue. If you'd like to start a discussion around storing the configuration in the lockfile, or other possible behavior changes, please create an RRFC or start a discussion in the repo.

That's not to say that there couldn't be bugs that need to be addressed related to lockfile consistency. If you come across one, please submit a bug report.

[paulius-valiunas]

The consistency between your package-lock.json vs your package.json has security implications, among other things. Additionally npm can't infer the configuration (like --legacy-peer-deps) used in npm install when you later use npm ci, which is what this specific bug report was about.

It's unfortunate that this fix has caused so many npm ci failures, but if your package.json isn't properly represented in your package-lock.json, then you'll need to regenerate your package-lock.json with a new npm install. We'll take some lessons learned here about what we could do better in the future when addressing bugs that might disrupt workflows when fixed.

Again, this particular report has to do with using a consistent configuration between the command that generates the lock and the command the uses the lock. As such, I'm going to close this issue. If you'd like to start a discussion around storing the configuration in the lockfile, or other possible behavior changes, please create an RRFC or start a discussion in the repo.

That's not to say that there couldn't be bugs that need to be addressed related to lockfile consistency. If you come across one, please submit a bug report.

it doesn't matter if you think the configurations people used were "correct" or not, as you can see a significant number of people used it this way and it worked before. Now it stopped working, which is a breaking change by definition. The correct way to fix this bug would have been either:

1. treat it as any other breaking change and do a semver-major version bump
2. add a new field to package-lock.json that would indicate that the file was generated using newer npm version, and use the old behavior for files without this flag. You can remove this behavior in the next major release.

What you just did is incompatible with semver because it breaks people's workflows without declaring a major version bump. What's even worse is that it's not even a minor bump, just a patch.

[nhowell]

It's unfortunate that this fix has caused so many npm ci failures.

It's unfortunate unacceptable that this fix has caused so many npm ci failures.

FTFY.

This broke my workplace and many other's CI pipelines, in a patch version no less, and you expect the community to be fine with an "oh well, we'll do better next time" response. Unacceptable. Go back and fix it in a backwards compatible way.

My workplace has never used any special flags when running npm i nor npm ci (e.g. --force, or --legacy-peer-deps), yet our CI pipeline was suddenly broken due to this so called "bugfix". So much for npm ci with a package-lock.json being deterministic behavior.

If NPM would've came out and said, "Oops, that was an unexpected consequence of this bugfix - we'll fix it up in the next patch version" I would be completely satisfied. Instead, this issue and its response has caused me to lose all confidence in NPM CLI updates. Guess we'll be pinning an exact version of Node/NPM in our CI pipeline from here on out, or moving to Yarn.

[fritzy]

I have created a pinned issue summarizing this discussion as well as #4664.

See the please refer to the pinned issue, #5113