[BUG] Starting with v10.4.0, --include=optional not working (for transitive dependency (sharp))

[bhallstein]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

We are deploying an app that uses the sharp image manipulation library. A deployment bundle is built on macs, with npm i --cpu=x64 --os=linux --include=optional sharp@0.33.3 called afterwards to install cross-platform deps that can't be bundled, following which a tarball is created and used for deployments.

On npm@10.3.0 and below, after running the above npm i command node_modules contains dirs for sharp and also the platform-specific transitive dependencies, @img/sharp-linux-x64, @img/sharp-libvips-linux-x64, etc.

Starting with npm@10.4.0, sharp is installed but @img/... is not, breaking our deployments.

Expected Behavior

I believe the npm@10.3.0 behaviour is correct, the platform-specific transitive deps should be installed.

Steps To Reproduce

1. Latest node, npm@10.4.0 or 10.5.0

2. With this config: none needed

3. Run:

mkdir temp; cd temp
npm i --cpu=x64 --os=linux --include=optional sharp@0.33.3

5. Note that node_modules/@img is an empty directory.

Environment

• npm: 10.4.0, 10.5.0, 10.5.2
• Node.js: 21.7.2
• OS Name: macOS Sonoma 14.4.1
• System Model Name: Mac Mini M1
• npm config:

; "builtin" config from /opt/homebrew/lib/node_modules/npm/npmrc

prefix = "/opt/homebrew" 

; "user" config from /Users/xyz/.npmrc

//registry.npmjs.org/:_authToken = (protected) 

; node bin location = /opt/homebrew/Cellar/node/21.7.3/bin/node
; node version = v21.7.3
; npm version = 10.3.0
; HOME = /Users/xyz
; Run `npm config ls -l` to show all defaults.

[bhallstein]

Further, tested on ubuntu as well, where cross-compilation continues to work on 10.5.0:

Mac:

Desktop:$ mkdir temp; cd temp
temp:$ node -p "[process.arch,process.platform]"
[ 'arm64', 'darwin' ]
temp:$ npm -v
10.5.0
temp:$ npm i --cpu=x64 --os=linux --include=optional sharp@0.33.3
added 11 packages in 1s
temp:$ ls -l node_modules/\@img/
total 0

Ubuntu:

muffins@localhost:~$ mkdir temp; cd temp
muffins@localhost:~/temp$ npm -v
10.5.0
muffins@localhost:~/temp$ npm i --cpu=arm64 --os=darwin --include=optional sharp@0.33.3
added 13 packages in 9s
muffins@localhost:~/temp$ ls -l node_modules/@img/
total 8
drwxrwxr-x 3 muffins muffins 4096 Apr 23 17:31 sharp-darwin-arm64
drwxrwxr-x 3 muffins muffins 4096 Apr 23 17:31 sharp-libvips-darwin-arm64

[bhallstein]

After some more digging over on lovell/sharp#4080 we found that on 10.4.0+ --libc=glibc is needed for the platform-specific deps to be installed when cross-compiling sharp from mac to linux. 10.3.0 and older it works without this.

At this point I'm uncertain if this is a regression in npm, or intended behaviour. I looked at the release notes but couldn't immediately see a justification for the change. The behaviour may reside in https://github.com/npm/npm-install-checks

[wraithgar]

This is one of those tricky situations where a bug was fixed and some folks may have been relying on the broken behavior. The fix was in arborist. The minified packument didn't include the libc field from the manifest so npm didn't know not to install it.

npm is now currently behaving as intended.

[bhallstein]

Thanks for clarifying @wraithgar 🙌