[BUG] Can't publish using scoped private registry from parent directory

[internalsystemerror]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

After upgrading from Node v14 (and npm@6) to Node v16 (and npm@8), we're no longer able to use npm with a scoped private gitlab repository.

Note there are some issues which may be related to this, however the descriptions and error results differ enough that I felt it necessary to file this new issue.

Expected Behavior

Continue to function as before.

Steps To Reproduce

Build a package to dist/admin with the name containing a scope "@vendorname/packagename" and the "publishConfig" key containing a "@vendorname:registry" key. Set the appropriate npm config settings to allow publishing on node v14 and npm@6.

Update to node v16 and npm@8, this no longer works giving the following error:

npm ERR! code 128
npm ERR! An unknown git error occurred
npm ERR! command git --no-replace-objects ls-remote ssh://git@github.com/dist/admin.git
npm ERR! ssh -oStrictHostKeyChecking=accept-new: line 1: ssh: not found
npm ERR! fatal: Could not read from remote repository.
npm ERR! 
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2021-11-04T11_17_19_459Z-debug.log
node:child_process:903
    throw err;
    ^
Error: Command failed: npm publish --non-interactive dist/adminnpm ERR! code 128
npm ERR! An unknown git error occurred
npm ERR! command git --no-replace-objects ls-remote ssh://git@github.com/dist/admin.git
npm ERR! ssh -oStrictHostKeyChecking=accept-new: line 1: ssh: not found
npm ERR! fatal: Could not read from remote repository.
npm ERR! 
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2021-11-04T11_17_19_459Z-debug.log
node:child_process:903
    throw err;
    ^
Error: Command failed: npm publish --non-interactive dist/admin

As stated, the private registry is on gitlab so I'm not sure why it's trying to contact github. When running whoami, the following error occurs:

npm whoami --registry=https://gitlab.com/api/v4/packages/npm/
npm ERR! code E401
npm ERR! 401 Unauthorized - GET https://registry.npmjs.org/-/whoami - Unauthorized

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/ise/.npm/_logs/2021-11-04T13_22_08_098Z-debug.log

Environment

• OS: Multiple (Ubuntu 20.04 + Alpine 3.14)
• Node: 16.13.0
• npm: 8.1.0

[internalsystemerror]

Further to this, doing cd dist/admin && npm publish --non-interactive instead appears to at last find the package, but again when trying to publish tells me I'm unauthorised with npm's registry. This appears to be ignoring publishConfig for a scoped repository. npm@6 works fine.

EDIT: This was a local auth token issue (my local token didn't have write permissions). Changing to the package directory before running publish actually works. So the issue is specifically when trying to publish a subdirectory of the current working directory.

[GasimGasimzada]

This happened to us as well and the following line got me curious git --no-replace-objects ls-remote ssh://git@github.com/dist/admin.git. Why dist/admin.git? From my understanding, there is a bug in npm8 where using slashes will result in the path being treated as a GIT repo. Doing cd dist/ && npm publish admin works as well

[GasimGasimzada]

I did a bit more digging and found the actual problem:

NPM CLI uses npm-package-arg to find out what the passed argument is. (Package: https://www.npmjs.com/package/npm-package-arg). So, I installed the package locally to do some tests and found the following results:

console.log(npa('dist'));

/*
Output: 

Result {
  type: 'tag',
  registry: true,
  where: undefined,
  raw: 'dist',
  name: 'dist',
  escapedName: 'dist',
  scope: undefined,
  rawSpec: '',
  saveSpec: null,
  fetchSpec: 'latest',
  gitRange: undefined,
  gitCommittish: undefined,
  hosted: undefined
}
*/

console.log(npa('dist/admin'));

/*
Output:
Result {
  type: 'git',
  registry: undefined,
  where: undefined,
  raw: 'dist/admin',
  name: undefined,
  escapedName: undefined,
  scope: undefined,
  rawSpec: 'dist/admin',
  saveSpec: 'github:dist/admin',
  fetchSpec: null,
  gitRange: undefined,
  gitCommittish: null,
  hosted: GitHost {
    sshtemplate: [Function: sshtemplate],
    sshurltemplate: [Function: sshurltemplate],
    browsetemplate: [Function: browsetemplate],
    browsefiletemplate: [Function: browsefiletemplate],
    docstemplate: [Function: docstemplate],
    httpstemplate: [Function: httpstemplate],
    filetemplate: [Function: filetemplate],
    shortcuttemplate: [Function: shortcuttemplate],
    pathtemplate: [Function: pathtemplate],
    bugstemplate: [Function: bugstemplate],
    hashformat: [Function: formatHashFragment],
    protocols: [ 'git:', 'http:', 'git+ssh:', 'git+https:', 'ssh:', 'https:' ],
    domain: 'github.com',
    treepath: 'tree',
    gittemplate: [Function: gittemplate],
    tarballtemplate: [Function: tarballtemplate],
    extract: [Function: extract],
    type: 'github',
    user: 'dist',
    auth: null,
    project: 'admin',
    committish: null,
    default: 'shortcut',
    opts: { noGitPlus: true, noCommittish: true }
  }
}
*/

As you can see, using npa returns returns GIT SSH result when subpaths are passed to it. If you add a trailing "/" at the end of your path, it should fix the issue:

console.log(npa('dist/admin/'));

/*
Result {
  type: 'directory',
  registry: undefined,
  where: '[PATH_TO_REPO]',
  raw: 'dist/admin/',
  name: undefined,
  escapedName: undefined,
  scope: undefined,
  rawSpec: 'dist/admin/',
  saveSpec: 'file:dist\\admin',
  fetchSpec: '[PATH_TO_REPO]\\dist\\admin',
  gitRange: undefined,
  gitCommittish: undefined,
  hosted: undefined
}
*/

[internalsystemerror]

@GasimGasimzada Well found! Thanks. I'll see if we can test adding a trailing slash as an alternate workaround.

[PowerKiKi]

Adding a trailing "/" at the end of the path solved it for us, thank you !

[oliverlockwood]

In case it helps, I suspect this issue might be a combination of:

• [BUG] npm v7 cannot publish to private registry (v6 works) #3010 (npm breaking change: we now need to prefix the path with ./) and
• [BUG] npm 7.18.1 ignores publishConfig.registry #3573 (npm regression: npm inexplicably ignores the publishConfig.registry within package.json files)

[hellwolf]

... I mean, okay, but why... thanks anyways, a head scratching for a few hours

[jpzwarte]

We ran into this as well; trailing slash fixed it.

[wraithgar]

Yes, the shorthand of x/y translating to a git url is what's happening here. The trailing slash is required to let npm know it's a path, not the shorthand.

[boutell]

My script is cd'ing into the appropriate folder and running npm publish the simple way, and I still get this error since upgrading to npm 8, even after a fresh npm login. It's a scoped private repo, as in these examples, but I've cd'd down into the actual directory. Not sure what I can do with that, but let me see what happens if I go out of my way not to cd into the folder... alas, no dice.

[zedfalco]

Thanks a lot @GasimGasimzada