Is it safe for Dependabot to modify lock files directly?

[tylergaw]

This may not be the correct place for this question, but wasn't sure where else it should go.

I'm seeing open PRs for the first time from Dependabot on one project. It has two PRs:

• https://github.com/streetcredlabs/categories/pull/41
• https://github.com/streetcredlabs/categories/pull/44

In both of those, the only changes are in yarn.lock. In both, it's updating versions of transitive dependencies. Not dependencies directly added to the project. In everything I've read about yarn and npm lock files, it says they should not be edited directly. Including the first line of yarn.lock itself:

# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.

My question is, is it safe for Dependabot to do this? We're not seeing anything adverse from the two mentioned PRs, but editing yarn.lock directly is something we never do anywhere else so it's raising concerns.

[hmarr]

We typically don’t modify lockfiles directly - we use the package manager CLI or internals to write the updated lockfile after updating a dependency.

In these cases, I think we’ve updated the transitive dependencies to fix known vulnerabilities.

I’ll close this for now but let us know if you spot anything unexpected!

[gabrielliwerant]

I'm still seeing dependabot open PRs where the only file modifications are the lockfiles. I do not believe dependabot should be doing this, as it likely to cause issues/get out of sync with actual dependencies.

[jurre]

I'm still seeing dependabot open PRs where the only file modifications are the lockfiles. I do not believe dependabot should be doing this, as it likely to cause issues/get out of sync with actual dependencies.

I'm not sure I follow re: "get out of sync with actual dependencies", the lockfile is what determines which dependencies will be installed right?

Depending on how dependencies are set up, updating only the lockfile is often entirely valid.

Are you maybe vendoring dependencies? That's something that's not (yet) supported in all ecosystems, it's possible to set up an action or some other CI job that does the vendoring step for you and commit the results back to the branch.

If it's something else, if you could link to an example PR that'd be helpful.

[JulianKingman]

I'm wondering this as well.

dependabot updates second-hand dependencies, for example dependency2 below

package.json

"dependencies": {
  "dependency1": "1.0.0"
}

yarn.lock

dependency1@^1.0.0:
  version "1.0.0",
  dependencies:
    dependency2 "0.0.1" // <-- dependabot updates this

[jurre]

@JulianKingman could you link to a PR or share a setup we can use to reproduce what you're seeing?

[gabrielliwerant]

I'm still seeing dependabot open PRs where the only file modifications are the lockfiles. I do not believe dependabot should be doing this, as it likely to cause issues/get out of sync with actual dependencies.

I'm not sure I follow re: "get out of sync with actual dependencies", the lockfile is what determines which dependencies will be installed right?

Depending on how dependencies are set up, updating only the lockfile is often entirely valid.

Are you maybe vendoring dependencies? That's something that's not (yet) supported in all ecosystems, it's possible to set up an action or some other CI job that does the vendoring step for you and commit the results back to the branch.

If it's something else, if you could link to an example PR that'd be helpful.

When updating an indirect dependency in the lockfile, i.e. not a direct dependency declared in the package file, there is no guarantee that the direct packages that rely on it will work correctly. Typically, the indirect dependencies get newer versions as the primary dependencies update to require them. If package A is a primary dependency and requires package B, and there's a security vulnerability with package B, then package A should be updated to a new version that includes the updated package B. Doing this the other way around means that

1. Package A could break
2. Package B version might be overwritten on the next install if it is out of range for Package A

Tracing through the dependencies to see what package A is from the dependabot alert for package B is a lot of work and is error prone, but seems to be necessary in order to get a guaranteed stable update. It would be more useful if dependabot did this effort as well, instead of crawling only the lockfile. Also, as the original poster mentioned, my understanding is that updating lockfiles directly is not recommended for the reasons I stated.

[AndyWright-ct]

I am interested in understanding this too

[djbobbydrake]

Seems like this should still be an open conversation? Dependabot is still modifying lock files only.

[Smmaca]

I would like an answer to this too. It was my understanding that configuring dependabot to only allow "direct" dependency updates would prevent lockfile-only updates but it hasn't.

[deivid-rodriguez]

@Smmaca I think that's correct, except for security updates where Dependabot will prioritize security over everything else and always offer a PR if possible, regardless of configuration.

In the Javascript ecosystem, this is essentially what npm audit fix does (or should do, anyways).

[snajera-livefront]

We're also seeing this, and would also appreciate an answer. gabrielliwerant's concern is still unanswered, and is a completely valid concern.

To me, updating the lock file directly defeats the purpose of having the libraries manage their own dependencies, and is concerning to be out of sync with what the library expects to exist.

Note: only the lock file is updated

[deivid-rodriguez]

I no longer maintain Dependabot but I don't understand the concerns raised.

When updating an indirect dependency in the lockfile, i.e. not a direct dependency declared in the package file, there is no guarantee that the direct packages that rely on it will work correctly. Typically, the indirect dependencies get newer versions as the primary dependencies update to require them. If package A is a primary dependency and requires package B, and there's a security vulnerability with package B, then package A should be updated to a new version that includes the updated package B. Doing this the other way around means that

1. Package A could break
2. Package B version might be overwritten on the next install if it is out of range for Package A

Generally security updates should never break anything, but rather fix things. Dependabot will only update an indirect dependency (in your example, package B) if the new version satisfies the requirement the dependant package has set on it (package A in your example). For example, if package A declares package B @ ^1.0.0 as a dependency, Dependabot will update the lockfile to use a security release named 1.0.1, but will not update the lockfile to use a security release named 2.0.1, because that would break things.

Also, for what it's worth, Dependabot does not modify lockfiles directly, but uses package managers to do it. For example, for NPM it runs the following command:

dependabot-core/npm_and_yarn/lib/dependabot/npm_and_yarn/native_helpers.rb

Lines 18 to 46 in 1121a33

	def self.run_npm8_subdependency_update_command(dependency_names)
	# NOTE: npm options
	# - `--force` ignores checks for platform (os, cpu) and engines
	# - `--dry-run=false` the updater sets a global .npmrc with dry-run: true to
	# work around an issue in npm 6, we don't want that here
	# - `--ignore-scripts` disables prepare and prepack scripts which are run
	# when installing git dependencies
	command = [
	"update",
	*dependency_names,
	"--force",
	"--dry-run",
	"false",
	"--ignore-scripts",
	"--package-lock-only"
	].join(" ")
	fingerprint = [
	"update",
	"<dependency_names>",
	"--force",
	"--dry-run",
	"false",
	"--ignore-scripts",
	"--package-lock-only"
	].join(" ")
	Helpers.run_npm_command(command, fingerprint: fingerprint)
	end

. And similarly with other package managers. Sometimes it needs to do "smarter" things like temporary updating the manifest to force the update since some package managers don't fully support upgrading subdependencies, but it never updates lockfiles manually as far as I recall.