Support yarn.lock maintenance

[rarkins]

This PR will close #96

It regenerates each yarn.lock file it finds and raises a PR if differences are found.

Still to do:

• Support templates for branch/PR strings?
• Decide how to handle monorepos

[codecov-io]

Codecov Report

Merging #105 into master will decrease coverage by 1.98%.
The diff coverage is 3.7%.

@@            Coverage Diff             @@
##           master     #105      +/-   ##
==========================================
- Coverage   69.92%   67.93%   -1.99%     
==========================================
  Files          19       19              
  Lines         838      864      +26     
  Branches      140      143       +3     
==========================================
+ Hits          586      587       +1     
- Misses        252      277      +25

Impacted Files	Coverage Δ
lib/config/definitions.js	100% <ø> (ø)	⬆️
lib/helpers/yarn.js	27.27% <0%> (-4.31%)	⬇️
lib/worker.js	53.26% <4.34%> (-16.31%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a4cb285...f6727c1. Read the comment docs.

[rarkins]

@hbetts this works when there's one yarn.lock per repository but does not support monorepo yet.

I think there are two main options:

1. Support single branch/PR that updates all yarn.lock files
2. Separate branch/PR per yarn.lock file

This is also actually a problem for regular renovate usage too, e.g. if the same dependency needs updating in two different package.json files in the same (mono)repository.

Given that package file paths are incompatible with branch naming, we need another approach, such as counters. Example: renovate/yarn-lock-1, renovate/yarn-lock-2, etc.

However, the approach should be deterministic so that we can be sure we get the same counter for a package file every time it runs. Users might add or remove package.json / yarn.lock files so we can't be sure that counters would remain the same forever, although this potentially isn't an issue if renovate can "heal" PRs that were open prior to this.

An alternative to counters would be to use a short hash of the package.json path, which would ensure that it is the same forever, even if other package.json files are added or removed.

Unfortunately all the above options make the branch/pr naming less human-understandable. Perhaps best to support a single branch/PR for yarn.lock maintenance to begin with? It will still need some improved logic to this current code I've pushed.

[rarkins]

@jonbretman do you have any requirements or opinions on this feature?

[hutson]

does not support monorepo yet.

I was under the impression that renovate supports monorepos, but that it supports monorepos in the way you described in bullet point 1 above. I don't personally maintain any monorepos, so I haven't looked closely at what support exists in the renovate code base.

I see the allure of option 2, but I know some projects, such as babel release in lock-step. Accepting a yarn.lock file into a single package within their monorepo might actually break their build (not entirely positive that's the case, but again, my monorepo experience is limited.).

Coincidentally, babel integrated yarn into their build 23 days ago - babel/babel#5175

Oddly they have a single global yarn.lock file, not per-package yarn.lock files. It looks like their development dependencies are in a global package.json file, while their runtime dependencies are in their individual packages under packages/. Therefore their only locking their development dependencies.

[hutson]

Given that package file paths are incompatible with branch naming

I was going to ask if you could elaborate on that statement, since I was under the impression that you could use full paths (Though I don't imagine full paths would be easy to read in a branch name).

However, it seems a git branch name can't end with .lock, which is quite unfortunate in this situation.

An alternative to counters would be to use a short hash of the package.json path

I agree that a hash, while a novel approach, would not create a good user experience. A hash is not only not readable, but from the branch name alone I can't tell what package within the monorepo is being targeted with an update.

Actually, coming back to the branch name, we don't need the branch name to end with yarn.lock. We already know the purpose of the branch is to update a yarn.lock file, so just the path to the package should be sufficient.

[hutson]

I guess what I'm getting at is perhaps we should engage the babel team:

• Determine whether they would benefit from a yarn.lock file in each package of their monorepo.
• What the pain points would be.
• Whether renovate could help mitigate, or eliminate, those pain points.
• What requirements would be asked of renovate.

[rarkins]

I was under the impression that renovate supports monorepos, but that it supports monorepos in the way you described in bullet point 1 above.

It does.. in fact my main repo has three package.json files that are auto-discovered and renovated. But branches are still named based on the dependency name, so if the same dependency is in multiple package.json files then the branch will have multiple commits to different files, which has the potential to be undesirable. I also haven't fully thought out what should happen if that branch becomes unmergeable - I suspect it's possible it will require two executions of renovate to recreate/replace the PR.

[rarkins]

Re: babel, I note that they use lerna and wonder if a more generic solution for that would be better than specifically focusing on babel.

[rarkins]

I guess that "putting all yarn.lock updates in the same branch" is a reasonable first step to merge though and would satisfy all single-repo users plus perhaps most monorepo. If they are done as separate commits then it also gives users the option to cherry-pick them too.

[rarkins]

Also, once this feature is done we could add a configuration to disable changes to package.json so that people using yarn with ranges in package.json could use renovate only for keeping yarn.lock updated.

[hutson]

Re: babel, I note that they use lerna and wonder if a more generic solution for that would be better than specifically focusing on babel.

larna actually evolved out of the scripts used by babel to maintain their monorepo. So providing a working solution for lerna-maintained monorepos would seem like a good approach.

[ghost]

Could this be added to the default definitions so that it can be configured by the project or renovate user? (We use Angular commit conventions for all our commits to help with other automation tooling)

[rarkins]

Yes, good idea. I'm a little concerned about the "scalability" of too many message definitions, however that's not an excuse for making them hardcoded!

[ghost]

@rarkins is this pull request in a state where I can begin testing it?

[rarkins]

@destroyerofbuilds this is now ready for testing. I have made the four strings in use (branch name, commit message, pr title, pr body) all configurable although didn't personally test they were working. Otherwise I've tested it on a repo with a single yarn.lock file and seems to work well. The only "quirk" can happen in situations like this:

• One dependency is out of date: renovate raises PR
• A subdependency of that dependency is also out of date: renovate raises yarn maintenance PR
• Merge dependency update PR, which results in the subdependency being updated
• Yarn.lock is no longer needing renovation
• Renovate will update the yarn maintenance PR, but now it's got 0 diff

It's harmless - the PR can be closed without merge, or left around until a day or week later when it does have a diff. It would be a nice-to-have feature to detect when an existing yarn lock maintenance PR can be closed automatically though - perhaps a future issue.

Also as noted in the documentation, this is only known to work when a repo has a single yarn.lock file. Some refactoring of renovate will be required to safely support multiple yarn lock file maintenance in the same repo.

[ghost]

I just realized this option is blocked on the command line.

Kept wondering why I couldn't specify it.

Would/could this be opened up so that it's available to be specified on the command line?