Skip to content

Commit

Permalink
integrate with renovate (#358)
Browse files Browse the repository at this point in the history
* Add renovate

* Migrate config renovate.json

* Do not match on type

* hmm

* Add solr

* Make sha update script more generic

* Add composer

* Add solr-ocrhighlighting

* fixup

* Use advanced capture for github releases

* Update matchPackageNames

* Use advanced capture for apache repos

* Add cantaloupe

* Add FITS

* Add blazegraph log4j

* Add some docs around renovate

* matchDepNames

* match all

* Remove double quotes around version

* Fix semver for some repos and better multi file support

* put blazegraph log4j hash back

* Update README.md

* Add renovate validation action

* Also update version in README

* fix typo

* Explicitly enable managers we want instead of disabling ones renovate auto-detected
  • Loading branch information
joecorall authored Jan 2, 2025
1 parent 9091aa4 commit 61477fc
Show file tree
Hide file tree
Showing 14 changed files with 282 additions and 10 deletions.
21 changes: 21 additions & 0 deletions .github/workflows/validate-renovate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: validate renovate.json

on:
pull_request:

env:
LOG_LEVEL: debug

jobs:
renovate-config-validator:
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4

- uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4
with:
node-version: 20

- run: npx -p renovate renovate-config-validator renovate.json
37 changes: 37 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
- [Docker Images](#docker-images)
- [Updating Dependencies](#updating-dependencies)
- [Updating Composer](#updating-composer)
- [Renovate](#renovate)
- [Updating Configuration](#updating-configuration)
- [Design Considerations](#design-considerations)
- [Confd](#confd)
Expand Down Expand Up @@ -518,6 +519,42 @@ shasum -a 256 ${ALPACA_FILE}
> N.B. Please read the release notes the new version and account for any changes
> to configuration that are required, as well as test locally.
#### Renovate

Several dependencies in this repo can be automatically updating using [renovate](https://www.mend.io/renovate/).

Currently these docker images have some depenencies managed by renovate:

```
activemq
blazegraph
cantaloupe
code-server
fcrepo6
fits
nginx
solr
tomcat
```

Since renovate does not natively support the ability to extract a sha256 from a file, we need [a custom shell script](./ci/update-sha.sh) in the [postUpgradeTasks](https://docs.renovatebot.com/configuration-options/#postupgradetasks) to calculate the sha256 of our files and update our Dockerfile accordingly.

Post upgrade tasks can only run on self-hosted Renovate instances, so this forces us to run renovate on a properly configured runner (instead of using mend.io's free GitHub app to manage our dependencies). Getting renovate setup locally looks like

```
npm install -g renovate
export RENOVATE_REPOSITORIES=islandora-devops/isle-buildkit
export RENOVATE_TOKEN=your-gh-pat
export RENOVATE_ALLOWED_POST_UPGRADE_COMMANDS='["bash ci/update-sha.sh \"{{{depName}}}\" \"{{{currentVersion}}}\" \"{{{newVersion}}}\""]'
# this will issue a PR with any updates to
# our docker images that have renovate configured properly
renovate --platform=github
```

Versions listed in GitHub tags or releases can use [advanced capture](https://docs.renovatebot.com/modules/manager/regex/#advanced-capture) in the Dockerfile to update the pinned version.


#### Updating Composer

A number of images like [crayfish] provide a `composer.lock` file to pin them to
Expand Down
3 changes: 2 additions & 1 deletion activemq/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
FROM java

ARG TARGETARCH
ARG ACTIVEMQ_VERSION="5.18.5"
# renovate: datasource=github-tags depName=apache-activemq packageName=apache/activemq
ARG ACTIVEMQ_VERSION=5.18.5
ARG ACTIVEMQ_FILE="apache-activemq-${ACTIVEMQ_VERSION}-bin.tar.gz"
ARG ACTIVEMQ_URL="https://archive.apache.org/dist/activemq/${ACTIVEMQ_VERSION}/${ACTIVEMQ_FILE}"
ARG ACTIVEMQ_FILE_SHA256="1190b809a35ceb5fb0273f4a47d4584684d75c43360942be10d9af1fc0a39394"
Expand Down
4 changes: 3 additions & 1 deletion blazegraph/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@ ARG BLAZEGRAPH_VERSION="CANDIDATE_2_1_5"
ARG BLAZEGRAPH_FILE="blazegraph.war"
ARG BLAZEGRAPH_URL="https://github.com/blazegraph/database/releases/download/BLAZEGRAPH_RELEASE_${BLAZEGRAPH_VERSION}/${BLAZEGRAPH_FILE}"
ARG BLAZEGRAPH_SHA256="b22f1a1aa8e536443db9a57da63720813374ef59e4021cfa9ad0e98f9a420e85"
ARG LOG4J_VERSION="2.22.0"

# renovate: datasource=github-releases depName=apache-log4j packageName=apache/logging-log4j2
ARG LOG4J_VERSION=2.22.0
ARG LOG4J_FILE="apache-log4j-${LOG4J_VERSION}-bin.zip"
ARG LOG4J_URL="https://archive.apache.org/dist/logging/log4j/${LOG4J_VERSION}/${LOG4J_FILE}"
ARG LOG4J_FILE_SHA256="c6d61ecf2563b1200e02587b89b7c75b58b6e62e6a16cdb6f333c2482167c2dc"
Expand Down
3 changes: 2 additions & 1 deletion cantaloupe/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
FROM java

ARG TARGETARCH
ARG CANTALOUPE_VERSION="5.0.6"
# renovate: datasource=github-releases depName=cantaloupe packageName=cantaloupe-project/cantaloupe
ARG CANTALOUPE_VERSION=5.0.6
ARG CANTALOUPE_FILE="cantaloupe-${CANTALOUPE_VERSION}.zip"
ARG CANTALOUPE_URL="https://github.com/cantaloupe-project/cantaloupe/releases/download/v${CANTALOUPE_VERSION}/${CANTALOUPE_FILE}"
ARG CANTALOUPE_SHA256="35311eb0d4d6f0578cab42fd5e51d6150e62821cb3b4ee3a265e2befbeeb5897"
Expand Down
116 changes: 116 additions & 0 deletions ci/update-sha.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
#!/usr/bin/env bash

set -eou pipefail

DEP=$1
OLD_VERSION=$2
NEW_VERSION=$3
URL=""
ARG=""
DOCKERFILES=()
README=""

echo "Updating SHA for $DEP@$NEW_VERSION"

if [ "$DEP" = "apache-tomcat" ]; then
URL="https://downloads.apache.org/tomcat/tomcat-9/v$NEW_VERSION/bin/apache-tomcat-$NEW_VERSION.tar.gz"
ARG="TOMCAT_FILE_SHA256"
DOCKERFILES=("tomcat/Dockerfile")
README="tomcat/README.md"

elif [ "$DEP" = "apache-activemq" ]; then
URL="https://downloads.apache.org/activemq/$NEW_VERSION/apache-activemq-$NEW_VERSION-bin.tar.gz"
ARG="ACTIVEMQ_FILE_SHA256"
DOCKERFILES=("activemq/Dockerfile")
README="activemq/README.md"

elif [ "$DEP" = "apache-solr" ]; then
URL="https://downloads.apache.org/solr/solr/$NEW_VERSION/solr-$NEW_VERSION.tgz"
ARG="SOLR_FILE_SHA256"
DOCKERFILES=("solr/Dockerfile")
README="solr/README.md"

elif [ "$DEP" = "custom-composer" ]; then
URL="https://getcomposer.org/download/${NEW_VERSION}/composer.phar"
ARG="COMPOSER_SHA256"
DOCKERFILES=("nginx/Dockerfile")

elif [ "$DEP" = "solr-ocrhighlighting" ]; then
URL=https://github.com/dbmdz/solr-ocrhighlighting/releases/download/${NEW_VERSION}/solr-ocrhighlighting-${NEW_VERSION}.jar
ARG="OCRHIGHLIGHT_FILE_SHA256"
DOCKERFILES=("solr/Dockerfile")

elif [ "$DEP" = "alpine-pkg-glibc" ]; then
URL="https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${NEW_VERSION}/glibc-${NEW_VERSION}.apk"
ARG="GLIBC_SHA256"
DOCKERFILES=("code-server/Dockerfile")

elif [ "$DEP" = "fcrepo" ]; then
URL="https://github.com/fcrepo/fcrepo/releases/download/fcrepo-${NEW_VERSION}/fcrepo-webapp-${NEW_VERSION}.war"
ARG="FCREPO_SHA256"
DOCKERFILES=("fcrepo6/Dockerfile")
README="fcrepo6/README.md"

elif [ "$DEP" = "islandora-syn" ]; then
URL="https://github.com/Islandora/Syn/releases/download/v${NEW_VERSION}/islandora-syn-${NEW_VERSION}-all.jar"
ARG="SYN_SHA256"
DOCKERFILES=("fcrepo6/Dockerfile")

elif [ "$DEP" = "fcrepo-import-export" ]; then
URL="https://github.com/fcrepo-exts/fcrepo-import-export/releases/download/fcrepo-import-export-${NEW_VERSION}/fcrepo-import-export-${NEW_VERSION}.jar"
ARG="IMPORT_EXPORT_SHA256"
DOCKERFILES=("fcrepo6/Dockerfile")

elif [ "$DEP" = "fcrepo-upgrade-utils" ]; then
URL="https://github.com/fcrepo-exts/fcrepo-upgrade-utils/releases/download/fcrepo-upgrade-utils-${NEW_VERSION}/fcrepo-upgrade-utils-${NEW_VERSION}.jar"
ARG="UPGRADE_UTILS_SHA256"
DOCKERFILES=("fcrepo6/Dockerfile")

elif [ "$DEP" = "cantaloupe" ]; then
URL="https://github.com/cantaloupe-project/cantaloupe/releases/download/v${NEW_VERSION}/cantaloupe-${NEW_VERSION}.zip"
ARG="CANTALOUPE_SHA256"
DOCKERFILES=("cantaloupe/Dockerfile")
README="cantaloupe/README.md"

elif [ "$DEP" = "fits-servlet" ]; then
URL="https://github.com/harvard-lts/FITSservlet/releases/download/${NEW_VERSION}/fits-service-${NEW_VERSION}.war"
ARG="FITSSERVLET_SHA256"
DOCKERFILES=("fits/Dockerfile")

elif [ "$DEP" = "fits" ]; then
URL="https://github.com/harvard-lts/fits/releases/download/${NEW_VERSION}/fits-${NEW_VERSION}.zip"
ARG="FITS_SHA256"
DOCKERFILES=("fits/Dockerfile")
README="fits/README.md"

elif [ "$DEP" = "apache-log4j" ]; then
URL="https://archive.apache.org/dist/logging/log4j/${NEW_VERSION}/apache-log4j-${NEW_VERSION}-bin.zip"
ARG="LOG4J_FILE_SHA256"
DOCKERFILES=(
"blazegraph/Dockerfile"
"fits/Dockerfile"
)

else
echo "DEP not found"
exit 0
fi

# update the Dockerfile(s) SHA256 with the file we're downloading
SHA=$(curl -s "$URL" \
| shasum -a 256 \
| awk '{print $1}')
if [[ "$OSTYPE" == "darwin"* ]]; then
sed -i '' 's|^ARG '"$ARG"'=.*|ARG '"$ARG"'="'"$SHA"'"|g' "${DOCKERFILES[@]}"
else
sed -i 's|^ARG '"$ARG"'=.*|ARG '"$ARG"'="'"$SHA"'"|g' "${DOCKERFILES[@]}"
fi

# update the README to specify the new version
if [ "$README" != "" ]; then
if [[ "$OSTYPE" == "darwin"* ]]; then
sed -i '' "s/${OLD_VERSION}\.$/${NEW_VERSION}\./" "$README"
else
sed -i "s/${OLD_VERSION}\.$/${NEW_VERSION}\./" "$README"
fi
fi
3 changes: 2 additions & 1 deletion code-server/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ FROM drupal

ARG TARGETARCH

ARG GLIBC_VERSION="2.35-r1"
# renovate: datasource=github-releases depName=alpine-pkg-glibc packageName=sgerrand/alpine-pkg-glibc
ARG GLIBC_VERSION=2.35-r1
ARG GLIBC_FILE="glibc-${GLIBC_VERSION}.apk"
ARG GLIBC_URL="https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/${GLIBC_FILE}"
ARG GLIBC_SHA256="276f43ce9b2d5878422bca94ca94e882a7eb263abe171d233ac037201ffcaf06"
Expand Down
7 changes: 6 additions & 1 deletion fcrepo6/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,26 @@
FROM tomcat

ARG TARGETARCH

# renovate: datasource=github-releases depName=fcrepo packageName=fcrepo/fcrepo
ARG FCREPO_VERSION=6.5.0
ARG FCREPO_FILE=fcrepo-webapp-${FCREPO_VERSION}.war
ARG FCREPO_URL=https://github.com/fcrepo/fcrepo/releases/download/fcrepo-${FCREPO_VERSION}/${FCREPO_FILE}
ARG FCREPO_SHA256=dd0ac889f253dbe9fb09aa3f1fe4444fd57b51af34ebaaf23daec72fbdc5cf55

# renovate: datasource=github-releases depName=islandora-syn packageName=Islandora/Syn
ARG SYN_VERSION=1.1.1
ARG SYN_FILE=islandora-syn-${SYN_VERSION}-all.jar
ARG SYN_URL=https://github.com/Islandora-CLAW/Syn/releases/download/v${SYN_VERSION}/${SYN_FILE}
ARG SYN_URL=https://github.com/Islandora/Syn/releases/download/v${SYN_VERSION}/${SYN_FILE}
ARG SYN_SHA256=24c589d16d579c48361d14c346e0d633a6985642f1c116295ed91eb32b283085

# renovate: datasource=github-releases depName=fcrepo-import-export packageName=fcrepo-exts/fcrepo-import-export
ARG IMPORT_EXPORT_VERSION=1.2.0
ARG IMPORT_EXPORT_FILE=fcrepo-import-export-${IMPORT_EXPORT_VERSION}.jar
ARG IMPORT_EXPORT_URL=https://github.com/fcrepo-exts/fcrepo-import-export/releases/download/fcrepo-import-export-${IMPORT_EXPORT_VERSION}/${IMPORT_EXPORT_FILE}
ARG IMPORT_EXPORT_SHA256=e767f43b0b4f639042b0e399ebfebdf61957515d585b7f0d2d8aeb9f877b12a0

# renovate: datasource=github-releases depName=fcrepo-upgrade-utils packageName=fcrepo-exts/fcrepo-upgrade-utils
ARG UPGRADE_UTILS_VERSION=6.3.0
ARG UPGRADE_UTILS_FILE=fcrepo-upgrade-utils-${UPGRADE_UTILS_VERSION}.jar
ARG UPGRADE_UTILS_URL=https://github.com/fcrepo-exts/fcrepo-upgrade-utils/releases/download/fcrepo-upgrade-utils-${UPGRADE_UTILS_VERSION}/${UPGRADE_UTILS_FILE}
Expand Down
2 changes: 1 addition & 1 deletion fcrepo6/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Fcrepo

Docker image for [fcrepo] version 6.5.0
Docker image for [fcrepo] version 6.5.0.

Please refer to the [Fcrepo Documentation] for more in-depth information.

Expand Down
10 changes: 7 additions & 3 deletions fits/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,21 @@
FROM tomcat

ARG TARGETARCH
ARG FITSSERVLET_VERSION="2.1.0"

# renovate: datasource=github-releases depName=fits-servlet packageName=harvard-lts/FITSservlet
ARG FITSSERVLET_VERSION=2.1.0
ARG FITSSERVLET_FILE="fits-service-${FITSSERVLET_VERSION}.war"
ARG FITSSERVLET_URL="https://github.com/harvard-lts/FITSservlet/releases/download/${FITSSERVLET_VERSION}/${FITSSERVLET_FILE}"
ARG FITSSERVLET_SHA256="ba3c4d5af4dfbe42a32b4321eb96fad8db1379c8b00cd2bc7b7b0e4f59cb7e67"

ARG FITS_VERSION="1.6.0"
# renovate: datasource=github-releases depName=fits packageName=harvard-lts/fits
ARG FITS_VERSION=1.6.0
ARG FITS_FILE="fits-${FITS_VERSION}.zip"
ARG FITS_URL="https://github.com/harvard-lts/fits/releases/download/${FITS_VERSION}/${FITS_FILE}"
ARG FITS_SHA256="32e436effe7251c5b067ec3f02321d5baf4944b3f0d1010fb8ec42039d9e3b73"

ARG LOG4J_VERSION="2.23.1"
# renovate: datasource=github-releases depName=apache-log4j packageName=apache/logging-log4j2
ARG LOG4J_VERSION=2.23.1
ARG LOG4J_FILE="apache-log4j-${LOG4J_VERSION}-bin.zip"
ARG LOG4J_URL="https://archive.apache.org/dist/logging/log4j/${LOG4J_VERSION}/${LOG4J_FILE}"
ARG LOG4J_FILE_SHA256="c53645414ce68f21dd347439df5852ad16b38cd7227a9d417ebaf411a1c09501"
Expand Down
1 change: 1 addition & 0 deletions nginx/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
FROM base

ARG TARGETARCH
# renovate: datasource=github-releases depName=custom-composer packageName=composer/composer
ARG COMPOSER_VERSION=2.8.2
ARG COMPOSER_FILE=composer.phar
ARG COMPOSER_URL=https://getcomposer.org/download/${COMPOSER_VERSION}/${COMPOSER_FILE}
Expand Down
80 changes: 80 additions & 0 deletions renovate.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
{
"extends": [
"config:best-practices",
":rebaseStalePrs"
],
"enabledManagers": [
"custom.regex"
],
"packageRules": [
{
"matchPackageNames": [
"apache/activemq",
"apache/logging-log4j2",
"fcrepo/fcrepo",
"fcrepo-exts/fcrepo-import-export",
"fcrepo-exts/fcrepo-upgrade-utils"
],
"extractVersion": "^(fcrepo|activemq|rel|fcrepo-import-export|fcrepo-upgrade-utils)(\/|-)(?<version>\\d+\\.\\d+\\.\\d+)$",
"versioning": "semver"
},
{
"packagePatterns": ["^apache/solr$"],
"extractVersion": "^releases/solr/(?<version>\\d+\\.\\d+\\.\\d+)$",
"versioning": "semver"
},
{
"matchUpdateTypes": [
"major"
],
"enabled": false
},
{
"groupName": "all non-major dependencies",
"matchUpdateTypes": [
"minor",
"patch",
"digest"
],
"matchDepNames": [
"alpine-pkg-glibc",
"apache-activemq",
"apache-log4j",
"apache-solr",
"apache-tomcat",
"cantaloupe",
"custom-composer",
"fcrepo",
"fcrepo-import-export",
"fcrepo-upgrade-utils",
"fits",
"fits-servlet",
"islandora-syn",
"solr-ocrhighlighting"
],
"postUpgradeTasks": {
"commands": [
"bash ci/update-sha.sh \"{{{depName}}}\" \"{{{currentVersion}}}\" \"{{{newVersion}}}\""
]
}
}
],
"customManagers": [
{
"customType": "regex",
"description": "Update _VERSION variables in Dockerfiles",
"fileMatch": ["(^|/|\\.)Dockerfile$", "(^|/)Dockerfile\\.[^/]*$"],
"matchStrings": [
"# renovate: datasource=(?<datasource>[a-z-]+?)(?: depName=(?<depName>.+?))? packageName=(?<packageName>.+?)(?: versioning=(?<versioning>[a-z-]+?))?\\s(?:ENV|ARG) .+?_VERSION=(?<currentValue>.+?)(\\s|$)"
]
}
],
"labels": [
"dependencies"
],
"osvVulnerabilityAlerts": true,
"dependencyDashboardOSVVulnerabilitySummary": "unresolved",
"vulnerabilityAlerts": {
"enabled": true
}
}
2 changes: 2 additions & 0 deletions solr/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@
FROM java

ARG TARGETARCH
# renovate: datasource=github-tags depName=apache-solr packageName=apache/solr
ARG SOLR_VERSION=9.5.0
ARG SOLR_FILE=solr-${SOLR_VERSION}.tgz
ARG SOLR_URL=https://archive.apache.org/dist/solr/solr/${SOLR_VERSION}/solr-${SOLR_VERSION}.tgz
ARG SOLR_FILE_SHA256=d8538502019af1945e0b124a4613b46ca43aedcf3f20e9912c482c080407ea21
# renovate: datasource=github-releases depName=solr-ocrhighlighting packageName=dbmdz/solr-ocrhighlighting
ARG OCRHIGHLIGHT_VERSION=0.9.0
ARG OCRHIGHLIGHT_FILE=solr-ocrhighlighting-${OCRHIGHLIGHT_VERSION}.jar
ARG OCRHIGHLIGHT_URL=https://github.com/dbmdz/solr-ocrhighlighting/releases/download/${OCRHIGHLIGHT_VERSION}/solr-ocrhighlighting-${OCRHIGHLIGHT_VERSION}.jar
Expand Down
Loading

0 comments on commit 61477fc

Please sign in to comment.