Copy container images from docker hub to gardener GCR

[timebertt]

/kind enhancement

What would you like to be added:

Let's copy all docker hub images that the gardener project consumes to our gardener GCR and consume them from there.

To support this effort with automation, let's add a CI job that copies a configured set of container images based on a simple configuration file:

images:
- source: kubernetesui/dashboard
  destination: eu.gcr.io/gardener-project/3rd/kubernetesui/dashboard
  tags:
  - v2.2.0
  - v2.4.0
  - v2.5.1
- source: envoyproxy/envoy-distroless
  destination: eu.gcr.io/gardener-project/3rd/envoyproxy/envoy-distroless
  tags:
  - v1.24.1

For each container image, the configuration file specifies one or multiple tags that should be copied from the given source repository to the given destination repository.
The copied images should include all platforms/architectures supported by the source images.

Why is this needed:

The gardener project consumes multiple images from docker hub. This leads to problems because

• docker hub enforces strict rate limits for image pulls
• docker hub doesn't support image pulls over IPv6 (ref ☂️ [GEP-21] IPv6 Single-Stack Support in Local Gardener gardener#7051)

With our gardener GCR, we already have the necessary infrastructure to solve both problems. GCR has no rate limit and supports IPv6.

Previously, some container images were already copied to the gardener GCR manually.
This is not enough for switching all container images over to our GCR because

• push access to the GCR is limited: we don't want to bother individuals with the task of copying container images, e.g. when versions change or new images should be used
• manual actions are not reviewed or audited where development is happening (GitHub)
• manual actions are error-prone: e.g., a simple docker pull + docker tag + docker push only includes the architecture of the used machine

Instead, we want to have a structured and automated way of copying all necessary container images based on a central inventory that can be updated in a self-service PR workflow.

Alternatives considered:

We can't use the GCP Docker Hub Mirror (mirror.gcr.io) because it is not a pull-through cache but only caches frequently access images, see the documentation (ref gardener/gardener#7568 (comment)):

Container Registry adds frequently requested images to the cache so they are available for future requests.
[...]
However, there is no guarantee that a particular image will remain cached for an extended period of time. Only obtain cached images on mirror.gcr.io by configuring the Docker daemon.
[...]
It also periodically removes images that are no longer requested.

We don't want to add repository-specific jobs based on their image vectors (images.yaml) because

• images specified in the image vector need to be present in the destination registry for pull request jobs to succeed
  • however, a CI job should only copy the images once a PR got approved and merged
  • this introduces a chicken-egg problem for adding new features based on new images
  • having an out-of-band configuration (centralized in this repository) allows configuring the copying first and then adding the new feature based on that
• only a subset of gardener's repositories use prow (some are still using concourse): configuring decentralized jobs everywhere in different tools is a high effort and troublesome in maintenance

Steps:

• Add job image to copy images from DockerHub to GCR #626
• Added prow job to copy images #631
• Configure list of docker hub images to copy #636
• Added images for other gardener repositories #646
• [GEP-21] Adapt documentation to docker hub image copying, drop docker.io registry mirror gardener#7698
• switch images in image vector:
  • gardener/gardener: Switched from DockerHub to Gardener GCR gardener#7686
  • gardener/gardener-extension-provider-equinix-metal: Switched from DockerHub to Gardener GCR gardener-extension-provider-equinix-metal#251
  • gardener/gardener-extension-provider-openstack: Switched from DockerHub to Gardener GCR gardener-extension-provider-openstack#592
  • gardener/gardener-extension-networking-calico: Switch from DockerHub to Gardener GCR gardener-extension-networking-calico#254
  • gardener/gardener-extension-registry-cache: Switched from DockerHub to Gardener GCR gardener-extension-registry-cache#17

[timebertt]

/assign

[timebertt]

Once this is done, the g/g documentation touched by gardener/gardener#7460 needs to be adapted accordingly.

[ccwienk]

OCI-Images from Gardener-BoM are already replicated to eu.gcr.io/sap-se-gcr-k8s-public, with a suffix consisting of mangled original registry + repository-name.

[timebertt]

Unfortunately, copying after releasing/merging doesn't suffice as written above:

We don't want to add repository-specific jobs based on their image vectors (images.yaml) because

• images specified in the image vector need to be present in the destination registry for pull request jobs to succeed

  • however, a CI job should only copy the images once a PR got approved and merged
  • this introduces a chicken-egg problem for adding new features based on new images
  • having an out-of-band configuration (centralized in this repository) allows configuring the copying first and then adding the new feature based on that

[timebertt]

All tasks have been completed. Well done @robinschneider!
/close

[gardener-prow]

@timebertt: Closing this issue.

In response to this:

All tasks have been completed. Well done @robinschneider!
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.