feat(dockerhub-mirror) set up a new dedicated ACR to mirror DockerHub inside the Jenkins Azure infrastructure

[dduportal]

Related to jenkins-infra/helpdesk#4192

Fixup of 91cf2dc

Reference Azure documentation: https://learn.microsoft.com/en-us/azure/container-registry/container-registry-artifact-cache?pivots=development-environment-azure-portal

This PR introduces an Azure Container Registry set up as a DockerHub mirror using a "Cache Rule" which mirrors docker.io/* to * (note: it forbids us to use other caching mechanism!).

This registry has the following properties:

• Only available in the "sponsorship" subscription
• Anonymous pull access (constraint due to Docker pull through cache - Docker is not passing auth informations when pulling from a mirror registry moby/moby#30880)
• Private network only: since we have anonymous pull policy (see above), then we restrict to only a subset of private networks. It uses "Azure Private Endpoints" for this
  • Note: it implies using Private DNS zones linked to networks. These zone might need to be reused in the future for other private link if required

The registry is available for the following (heavy DockerHub users) services (I've only setup the Azure ephemeral VM agents subnets for now) through a combination of (private endpoint with a NIC in the subnet + private DNS zone with automatic records + inbound and outbound NSG rules):

• ci.jenkins.io
• cert.ci.jenkins.io
• trusted.jenkins.io
• infra.jenkins.io

Azure makes it mandatory to log-in on DockerHub for such a mirror system. As such, we use a distinct token stored in an Azure Keyvault which is "Public Images Read Only" associated to the jenkinsciinfra organization to avoid the "application" rate limit (e.g. 5k pull / day / IP) and only have the DockerHub anti-abuse system as upper limit (which seems to be a combination of requests and amount of data).

---

Testing and approving

This PR is expected to have no changes in the plan as it was applied manually:

• End to end testing was done on each controller by:
  • Starting an Azure VM ephemeral agent using a pipeline replay with correct label
    • The pipeline tries to resolve the DNS name dockerhubmirror.azurecr.io and should resolve to an IP local to the VM subnet
  • Once the VM is up, checking the connectivity in Azure UI portal (Network Watcher -> Connection troubleshoot)
    • Source VM is the agent VM, which name is retrieved from build log
    • Destination is https://dockerhubmirror.azurecr.io

• The bootstrap must be done in 2 terraform apply commands as documented, because the ACR component CredentialSet is not supported by Terraform yet (see comments in TF code).

[dduportal]

Merging to fix the Terraform errors on main branch and avoid accidental roll backs.

@timja do not hesitate to review: I'll fix any feedbacks!