Strange behavior when downloading kubeadm from dl.k8s.io

[sebypp]

Hello everyone,

Background information: I'm trying to create a Kubernetes cluster using kubespray and some VMs from Hetzner. The setup is pretty simple, I have 3 control and 3 worker nodes, each with public IP addresses (so nothing fancy).

Problem statement: when kubespray is trying to download kubeadm from dl.k8s.io, 5 hosts work, but for the 6th one I get HTTP Error 403: Forbidden. This might indicate blacklisting is configured on dl.k8s.io, which seems a bit strange, considering Hetzner is a reputable cloud vendor. Full evidence below

TASK [kubernetes_sigs.kubespray.download : Download_file | Download item] ***********************************************************************************************************************************************
ok: [controller03]
ok: [controller02]
ok: [controller01]
FAILED - RETRYING: [node02]: Download_file | Download item (4 retries left).
changed: [node01]
changed: [node03]
FAILED - RETRYING: [node02]: Download_file | Download item (3 retries left).
FAILED - RETRYING: [node02]: Download_file | Download item (2 retries left).
FAILED - RETRYING: [node02]: Download_file | Download item (1 retries left).
fatal: [node02]: FAILED! => {
    "attempts": 4,
    "changed": false,
    "dest": "/tmp/releases/kubeadm-v1.30.4-amd64",
    "elapsed": 0,
    "response": "HTTP Error 403: Forbidden",
    "status_code": 403,
    "url": "https://dl.k8s.io/release/v1.30.4/bin/linux/amd64/kubeadm"
}

MSG:

Request failed

Any ideas?

Thx!

[dims]

@sebypp we've seen this before with other artifacts, so please create a local mirror of things you need and use it.

kubernetes/registry.k8s.io#138

You can search for hetzner in the k8s slack and see more discussions that keep popping up.

[sebypp]

@dims , thanks for the heads up. I've also got a reply from Hetzner support saying that some GeoIP databases are incorrectly flagging their IP addresses as belonging to different geographies.

The suggested solution was "please assign an alternative IP address using the "Primary IPs" menu of your respective project." Curious how that will work with Terraform :)

[dims]

@sebypp yea, unfortunate :(