You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TASK [kubernetes/control-plane : Kubeadm | Initialize first master]
fatal: [k8s-dev-master01.testkube.local]: FAILED! => {"attempts": 3, "changed": true, "cmd": ["timeout", "-k", "300s", "300s", "/usr/local/bin/kubeadm", "init", "--config=/etc/kubernetes/kubeadm-config.yaml", "--ignore-preflight-errors=all", "--skip-phases=addon/coredns,addon/kube-proxy", "--upload-certs"], "delta": "0:05:00.019405", "end": "2024-06-30 17:29:43.438077", "failed_when_result": true, "msg": "non-zero return code", "rc": 124, "start": "2024-06-30 17:24:43.418672", "stderr": "W0630 17:24:43.484417 10215 utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.234.0.10]; the provided value is: [169.254.25.10]\n\t[WARNING Port-6443]: Port 6443 is in use\n\t[WARNING Port-10259]: Port 10259 is in use\n\t[WARNING Port-10257]: Port 10257 is in use\n\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml]: /etc/kubernetes/manifests/kube-apiserver.yaml already exists\n\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml]: /etc/kubernetes/manifests/kube-controller-manager.yaml already exists\n\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml]: /etc/kubernetes/manifests/kube-scheduler.yaml already exists\n\t[WARNING Port-10250]: Port 10250 is in use", "stderr_lines": ["W0630 17:24:43.484417 10215 utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.234.0.10]; the provided value is: [169.254.25.10]", "\t[WARNING Port-6443]: Port 6443 is in use", "\t[WARNING Port-10259]: Port 10259 is in use", "\t[WARNING Port-10257]: Port 10257 is in use", "\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml]: /etc/kubernetes/manifests/kube-apiserver.yaml already exists", "\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml]: /etc/kubernetes/manifests/kube-controller-manager.yaml already exists", "\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml]: /etc/kubernetes/manifests/kube-scheduler.yaml already exists", "\t[WARNING Port-10250]: Port 10250 is in use"], "stdout": "[init] Using Kubernetes version: v1.29.5\n[preflight] Running pre-flight checks\n[preflight] Pulling images required for setting up a Kubernetes cluster\n[preflight] This might take a minute or two, depending on the speed of your internet connection\n[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'\n[certs] Using certificateDir folder "/etc/kubernetes/ssl"\n[certs] Using existing ca certificate authority\n[certs] Using existing apiserver certificate and key on disk\n[certs] Using existing apiserver-kubelet-client certificate and key on disk\n[certs] Using existing front-proxy-ca certificate authority\n[certs] Using existing front-proxy-client certificate and key on disk\n[certs] External etcd mode: Skipping etcd/ca certificate authority generation\n[certs] External etcd mode: Skipping etcd/server certificate generation\n[certs] External etcd mode: Skipping etcd/peer certificate generation\n[certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation\n[certs] External etcd mode: Skipping apiserver-etcd-client certificate generation\n[certs] Using the existing "sa" key\n[kubeconfig] Using kubeconfig folder "/etc/kubernetes"\n[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/admin.conf"\n[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/super-admin.conf"\n[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/kubelet.conf"\n[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/controller-manager.conf"\n[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/scheduler.conf"\n[control-plane] Using manifest folder "/etc/kubernetes/manifests"\n[control-plane] Creating static Pod manifest for "kube-apiserver"\n[control-plane] Creating static Pod manifest for "kube-controller-manager"\n[control-plane] Creating static Pod manifest for "kube-scheduler"\n[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"\n[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"\n[kubelet-start] Starting the kubelet\n[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 5m0s\n[kubelet-check] Initial timeout of 40s passed.", "stdout_lines": ["[init] Using Kubernetes version: v1.29.5", "[preflight] Running pre-flight checks", "[preflight] Pulling images required for setting up a Kubernetes cluster", "[preflight] This might take a minute or two, depending on the speed of your internet connection", "[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'", "[certs] Using certificateDir folder "/etc/kubernetes/ssl"", "[certs] Using existing ca certificate authority", "[certs] Using existing apiserver certificate and key on disk", "[certs] Using existing apiserver-kubelet-client certificate and key on disk", "[certs] Using existing front-proxy-ca certificate authority", "[certs] Using existing front-proxy-client certificate and key on disk", "[certs] External etcd mode: Skipping etcd/ca certificate authority generation", "[certs] External etcd mode: Skipping etcd/server certificate generation", "[certs] External etcd mode: Skipping etcd/peer certificate generation", "[certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation", "[certs] External etcd mode: Skipping apiserver-etcd-client certificate generation", "[certs] Using the existing "sa" key", "[kubeconfig] Using kubeconfig folder "/etc/kubernetes"", "[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/admin.conf"", "[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/super-admin.conf"", "[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/kubelet.conf"", "[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/controller-manager.conf"", "[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/scheduler.conf"", "[control-plane] Using manifest folder "/etc/kubernetes/manifests"", "[control-plane] Creating static Pod manifest for "kube-apiserver"", "[control-plane] Creating static Pod manifest for "kube-controller-manager"", "[control-plane] Creating static Pod manifest for "kube-scheduler"", "[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"", "[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"", "[kubelet-start] Starting the kubelet", "[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 5m0s", "[kubelet-check] Initial timeout of 40s passed."]}
I have the same issue when kube_vip is enabled and the cluster.yml playbook is used to create the cluster. The workaround is to create the cluster without kube_vip and then run cluster.yml again, with kube_vip enabled.
What happened?
Running an initial cluster creation breaks always on registering first master if kube-vip, metallb is enabled and cillium with kube-proxy replacment
What did you expect to happen?
Initialization of the first master failed.
How can we reproduce it (as minimally and precisely as possible)?
Deploy a minimal cluster in a fresh environment and activate kube-vip, metallb and cillium with kube-proxy replacment beforehand via addons.yml.
OS
Linux 6.8.0-36-lowlatency x86_64
PRETTY_NAME="Ubuntu 24.04 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
Version of Ansible
ansible [core 2.16.7]
config file = /Users/rexic/Docs/testkube_dev/kubespray/kubespray-install/kubespray-2.25.0/ansible.cfg
configured module search path = ['/Users/rexic/Docs/testkube_dev/kubespray/kubespray-install/kubespray-2.25.0/library']
ansible python module location = /Users/rexic/Docs/testkube_dev/kubespray/kubespray-install/.venv/lib/python3.12/site-packages/ansible
ansible collection location = /Users/sam/.ansible/collections:/usr/share/ansible/collections
executable location = /Users/rexic/Docs/testkube_dev/kubespray/kubespray-install/.venv/bin/ansible
python version = 3.12.4 (main, Jun 6 2024, 18:26:44) [Clang 15.0.0 (clang-1500.3.9.4)] (/Users/rexic/Docs/testkube_dev/kubespray/kubespray-install/.venv/bin/python)
jinja version = 3.1.4
libyaml = True
Version of Python
Python 3.12.4
Version of Kubespray (commit)
Collection (2.25.0)
Network plugin used
cilium
Full inventory with variables
[all]
k8s-dev-master01.testkube.local ansible_host=10.10.10.16 ip=10.10.10.16 etcd_member_name=etcd1
k8s-dev-worker01.testkube.local ansible_host=10.10.10.23 ip=10.10.10.23
k8s-dev-worker02.testkube.local ansible_host=10.10.10.24 ip=10.10.10.24
k8s-dev-worker03.testkube.local ansible_host=10.10.10.25 ip=10.10.10.25
[kube_control_plane]
k8s-dev-master01.testkube.local
[etcd]
k8s-dev-master01.testkube.local
[kube_node]
k8s-dev-worker01.testkube.local
k8s-dev-worker02.testkube.local
k8s-dev-worker03.testkube.local
[calico_rr]
[k8s_cluster:children]
kube_control_plane
kube_node
calico_rr
Command used to invoke ansible
ansible-playbook -i inventory/testkube-dev/inventory.ini -b -u ubuntu -v --private-key=~/.ssh/k8s_id_rsa cluster.yml
Output of ansible run
TASK [kubernetes/control-plane : Kubeadm | Initialize first master]
fatal: [k8s-dev-master01.testkube.local]: FAILED! => {"attempts": 3, "changed": true, "cmd": ["timeout", "-k", "300s", "300s", "/usr/local/bin/kubeadm", "init", "--config=/etc/kubernetes/kubeadm-config.yaml", "--ignore-preflight-errors=all", "--skip-phases=addon/coredns,addon/kube-proxy", "--upload-certs"], "delta": "0:05:00.019405", "end": "2024-06-30 17:29:43.438077", "failed_when_result": true, "msg": "non-zero return code", "rc": 124, "start": "2024-06-30 17:24:43.418672", "stderr": "W0630 17:24:43.484417 10215 utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.234.0.10]; the provided value is: [169.254.25.10]\n\t[WARNING Port-6443]: Port 6443 is in use\n\t[WARNING Port-10259]: Port 10259 is in use\n\t[WARNING Port-10257]: Port 10257 is in use\n\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml]: /etc/kubernetes/manifests/kube-apiserver.yaml already exists\n\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml]: /etc/kubernetes/manifests/kube-controller-manager.yaml already exists\n\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml]: /etc/kubernetes/manifests/kube-scheduler.yaml already exists\n\t[WARNING Port-10250]: Port 10250 is in use", "stderr_lines": ["W0630 17:24:43.484417 10215 utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.234.0.10]; the provided value is: [169.254.25.10]", "\t[WARNING Port-6443]: Port 6443 is in use", "\t[WARNING Port-10259]: Port 10259 is in use", "\t[WARNING Port-10257]: Port 10257 is in use", "\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml]: /etc/kubernetes/manifests/kube-apiserver.yaml already exists", "\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml]: /etc/kubernetes/manifests/kube-controller-manager.yaml already exists", "\t[WARNING FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml]: /etc/kubernetes/manifests/kube-scheduler.yaml already exists", "\t[WARNING Port-10250]: Port 10250 is in use"], "stdout": "[init] Using Kubernetes version: v1.29.5\n[preflight] Running pre-flight checks\n[preflight] Pulling images required for setting up a Kubernetes cluster\n[preflight] This might take a minute or two, depending on the speed of your internet connection\n[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'\n[certs] Using certificateDir folder "/etc/kubernetes/ssl"\n[certs] Using existing ca certificate authority\n[certs] Using existing apiserver certificate and key on disk\n[certs] Using existing apiserver-kubelet-client certificate and key on disk\n[certs] Using existing front-proxy-ca certificate authority\n[certs] Using existing front-proxy-client certificate and key on disk\n[certs] External etcd mode: Skipping etcd/ca certificate authority generation\n[certs] External etcd mode: Skipping etcd/server certificate generation\n[certs] External etcd mode: Skipping etcd/peer certificate generation\n[certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation\n[certs] External etcd mode: Skipping apiserver-etcd-client certificate generation\n[certs] Using the existing "sa" key\n[kubeconfig] Using kubeconfig folder "/etc/kubernetes"\n[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/admin.conf"\n[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/super-admin.conf"\n[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/kubelet.conf"\n[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/controller-manager.conf"\n[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/scheduler.conf"\n[control-plane] Using manifest folder "/etc/kubernetes/manifests"\n[control-plane] Creating static Pod manifest for "kube-apiserver"\n[control-plane] Creating static Pod manifest for "kube-controller-manager"\n[control-plane] Creating static Pod manifest for "kube-scheduler"\n[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"\n[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"\n[kubelet-start] Starting the kubelet\n[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 5m0s\n[kubelet-check] Initial timeout of 40s passed.", "stdout_lines": ["[init] Using Kubernetes version: v1.29.5", "[preflight] Running pre-flight checks", "[preflight] Pulling images required for setting up a Kubernetes cluster", "[preflight] This might take a minute or two, depending on the speed of your internet connection", "[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'", "[certs] Using certificateDir folder "/etc/kubernetes/ssl"", "[certs] Using existing ca certificate authority", "[certs] Using existing apiserver certificate and key on disk", "[certs] Using existing apiserver-kubelet-client certificate and key on disk", "[certs] Using existing front-proxy-ca certificate authority", "[certs] Using existing front-proxy-client certificate and key on disk", "[certs] External etcd mode: Skipping etcd/ca certificate authority generation", "[certs] External etcd mode: Skipping etcd/server certificate generation", "[certs] External etcd mode: Skipping etcd/peer certificate generation", "[certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation", "[certs] External etcd mode: Skipping apiserver-etcd-client certificate generation", "[certs] Using the existing "sa" key", "[kubeconfig] Using kubeconfig folder "/etc/kubernetes"", "[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/admin.conf"", "[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/super-admin.conf"", "[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/kubelet.conf"", "[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/controller-manager.conf"", "[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/scheduler.conf"", "[control-plane] Using manifest folder "/etc/kubernetes/manifests"", "[control-plane] Creating static Pod manifest for "kube-apiserver"", "[control-plane] Creating static Pod manifest for "kube-controller-manager"", "[control-plane] Creating static Pod manifest for "kube-scheduler"", "[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"", "[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"", "[kubelet-start] Starting the kubelet", "[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 5m0s", "[kubelet-check] Initial timeout of 40s passed."]}
NO MORE HOSTS LEFT
PLAY RECAP
k8s-dev-master01.testkube.local : ok=586 changed=120 unreachable=0 failed=1 skipped=628 rescued=0 ignored=3
k8s-dev-worker01.testkube.local : ok=419 changed=89 unreachable=0 failed=0 skipped=454 rescued=0 ignored=1
k8s-dev-worker02.testkube.local : ok=419 changed=89 unreachable=0 failed=0 skipped=450 rescued=0 ignored=1
k8s-dev-worker03.testkube.local : ok=419 changed=89 unreachable=0 failed=0 skipped=450 rescued=0 ignored=1
Sunday 30 June 2024 17:29:43 +0300 (0:20:26.892) 0:43:19.769
kubernetes/control-plane : Kubeadm | Initialize first master ------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1226.89s
download : Download_container | Download image if required ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 91.76s
download : Download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 90.40s
download : Download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 43.59s
download : Download_container | Download image if required ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 42.35s
download : Download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 41.43s
download : Download_container | Download image if required ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 33.83s
download : Download_container | Download image if required ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 23.72s
download : Download_container | Download image if required ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 22.03s
download : Download_container | Download image if required ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 20.11s
download : Download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 17.40s
download : Download_container | Download image if required ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 17.20s
download : Download_container | Download image if required ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 15.62s
download : Download_container | Download image if required ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 15.31s
container-engine/skopeo : Download_file | Download item -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 14.77s
container-engine/crictl : Download_file | Download item -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 14.20s
download : Download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 14.12s
download : Download_container | Download image if required ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 13.59s
container-engine/cri-o : Download_file | Download item --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 12.61s
download : Download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 12.32s
Anything else we need to know
No response
The text was updated successfully, but these errors were encountered: