From 589cbe0d99a6f0402a8f7b0152df1c79519443f7 Mon Sep 17 00:00:00 2001 From: Stefan Bueringer Date: Mon, 9 Sep 2024 17:42:30 +0200 Subject: [PATCH] Add more documentation about the KCP pre-terminate hook MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Stefan Büringer buringerst@vmware.com --- api/v1beta1/machine_types.go | 9 +++++++++ .../src/developer/providers/migrations/v1.8-to-v1.9.md | 9 +++++++++ 2 files changed, 18 insertions(+) diff --git a/api/v1beta1/machine_types.go b/api/v1beta1/machine_types.go index e6e0fa8fe0cf..ebe3030d6eb1 100644 --- a/api/v1beta1/machine_types.go +++ b/api/v1beta1/machine_types.go @@ -61,6 +61,15 @@ const ( // search each annotation for during the pre-terminate.delete lifecycle hook // to pause reconciliation of deletion. These hooks will prevent removal of // an instance from an infrastructure provider until all are removed. + // + // Notes for Machines managed by KCP (starting with Cluster API v1.8.2): + // * KCP adds its own pre-terminate hook on all Machines it controls. This is done to ensure it can later remove + // the etcd member right before Machine termination (i.e. before InfraMachine deletion). + // * Starting with Kubernetes v1.31 the KCP pre-terminate hook will wait for all other pre-terminate hooks to finish to + // ensure it runs last (thus ensuring that kubelet is still working while other pre-terminate hooks run). This is only done + // for v1.31 or above because the kubeadm ControlPlaneKubeletLocalMode was introduced with kubeadm 1.31. This feature configures + // the kubelet to communicate with the local apiserver. Only because of that the kubelet immediately starts failing after the etcd + // member is removed. We need the ControlPlaneKubeletLocalMode feature with 1.31 to adhere to the kubelet skew policy. PreTerminateDeleteHookAnnotationPrefix = "pre-terminate.delete.hook.machine.cluster.x-k8s.io" // MachineCertificatesExpiryDateAnnotation annotation specifies the expiry date of the machine certificates in RFC3339 format. diff --git a/docs/book/src/developer/providers/migrations/v1.8-to-v1.9.md b/docs/book/src/developer/providers/migrations/v1.8-to-v1.9.md index 45f500ff0886..e5eed60c2044 100644 --- a/docs/book/src/developer/providers/migrations/v1.8-to-v1.9.md +++ b/docs/book/src/developer/providers/migrations/v1.8-to-v1.9.md @@ -17,6 +17,15 @@ maintainers of providers and consumers of our Go API. ### Other +- Notes for Machines managed by KCP (starting with Cluster API v1.8.2): + - KCP adds its own pre-terminate hook on all Machines it controls. This is done to ensure it can later remove + the etcd member right before Machine termination (i.e. before InfraMachine deletion). + - Starting with Kubernetes v1.31 the KCP pre-terminate hook will wait for all other pre-terminate hooks to finish to + ensure it runs last (thus ensuring that kubelet is still working while other pre-terminate hooks run). This is only done + for v1.31 or above because the kubeadm ControlPlaneKubeletLocalMode was introduced with kubeadm 1.31. This feature configures + the kubelet to communicate with the local apiserver. Only because of that the kubelet immediately starts failing after the etcd + member is removed. We need the ControlPlaneKubeletLocalMode feature with 1.31 to adhere to the kubelet skew policy. + ### Suggested changes for providers - The Errors package was created when capi provider implementation was running as machineActuators that needed to vendor core capi to function. There is no usage recommendations today and its value is questionable since we moved to CRDs that inter-operate mostly via conditions. Instead we plan to drop the dedicated semantic for terminal failure and keep improving Machine lifecycle signal through conditions. Therefore the Errors package [has been deprecated in v1.8](https://github.com/kubernetes-sigs/cluster-api/issues/10784). It's recommented to remove any usage of the currently exported variables.