-
Notifications
You must be signed in to change notification settings - Fork 321
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade for AKS Cluster: Can't drain because Too Many Requests #1457
Comments
I'm having this problem again/still. I even tried setting I have to delete the PDB to upgrade my AKS cluster. |
Hi @DaleyKD this is likely addressed by https://github.com/hashicorp/consul-k8s/pull/1530/files. Will close this issue, and we should have that addressed in 0.49.0 which should happen sometime later this week or early next week. |
Would you consider reopening this? I'm currently trying to upgrade AKS from 1.25.6 to 1.26.6. Before upgrading, I upgraded It seems that nothing with the disruptionBudget changed for connect inject. kyle@Azure:~$ helm get values consul -n consul
USER-SUPPLIED VALUES:
connectInject:
transparentProxy:
defaultEnabled: false
dns:
enabled: false
global:
acls:
manageSystemACLs: true
datacenter: stratusdevdc1
gossipEncryption:
autoGenerate: true
name: consul
tls:
enableAutoEncrypt: true
enabled: true
server:
disruptionBudget:
enabled: false
replicas: 1 kyle@Azure:~$ kubectl get pdb -A
NAMESPACE NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE
consul consul-connect-injector N/A 0 0 13h
ingress nginxingress-ingress-nginx-controller 1 N/A 1 420d
kube-system coredns-pdb 1 N/A 1 420d
kube-system konnectivity-agent 1 N/A 1 420d
kube-system metrics-server-pdb 1 N/A 1 420d kyle@Azure:~$ kubectl get pdb/consul-connect-injector -n consul -o yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
annotations:
meta.helm.sh/release-name: consul
meta.helm.sh/release-namespace: consul
creationTimestamp: "2023-11-09T01:32:04Z"
generation: 1
labels:
app: consul
app.kubernetes.io/managed-by: Helm
chart: consul-helm
component: connect-injector
heritage: Helm
release: consul
name: consul-connect-injector
namespace: consul
resourceVersion: "257320864"
uid: 81f1e341-e87c-44f9-9faa-49375b0299e9
spec:
maxUnavailable: 0
selector:
matchLabels:
app: consul
component: connect-injector
release: consul
status:
conditions:
- lastTransitionTime: "2023-11-09T02:05:35Z"
message: ""
observedGeneration: 1
reason: InsufficientPods
status: "False"
type: DisruptionAllowed
currentHealthy: 1
desiredHealthy: 1
disruptionsAllowed: 0
expectedPods: 1
observedGeneration: 1 I have a hard time believe that, if I'm doing it correctly, I'm the only one who can't ever upgrade AKS. I suspect I'm missing something obvious. |
Community Note
Overview of the Issue
Currently trying to upgrade my AKS cluster from 1.23.8 to 1.24.0. Two of my nodes aren't able to drain. Please note: the other nodes with pods who have PDBs were able to upgrade successfully. Could it be that the
Max Unavailable
should rarely? be less than 1 and default to that?#1278 says that the
spec.maxUnavailable
is(n/2) - 1
, and I've seen somewhere else in these issues that the main recommended number of replicas forconsul-connect-injector
is 2, which meansspec.maxUnavailable
will be 0.I am almost completely new to K8S and especially PDBs, so I don't know what I'm talking about.
Reproduction Steps
Used Terraform to install this:
Expected behavior
Ideally, a problem-free AKS cluster upgrade.
Environment details
consul-k8s
version: 0.46.1/1.12.3Additional Context
kyle@Azure:~$ kubectl get pdb -A NAMESPACE NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE consulns consul-connect-injector N/A 0 0 19h ingress nginxingress-ingress-nginx-controller 1 N/A 1 19h kube-system coredns-pdb 1 N/A 1 19h kube-system konnectivity-agent 1 N/A 1 19h kube-system metrics-server-pdb 1 N/A 1 19h
The text was updated successfully, but these errors were encountered: