Skip to content

Commit

Permalink
If no_proxy_exclude_workers is true, workers will be excluded from th…
Browse files Browse the repository at this point in the history
…e no_proxy variable.  This prevents docker engine restarting when scaling workers. (kubernetes-sigs#6520)

Signed-off-by: holmesb <5072156+holmesb@users.noreply.github.com>
  • Loading branch information
holmesb authored and LuckySB committed Jan 16, 2021
1 parent a364c94 commit 258ee3f
Show file tree
Hide file tree
Showing 4 changed files with 51 additions and 34 deletions.
7 changes: 7 additions & 0 deletions docs/proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,10 @@ If you set http and https proxy, all nodes and loadbalancer will be excluded fro
## Set additional addresses to default no_proxy (all cluster nodes and loadbalancer)

`additional_no_proxy: "aditional_host,"`

## Exclude workers from no_proxy

Since workers are included in the no_proxy variable, by default, docker engine will be restarted on all nodes (all
pods will restart) when adding or removing workers. To override this behaviour by only including master nodes in the
no_proxy variable, set:
`no_proxy_exclude_workers: true`
2 changes: 1 addition & 1 deletion docs/vars.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ Stack](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/dns-stack.m
* *docker_plugins* - This list can be used to define [Docker plugins](https://docs.docker.com/engine/extend/) to install.
* *containerd_config* - Controls some parameters in containerd configuration file (usually /etc/containerd/config.toml).
[Default config](https://github.com/kubernetes-sigs/kubespray/blob/master/roles/container-engine/containerd/defaults/main.yml) can be overriden in inventory vars.
* *http_proxy/https_proxy/no_proxy* - Proxy variables for deploying behind a
* *http_proxy/https_proxy/no_proxy/no_proxy_exclude_workers/additional_no_proxy* - Proxy variables for deploying behind a
proxy. Note that no_proxy defaults to all internal cluster IPs and hostnames
that correspond to each node.
* *kubelet_cgroup_driver* - Allows manual override of the
Expand Down
5 changes: 5 additions & 0 deletions inventory/sample/group_vars/all/all.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,11 @@ skip_non_kubeadm_warning: false
## If you need exclude all cluster nodes from proxy and other resources, add other resources here.
# additional_no_proxy: ""

## Since workers are included in the no_proxy variable by default, docker engine will be restarted on all nodes (all
## pods will restart) when adding or removing workers. To override this behaviour by only including master nodes in the
## no_proxy variable, set below to true:
no_proxy_exclude_workers: false

## Certificate Management
## This setting determines whether certs are generated via scripts.
## Chose 'none' if you provide your own certificates.
Expand Down
71 changes: 38 additions & 33 deletions roles/kubespray-defaults/tasks/no_proxy.yml
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,33 +1,38 @@
---
- name: Set no_proxy to all assigned cluster IPs and hostnames
set_fact:
no_proxy_prepare: >-
{%- if loadbalancer_apiserver is defined -%}
{{ apiserver_loadbalancer_domain_name| default('') }},
{{ loadbalancer_apiserver.address | default('') }},
{%- endif -%}
{%- for item in (groups['k8s-cluster'] + groups['etcd'] + groups['calico-rr']|default([]))|unique -%}
{{ hostvars[item]['access_ip'] | default(hostvars[item]['ip'] | default(fallback_ips[item])) }},
{%- if item != hostvars[item].get('ansible_hostname', '') -%}
{{ hostvars[item]['ansible_hostname'] }},
{{ hostvars[item]['ansible_hostname'] }}.{{ dns_domain }},
{%- endif -%}
{{ item }},{{ item }}.{{ dns_domain }},
{%- endfor -%}
{%- if additional_no_proxy is defined -%}
{{ additional_no_proxy }},
{%- endif -%}
127.0.0.1,localhost,{{ kube_service_addresses }},{{ kube_pods_subnet }}
delegate_to: localhost
connection: local
delegate_facts: yes
become: no
run_once: yes

- name: Populates no_proxy to all hosts
set_fact:
no_proxy: "{{ hostvars.localhost.no_proxy_prepare }}"
proxy_env: "{{ proxy_env | combine({
'no_proxy': hostvars.localhost.no_proxy_prepare,
'NO_PROXY': hostvars.localhost.no_proxy_prepare
}) }}"
---
- name: Set no_proxy to all assigned cluster IPs and hostnames
set_fact:
no_proxy_prepare: >-
{%- if loadbalancer_apiserver is defined -%}
{{ apiserver_loadbalancer_domain_name| default('') }},
{{ loadbalancer_apiserver.address | default('') }},
{%- endif -%}
{%- if ( (no_proxy_exclude_workers is defined) and (no_proxy_exclude_workers) ) -%}
{% set cluster_or_master = 'kube-master' %}
{% else %}
{% set cluster_or_master = 'k8s-cluster' %}
{% endif %}
{%- for item in (groups[cluster_or_master] + groups['etcd'] + groups['calico-rr']|default([]))|unique -%}
{{ hostvars[item]['access_ip'] | default(hostvars[item]['ip'] | default(fallback_ips[item])) }},
{%- if item != hostvars[item].get('ansible_hostname', '') -%}
{{ hostvars[item]['ansible_hostname'] }},
{{ hostvars[item]['ansible_hostname'] }}.{{ dns_domain }},
{%- endif -%}
{{ item }},{{ item }}.{{ dns_domain }},
{%- endfor -%}
{%- if additional_no_proxy is defined -%}
{{ additional_no_proxy }},
{%- endif -%}
127.0.0.1,localhost,{{ kube_service_addresses }},{{ kube_pods_subnet }}
delegate_to: localhost
connection: local
delegate_facts: yes
become: no
run_once: yes

- name: Populates no_proxy to all hosts
set_fact:
no_proxy: "{{ hostvars.localhost.no_proxy_prepare }}"
proxy_env: "{{ proxy_env | combine({
'no_proxy': hostvars.localhost.no_proxy_prepare,
'NO_PROXY': hostvars.localhost.no_proxy_prepare
}) }}"

0 comments on commit 258ee3f

Please sign in to comment.