forked from kubernetes-sigs/kubespray
-
Notifications
You must be signed in to change notification settings - Fork 51
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add vSphere external cloud provider (kubernetes-sigs#5959)
- Loading branch information
1 parent
d528aae
commit 1c6b385
Showing
22 changed files
with
881 additions
and
12 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| # vSphere CSI Driver | ||
|
|
||
| vSphere CSI driver allows you to provision volumes over a vSphere deployment. The Kubernetes historic in-tree cloud provider is deprecated and will be removed in future versions. | ||
|
|
||
| To enable vSphere CSI driver, uncomment the `vsphere_csi_enabled` option in `group_vars/all/vsphere.yml` and set it to `true`. | ||
|
|
||
| To set the number of replicas for the vSphere CSI controller, you can change `vsphere_csi_controller_replicas` option in `group_vars/all/vsphere.yml`. | ||
|
|
||
| You need to source the vSphere credentials you use to deploy your machines that will host Kubernetes. | ||
|
|
||
| | Variable | Required | Type | Choices | Default | Comment | | ||
| |---------------------------------------------|----------|---------|----------------------------|---------------------------|----------------------------------------------------------------| | ||
| | external_vsphere_vcenter_ip | TRUE | string | | | IP/URL of the vCenter | | ||
| | external_vsphere_vcenter_port | TRUE | string | | "443" | Port of the vCenter API | | ||
| | external_vsphere_insecure | TRUE | string | "true", "false" | "true" | set to "true" if the host above uses a self-signed cert | | ||
| | external_vsphere_user | TRUE | string | | | User name for vCenter with required privileges | | ||
| | external_vsphere_password | TRUE | string | | | Password for vCenter | | ||
| | external_vsphere_datacenter | TRUE | string | | | Datacenter name to use | | ||
| | external_vsphere_kubernetes_cluster_id | TRUE | string | | "kubernetes-cluster-id" | Kubernetes cluster ID to use | | ||
| | vsphere_cloud_controller_image_tag | TRUE | string | | "latest" | Kubernetes cluster ID to use | | ||
| | vsphere_syncer_image_tag | TRUE | string | | "v1.0.2" | Syncer image tag to use | | ||
| | vsphere_csi_attacher_image_tag | TRUE | string | | "v1.1.1" | CSI attacher image tag to use | | ||
| | vsphere_csi_controller | TRUE | string | | "v1.0.2" | CSI controller image tag to use | | ||
| | vsphere_csi_controller_replicas | TRUE | integer | | 1 | Number of pods Kubernetes should deploy for the CSI controller | | ||
| | vsphere_csi_liveness_probe_image_tag | TRUE | string | | "v1.1.0" | CSI liveness probe image tag to use | | ||
| | vsphere_csi_provisioner_image_tag | TRUE | string | | "v1.2.2" | CSI provisioner image tag to use | | ||
| | vsphere_csi_node_driver_registrar_image_tag | TRUE | string | | "v1.1.0" | CSI node driver registrat image tag to use | | ||
| | vsphere_csi_driver_image_tag | TRUE | string | | "v1.0.2" | CSI driver image tag to use | | ||
|
|
||
| ## Usage example | ||
|
|
||
| To test the dynamic provisioning using vSphere CSI driver, make sure to create a [storage policy](https://github.com/kubernetes/cloud-provider-vsphere/blob/master/docs/book/tutorials/kubernetes-on-vsphere-with-kubeadm.md#create-a-storage-policy) and [storage class](https://github.com/kubernetes/cloud-provider-vsphere/blob/master/docs/book/tutorials/kubernetes-on-vsphere-with-kubeadm.md#create-a-storageclass), then apply the following manifest: | ||
|
|
||
| ```yml | ||
| --- | ||
| apiVersion: v1 | ||
| kind: PersistentVolumeClaim | ||
| metadata: | ||
| name: csi-pvc-vsphere | ||
| spec: | ||
| accessModes: | ||
| - ReadWriteOnce | ||
| resources: | ||
| requests: | ||
| storage: 1Gi | ||
| storageClassName: Space-Efficient | ||
|
|
||
| --- | ||
| apiVersion: v1 | ||
| kind: Pod | ||
| metadata: | ||
| name: nginx | ||
| spec: | ||
| containers: | ||
| - image: nginx | ||
| imagePullPolicy: IfNotPresent | ||
| name: nginx | ||
| ports: | ||
| - containerPort: 80 | ||
| protocol: TCP | ||
| volumeMounts: | ||
| - mountPath: /var/lib/www/html | ||
| name: csi-data-vsphere | ||
| volumes: | ||
| - name: csi-data-vsphere | ||
| persistentVolumeClaim: | ||
| claimName: csi-pvc-vsphere | ||
| readOnly: false | ||
| ``` | ||
| Apply this conf to your cluster: ```kubectl apply -f nginx.yml``` | ||
|
|
||
| You should see the PVC provisioned and bound: | ||
|
|
||
| ```ShellSession | ||
| $ kubectl get pvc | ||
| NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE | ||
| csi-pvc-vsphere Bound pvc-dc7b1d21-ee41-45e1-98d9-e877cc1533ac 1Gi RWO Space-Efficient 10s | ||
| ``` | ||
|
|
||
| And the volume mounted to the Nginx Pod (wait until the Pod is Running): | ||
|
|
||
| ```ShellSession | ||
| kubectl exec -it nginx -- df -h | grep /var/lib/www/html | ||
| /dev/sdb 976M 2.6M 907M 1% /var/lib/www/html | ||
| ``` | ||
|
|
||
| ## More info | ||
|
|
||
| For further information about the vSphere CSI Driver, you can refer to the official [vSphere Cloud Provider documentation](https://cloud-provider-vsphere.sigs.k8s.io/container_storage_interface.html). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| ## Values for the external vSphere Cloud Provider | ||
| # external_vsphere_vcenter_ip: "myvcenter.domain.com" | ||
| # external_vsphere_vcenter_port: "443" | ||
| # external_vsphere_insecure: "true" | ||
| # external_vsphere_user: "administrator@vsphere.local" | ||
| # external_vsphere_password: "K8s_admin" | ||
| # external_vsphere_datacenter: "DATACENTER_name" | ||
| # external_vsphere_kubernetes_cluster_id: "kubernetes-cluster-id" | ||
|
|
||
| ## Tags for the external vSphere Cloud Provider images | ||
| # external_vsphere_cloud_controller_image_tag: "latest" | ||
| # vsphere_syncer_image_tag: "v1.0.2" | ||
| # vsphere_csi_attacher_image_tag: "v1.1.1" | ||
| # vsphere_csi_controller: "v1.0.2" | ||
| # vsphere_csi_liveness_probe_image_tag: "v1.1.0" | ||
| # vsphere_csi_provisioner_image_tag: "v1.2.2" | ||
|
|
||
| ## To use vSphere CSI plugin to provision volumes set this value to true | ||
| # vsphere_csi_enabled: true | ||
| # vsphere_csi_controller_replicas: 1 |
14 changes: 14 additions & 0 deletions
14
roles/kubernetes-apps/csi_driver/vsphere/defaults/main.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| --- | ||
| external_vsphere_vcenter_port: "443" | ||
| external_vsphere_insecure: "true" | ||
| external_vsphere_kubernetes_cluster_id: "kubernetes-cluster-id" | ||
|
|
||
| vsphere_syncer_image_tag: "v1.0.2" | ||
| vsphere_csi_attacher_image_tag: "v1.1.1" | ||
| vsphere_csi_controller: "v1.0.2" | ||
| vsphere_csi_liveness_probe_image_tag: "v1.1.0" | ||
| vsphere_csi_provisioner_image_tag: "v1.2.2" | ||
| vsphere_csi_node_driver_registrar_image_tag: "v1.1.0" | ||
| vsphere_csi_driver_image_tag: "v1.0.2" | ||
|
|
||
| vsphere_csi_controller_replicas: 1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| --- | ||
| - include_tasks: vsphere-credentials-check.yml | ||
| tags: vsphere-csi-driver | ||
|
|
||
| - name: vSphere CSI Driver | Generate CSI cloud-config | ||
| template: | ||
| src: "{{ item }}.j2" | ||
| dest: "{{ kube_config_dir }}/{{ item }}" | ||
| mode: 0640 | ||
| with_items: | ||
| - vsphere-csi-cloud-config | ||
| when: inventory_hostname == groups['kube-master'][0] | ||
| tags: vsphere-csi-driver | ||
|
|
||
| - name: vSphere CSI Driver | Generate Manifests | ||
| template: | ||
| src: "{{ item }}.j2" | ||
| dest: "{{ kube_config_dir }}/{{ item }}" | ||
| with_items: | ||
| - vsphere-csi-controller-rbac.yml | ||
| - vsphere-csi-controller-ss.yml | ||
| - vsphere-csi-node.yml | ||
| register: vsphere_csi_manifests | ||
| when: inventory_hostname == groups['kube-master'][0] | ||
| tags: vsphere-csi-driver | ||
|
|
||
| - name: vSphere CSI Driver | Create a CSI secret | ||
| command: "{{ bin_dir }}/kubectl create secret generic vsphere-config-secret --from-file=csi-vsphere.conf={{ kube_config_dir }}/vsphere-csi-cloud-config -n kube-system" | ||
| when: inventory_hostname == groups['kube-master'][0] | ||
| tags: vsphere-csi-driver | ||
|
|
||
| - name: vSphere CSI Driver | Apply Manifests | ||
| kube: | ||
| kubectl: "{{ bin_dir }}/kubectl" | ||
| filename: "{{ kube_config_dir }}/{{ item.item }}" | ||
| state: "latest" | ||
| with_items: | ||
| - "{{ vsphere_csi_manifests.results }}" | ||
| when: | ||
| - inventory_hostname == groups['kube-master'][0] | ||
| - not item is skipped | ||
| loop_control: | ||
| label: "{{ item.item }}" | ||
| tags: vsphere-csi-driver |
38 changes: 38 additions & 0 deletions
38
roles/kubernetes-apps/csi_driver/vsphere/tasks/vsphere-credentials-check.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| --- | ||
| - name: External vSphere Cloud Provider | check external_vsphere_vcenter_ip value | ||
| fail: | ||
| msg: "external_vsphere_vcenter_ip is missing" | ||
| when: external_vsphere_vcenter_ip is not defined or not external_vsphere_vcenter_ip | ||
|
|
||
| - name: External vSphere Cloud Provider | check external_vsphere_vcenter_port value | ||
| fail: | ||
| msg: "external_vsphere_vcenter_port is missing" | ||
| when: external_vsphere_vcenter_port is not defined or not external_vsphere_vcenter_port | ||
|
|
||
| - name: External vSphere Cloud Provider | check external_vsphere_insecure value | ||
| fail: | ||
| msg: "external_vsphere_insecure is missing" | ||
| when: external_vsphere_insecure is not defined or not external_vsphere_insecure | ||
|
|
||
| - name: External vSphere Cloud Provider | check external_vsphere_user value | ||
| fail: | ||
| msg: "external_vsphere_user is missing" | ||
| when: external_vsphere_user is not defined or not external_vsphere_user | ||
|
|
||
| - name: External vSphere Cloud Provider | check external_vsphere_password value | ||
| fail: | ||
| msg: "external_vsphere_password is missing" | ||
| when: | ||
| - external_vsphere_password is not defined or not external_vsphere_password | ||
|
|
||
| - name: External vSphere Cloud Provider | check external_vsphere_datacenter value | ||
| fail: | ||
| msg: "external_vsphere_datacenter is missing" | ||
| when: | ||
| - external_vsphere_datacenter is not defined or not external_vsphere_datacenter | ||
|
|
||
| - name: External vSphere Cloud Provider | check external_vsphere_kubernetes_cluster_id value | ||
| fail: | ||
| msg: "external_vsphere_kubernetes_cluster_id is missing" | ||
| when: | ||
| - external_vsphere_kubernetes_cluster_id is not defined or not external_vsphere_kubernetes_cluster_id |
9 changes: 9 additions & 0 deletions
9
roles/kubernetes-apps/csi_driver/vsphere/templates/vsphere-csi-cloud-config.j2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| [Global] | ||
| cluster-id = "{{ external_vsphere_kubernetes_cluster_id }}" | ||
|
|
||
| [VirtualCenter "{{ external_vsphere_vcenter_ip }}"] | ||
| insecure-flag = "{{ external_vsphere_insecure }}" | ||
| user = "{{ external_vsphere_user }}" | ||
| password = "{{ external_vsphere_password }}" | ||
| port = "{{ external_vsphere_vcenter_port }}" | ||
| datacenters = "{{ external_vsphere_datacenter }}" |
42 changes: 42 additions & 0 deletions
42
roles/kubernetes-apps/csi_driver/vsphere/templates/vsphere-csi-controller-rbac.yml.j2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| kind: ServiceAccount | ||
| apiVersion: v1 | ||
| metadata: | ||
| name: vsphere-csi-controller | ||
| namespace: kube-system | ||
| --- | ||
| kind: ClusterRole | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| metadata: | ||
| name: vsphere-csi-controller-role | ||
| rules: | ||
| - apiGroups: [""] | ||
| resources: ["nodes", "persistentvolumeclaims", "pods"] | ||
| verbs: ["get", "list", "watch"] | ||
| - apiGroups: [""] | ||
| resources: ["persistentvolumes"] | ||
| verbs: ["get", "list", "watch", "create", "update", "delete"] | ||
| - apiGroups: [""] | ||
| resources: ["events"] | ||
| verbs: ["get", "list", "watch", "create", "update", "patch"] | ||
| - apiGroups: ["storage.k8s.io"] | ||
| resources: ["storageclasses"] | ||
| verbs: ["get", "list", "watch"] | ||
| - apiGroups: ["storage.k8s.io"] | ||
| resources: ["csinodes"] | ||
| verbs: ["get", "list", "watch"] | ||
| - apiGroups: ["storage.k8s.io"] | ||
| resources: ["volumeattachments"] | ||
| verbs: ["get", "list", "watch", "update"] | ||
| --- | ||
| kind: ClusterRoleBinding | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| metadata: | ||
| name: vsphere-csi-controller-binding | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: vsphere-csi-controller | ||
| namespace: kube-system | ||
| roleRef: | ||
| kind: ClusterRole | ||
| name: vsphere-csi-controller-role | ||
| apiGroup: rbac.authorization.k8s.io |
Oops, something went wrong.