Skip to content

Commit

Permalink
Initial implementation for YAML based config
Browse files Browse the repository at this point in the history 
Fix lint and format

Fixing go tests

Update go tests

Add more go tests
  • Loading branch information
@davidvonthenen
davidvonthenen committed Jun 8, 2020
1 parent 9ba2540 commit f070ca0
Show file tree
Hide file tree
Showing 38 changed files with 2,571 additions and 751 deletions.
57 changes: 32 additions & 25 deletions docs/book/tutorials/deploying_cpi_and_csi_with_multi_dc_vc_aka_zones.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,35 +67,42 @@ Steps that will be covered in order to setup zones for the vSphere CPI, vSphere

> ***Note:*** The CSI and CPI drivers have their own vsphere.conf files. The following modifications need to be made in both configurations.
The zones implementation depends on 2 sets of vSphere tags to be used on objects, such as datacenters or clusters. The first is a `region` tag and the second is a `zone` tag. vSphere tags are very simply put key/value pairs that can be assigned to objects and instead of using fixed keys to denote a `region` or a `zone`, we give the end-user the ability to come up with their own keys for a `region` and `zone` in the form of vSphere Tag Catagory. It just allows for a level of indirection in case you already have regions and zones setup in your configuration. Once a key/label or vSphere Tag Category is selected for each, create a `[Labels]` section in the `vsphere.conf` then assign tag names for both `region` and `zone`.
The zones implementation depends on 2 sets of vSphere tags to be used on objects, such as datacenters or clusters. The first is a `region` tag and the second is a `zone` tag. vSphere tags are very simply put key/value pairs that can be assigned to objects and instead of using fixed keys to denote a `region` or a `zone`, we give the end-user the ability to come up with their own keys for a `region` and `zone` in the form of vSphere Tag Catagory. It just allows for a level of indirection in case you already have regions and zones setup in your configuration. Once a key/label or vSphere Tag Category is selected for each, create a `labels:` section in the `vsphere.conf` then assign tag names for both `region` and `zone`.

In the example `vsphere.conf` below, `k8s-region` and `k8s-zone` was selected:

```bash
[Global]
# properties in this section will be used for all specified vCenters unless overridden in VirtualCenter section.

user = "vCenter username for cloud provider"
password = "password"
port = "443" #Optional
insecure-flag = "1" #set to 1 if the vCenter uses a self-signed cert
datacenters = "list of datacenters where Kubernetes node VMs are present"

[VirtualCenter "1.2.3.4"]
# Override specific properties for this Virtual Center.
user = "vCenter username for cloud provider"
password = "password"
# port, insecure-flag, datacenters will be used from Global section.

[VirtualCenter "10.0.0.1"]
# Override specific properties for this Virtual Center.
port = "448"
insecure-flag = "0"
# user, password, datacenters will be used from Global section.

[Labels]
region = k8s-region
zone = k8s-zone
# Global properties in this section will be used for all specified vCenters unless overriden in VirtualCenter section.
global:
user: YourVCenterUser
password: YourVCenterPass
port: "443"
# set insecure-flag to true if the vCenter uses a self-signed cert
insecure-flag: true
# settings for using k8s secret
secret-name: cpi-secret
secret-namespace: kube-system

# VirtualCenter section
virtualcenter:
tenant1:
user: YourVCenterUser
password: YourVCenterPass
server: 10.0.0.1
datacenters:
- mydc1
tenant2:
server: 127.0.0.1
port: 448
insecure-flag: false
datacenters:
- myotherdc1
- myotherdc2

# labels for regions and zones
labels:
region: k8s-region
zone: k8s-zone
```

### 2. Creating Zones in your vSphere Environment via Tags
Expand Down
55 changes: 36 additions & 19 deletions docs/book/tutorials/kubernetes-on-vsphere-with-kubeadm.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -451,34 +451,51 @@ This cloud-config configmap file, passed to the CPI on initialization, contains
```bash
# tee /etc/kubernetes/vsphere.conf >/dev/null <<EOF
[Global]
port = "443"
insecure-flag = "true"
secret-name = "cpi-global-secret"
secret-namespace = "kube-system"
[VirtualCenter "1.1.1.1"]
datacenters = "finance"
[VirtualCenter "192.168.0.1"]
datacenters = "hr"
[VirtualCenter "10.0.0.1"]
datacenters = "engineering"
secret-name = "cpi-engineering-secret"
secret-namespace = "kube-system"
# Global properties in this section will be used for all specified vCenters unless overriden in VirtualCenter section.
global:
port: "443"
# set insecure-flag to true if the vCenter uses a self-signed cert
insecure-flag: true
# settings for using k8s secret
secret-name: cpi-global-secret
secret-namespace: kube-system
# VirtualCenter section
virtualcenter:
tenant-finance:
server: 1.1.1.1
datacenters:
- globalfinanace
tenant-hr:
server: 192.168.0.1
datacenters:
- hrwest
- hreast
tenant-engineering:
server: 10.0.0.1
datacenters:
- engrwest1
- engrwest2
secret-name: cpi-engineering-secret
secret-namespace: kube-system
# labels for regions and zones
labels:
region: k8s-region
zone: k8s-zone
EOF
```
Here is a description of the fields used in the vsphere.conf configmap.
* `insecure-flag` should be set to true to use self-signed certificate for login
* `VirtualCenter` section is defined to hold property of vcenter. IP address and FQDN should be specified here.
* `insecure-flag` should be set to true to use self-signed certificate for login.
* `server` section is defined to hold property of vcenter IP address or FQDN.
* `secret-name` holds the credential(s) for a single or list of vCenter Servers.
* `secret-namespace` is set to the namespace where the secret has been created.
* `port` is the vCenter Server Port. The default is 443 if not specified.
* `datacenters` should be the list of all comma separated datacenters where kubernetes node VMs are present.
* `datacenters` should be the list of datacenters where kubernetes node VMs are present.
Create the configmap by running the following command:
Expand All @@ -505,7 +522,7 @@ The CPI supports storing vCenter credentials either in:
* a shared global secret containing all vCenter credentials, or
* a secret dedicated for a particular vCenter configuration which takes precedence over anything that might be configured within the global secret
In the example `vsphere.conf` above, there are two configured [Kubernetes secret](https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets). The vCenter at `10.0.0.1` contains credentials in the secret named `cpi-engineering-secret` in the namespace `kube-system` and the vCenter at `1.1.1.1` and `192.168.0.1` contains credentials in the secret named `cpi-global-secret` in the namespace `kube-system` defined in the `[Global]` section.
In the example `vsphere.conf` above, there are two configured [Kubernetes secret](https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets). The vCenter at `10.0.0.1` contains credentials in the secret named `cpi-engineering-secret` in the namespace `kube-system` and the vCenter at `1.1.1.1` and `192.168.0.1` contains credentials in the secret named `cpi-global-secret` in the namespace `kube-system` defined in the `global:` section.
An example [Secrets YAML](https://github.com/kubernetes/cloud-provider-vsphere/raw/master/manifests/controller-manager/vccm-secret.yaml) can be used for reference when creating your own `secrets`. If the example secret YAML is used, update the secret name to use a `<unique secret name>`, the vCenter IP address in the keys of `stringData`, and the `username` and `password` for each key.
Expand Down
13 changes: 7 additions & 6 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,17 @@ go 1.12
require (
github.com/dgrijalva/jwt-go v3.2.0+incompatible // indirect
github.com/docker/distribution v2.7.1+incompatible // indirect
github.com/evanphx/json-patch v4.5.0+incompatible // indirect
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6 // indirect
github.com/golang/protobuf v1.3.2
github.com/google/btree v1.0.0 // indirect
github.com/google/uuid v1.1.1
github.com/imdario/mergo v0.3.7 // indirect
github.com/opencontainers/go-digest v1.0.0-rc1 // indirect
github.com/pkg/errors v0.8.0
github.com/pkg/errors v0.9.0
github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829
github.com/spf13/cobra v0.0.3
github.com/spf13/pflag v1.0.3
github.com/vmware/govmomi v0.21.0
github.com/spf13/cobra v0.0.5
github.com/spf13/pflag v1.0.5
github.com/vmware/govmomi v0.22.1
github.com/vmware/vsphere-automation-sdk-go/lib v0.1.1
github.com/vmware/vsphere-automation-sdk-go/runtime v0.1.1
github.com/vmware/vsphere-automation-sdk-go/services/nsxt v0.1.1
Expand All @@ -28,16 +27,18 @@ require (
gopkg.in/gcfg.v1 v1.2.3
gopkg.in/square/go-jose.v2 v2.3.1 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.2.7
honnef.co/go/tools v0.0.1-2020.1.3 // indirect
k8s.io/api v0.0.0
k8s.io/apimachinery v0.0.0
k8s.io/apimachinery v0.17.0
k8s.io/client-go v0.0.0
k8s.io/cloud-provider v0.0.0
k8s.io/component-base v0.0.0
k8s.io/klog v0.3.2
k8s.io/kube-openapi v0.0.0-20190401085232-94e1e7b7574c // indirect
k8s.io/kubernetes v1.15.0
k8s.io/sample-controller v0.0.0-20190731144349-6f8905ae4ee5
sigs.k8s.io/kind v0.7.0 // indirect
)

replace (
Expand Down
Loading

0 comments on commit f070ca0

Please sign in to comment.