kola/tests: add kubernetes static pod test #910
Conversation
kola/tests/kubernetes/kubelet.go
Outdated
| "systemd": { | ||
| "units": [ | ||
| { | ||
| "contents": "[Unit]\nDescription=Kubernetes Kubelet\nRequires=docker.service\nAfter=docker.service\n\n[Service]\nExecStart=/usr/bin/hyperkube \\\n kubelet \\\n --config /etc/kubernetes/kubeletconfig\n\nRestart=always\nRestartSec=10\n\n[Install]\nWantedBy=multi-user.target\n", |
There was a problem hiding this comment.
Just a note, this and below assume kubelet is using docker (and that the host has docker at all) - so it'll need some adaption for Fedora CoreOS at least, and shortly RHCOS isn't going to use Docker either. But for now we can do this, just let's keep in mind it'll (hopefully shortly) need to change.
There was a problem hiding this comment.
I reduced the verification down to a curl command that doesn't depend on the the container runtime. I left docker in the kubelet unit file though.
kola/tests/kubernetes/kubelet.go
Outdated
| "files": [ | ||
| { | ||
| "contents": { | ||
| "source": "data:,kind%3A%20KubeletConfiguration%0AapiVersion%3A%20kubelet.config.k8s.io%2Fv1beta1%0AclusterDomain%3A%20kube.local%0AclusterDNS%3A%20%5B10.254.0.10%5D%0AcgroupDriver%3A%20systemd%0AstaticPodPath%3A%20%2Fetc%2Fkubernetes%2Fmanifests%2F%0Aauthorization%3A%0A%20%20mode%3A%20AlwaysAllow%0Aauthentication%3A%0A%20%20anonymous%3A%0A%20%20%20%20enabled%3A%20true%0A%20%20webhook%3A%0A%20%20%20%20enabled%3A%20false%0A", |
There was a problem hiding this comment.
Can you translate these into english for review at least? @sjennings should review this, to make sure it looks like the bootstrap config we are going to use.
There was a problem hiding this comment.
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
clusterDomain: kube.local
clusterDNS: [10.254.0.10]
cgroupDriver: systemd
staticPodPath: /etc/kubernetes/manifests/
authorization:
mode: AlwaysAllow
authentication:
anonymous:
enabled: true
webhook:
enabled: false
I noticed https://github.com/openshift/installer/blob/master/modules/ignition/resources/services/kubelet.service but kubelet in RHCOS is running 1.11 which uses a a kubeletconfiguration file for most of the options. If there is already an existing kubeletconfiguration, let me know so I can use that instead of the one I created.
There was a problem hiding this comment.
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
clusterDomain: kube.local
clusterDNS: [10.254.0.10]
cgroupDriver: systemd
staticPodPath: /etc/kubernetes/manifests/
authorization:
mode: AlwaysAllow
authentication:
anonymous:
enabled: true
webhook:
enabled: false
currently our manifest dir is /etc/origin/node/pods. might change that to match.
There was a problem hiding this comment.
if 4.0 is switching to a kubelet, and the isntaller is going to use /etc/kubernetes/manifests like upstream, I think we should take this change to just abandon /etc/origin/node/pods....
There was a problem hiding this comment.
I'll keep it as is if that is the destination going forward. Is there anything else that sticks out in terms of the kubeletconfiguration? I created it based on the API with the least amount of options to get it working.
mike-nguyen
force-pushed
the
static_pod
branch
from
e12a306 to
d7b07eb
Compare
|
Pushed some new changes:
|
kola/tests/kubernetes/kubelet.go
Outdated
| "github.com/coreos/mantle/platform/conf" | ||
| "github.com/coreos/mantle/util" | ||
|
|
||
| "github.com/vincent-petithory/dataurl" |
There was a problem hiding this comment.
Can you please re-order the imports to match the rest of the files?
stdlib imports
external imports
internal imports (in mantle)
|
Docker was dropped from RHCOS so I'll be re-working the test to run on crio. |
mike-nguyen
force-pushed
the
static_pod
branch
from
d7b07eb to
1e035c7
Compare
This is a basic smoke test to ensure a node can handle static pods on RHCOS. The test configures the static pod through ignition (drops in kubelet configuration, static pod definition, and kubelet unit file). Since there can be different container runtimes, the test verify that nginx is running on port 80 using curl.
mike-nguyen
force-pushed
the
static_pod
branch
from
1e035c7 to
6c3059a
Compare
|
Switched over to using CRI-O / runc but will need cni fix/workaround [0] for the port forward to work. [0] cri-o/cri-o#1804 |
|
Is this still blocked? |
This is a basic smoke test to ensure a node can handle static pods on
RHCOS. The test configures the static pod through ignition (drops in
kubelet configuration, static pod definition, and kubelet unit file).