Kubeadm bootstrapper #1903

r2d4 · 2017-08-31T00:43:59Z

This is the last phase of and supersedes #1825

codecov-io · 2017-08-31T01:22:57Z

Codecov Report

Merging #1903 into master will decrease coverage by 0.11%.
The diff coverage is 7.31%.

@@            Coverage Diff            @@
##           master   #1903      +/-   ##
=========================================
- Coverage   30.11%     30%   -0.12%     
=========================================
  Files          76      77       +1     
  Lines        4722    4740      +18     
=========================================
  Hits         1422    1422              
- Misses       3120    3138      +18     
  Partials      180     180

Impacted Files	Coverage Δ
pkg/minikube/bootstrapper/localkube/localkube.go	`30.52% <ø> (+0.31%)`	⬆️
cmd/minikube/cmd/start.go	`9.93% <0%> (-0.1%)`	⬇️
cmd/minikube/cmd/root.go	`57.64% <0%> (-3.61%)`	⬇️
pkg/minikube/machine/cache_images.go	`2.56% <0%> (+0.06%)`	⬆️
pkg/minikube/bootstrapper/bootstrapper.go	`0% <0%> (ø)`
pkg/minikube/service/service.go	`29.76% <20%> (-0.86%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 34d840b...b291b0f. Read the comment docs.

MHBauer · 2017-08-31T18:31:11Z

Much smaller than the previous PR, nice.

Any special pitfalls I should know about if I try to give this a test?

r2d4 · 2017-08-31T18:40:14Z

There is no default hostpath storage provisioner (yet). Planning to run that as its own container once we get #1881 in

MHBauer · 2017-09-06T22:39:08Z

Do I have to do the 'make an iso' step as well as the standard build?

r2d4 · 2017-09-06T22:40:28Z

No, if you build the minikube binary from HEAD, it should be pointing to the correct ISO image.

r2d4 · 2017-09-07T16:33:17Z

Stop, then restart functionality here is still broken - due to the fact that kube-proxy uses a configmap with the hardcoded IP to the apiserver.

On stop/start, the IP can change and kube-proxy won't be updated with the new IP. I think I have a few ideas on how to handle this.

MHBauer · 2017-09-07T21:51:54Z

I think I have to do minikube -b kubeadm or minikube config set bootstrapper kubeadm. Is that right?

r2d4 · 2017-09-07T21:55:59Z

Thats correct @MHBauer.

The following are all equivalent:
minikube start --bootstrapper kubeadm

minikube config set bootstrapper kubeadm (will apply to all subsequent invocations of minikube start)

minikube start -b kubeadm

MHBauer · 2017-09-07T22:21:34Z

I can confirm it starts up!

% minikube start
Starting local Kubernetes v1.7.5 cluster...
Starting VM...
Getting VM IP address...
Moving files into cluster...
Downloading kubelet v1.7.5
Downloading kubeadm v1.7.5
Finished Downloading kubeadm v1.7.5
Finished Downloading kubelet v1.7.5
Setting up certs...
Connecting to cluster...
Setting up kubeconfig...
Starting cluster components...
Kubectl is now configured to use the cluster.

% minikube status
minikube: Running
cluster: Running
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.64.23

% kubectl get nodes
NAME       STATUS    AGE       VERSION
minikube   Ready     4m        v1.7.3

Not sure what's up with the version string mismatch there.

I see a bunch of containerized bits of kube started inside. Makes me happy.

rbac is working nicely.

service-catalog installs correctly.

service-catalog e2e's run and pass.

I'm making a note here: HUGE SUCCESS! 🎉

MHBauer · 2017-09-07T22:23:14Z

kubectl version does say 1.7.5, so not sure what's up with the node bit.

arschles

Mostly nits and "peanut gallery" style commentary. This looks great @r2d4

To repeat what we discussed on slack, I am going to wait until this is in, and then rebase #1904 on master, since this fixes a few of the integration test failures over there.

arschles · 2017-09-08T22:24:37Z

pkg/minikube/bootstrapper/kubeadm/kubeadm.go

+	kubeadmTmpl := "sudo /usr/bin/kubeadm init --config {{.KubeadmConfigFile}} --skip-preflight-checks"
+	t := template.Must(template.New("kubeadmTmpl").Parse(kubeadmTmpl))
+	b := bytes.Buffer{}
+	if err := t.Execute(&b, struct{ KubeadmConfigFile string }{constants.KubeadmConfigFile}); err != nil {


if you like, you can just use map[string]string{"KubeadmConfigFile: constants.KubeadmConfigFile} instead of the anonymous struct. it might read cleaner

Ah that's a good idea. I'll do that

arschles · 2017-09-08T22:26:44Z

pkg/minikube/bootstrapper/kubeadm/kubeadm.go

+	sudo /usr/bin/kubeadm alpha phase kubeconfig all --config {{.KubeadmConfigFile}} &&
+	sudo /usr/bin/kubeadm alpha phase controlplane all --config {{.KubeadmConfigFile}} &&
+	sudo /usr/bin/kubeadm alpha phase etcd local --config {{.KubeadmConfigFile}}
+	`


each line in this string are going to be indented. is that ok?

I'll undent it. I don't think it matters either way

arschles · 2017-09-08T22:28:11Z

pkg/minikube/bootstrapper/kubeadm/kubeadm.go

+func (k *KubeadmBootstrapper) UpdateCluster(cfg bootstrapper.KubernetesConfig) error {
+	if cfg.ShouldLoadCachedImages {
+		// Make best effort to load any cached images
+		go machine.LoadImages(k.c, constants.GetKubeadmCachedImages(cfg.KubernetesVersion), constants.ImageCacheDir)


for my understanding ... do you know that this goroutine will finish before this function returns (do you need to)?

No, there no guarantee. Since this is just a "best efforts" feature. The main function exits when the apiserver is up, so at that point caching and loading is won't have much effect. In practice usually we have more than enough time to cache and load the images

arschles · 2017-09-08T22:30:00Z

pkg/minikube/bootstrapper/kubeadm/kubeadm.go

+}
+
+func (k *KubeadmBootstrapper) generateConfig(k8s bootstrapper.KubernetesConfig) (string, error) {
+	t := template.Must(template.New("kubeadmConfigTmpl").Parse(kubeadmConfigTmpl))


as a defensive measure, you could put this (and the other function-scoped template.Must calls above) at global scope, so that minikube panics on startup if any template becomes invalid

That's a good idea. Will do

Until we get caching and can run an external storage provisioner.

Add status for kubeadm integration tests

aaron-prindle · 2017-09-14T22:54:51Z

pkg/minikube/constants/constants.go

+func GetKubernetesReleaseURL(binaryName, version string) string {
+	// TODO(r2d4): change this to official releases when the alpha controlplane commands are released.
+	// We are working with unreleased kubeadm changes at HEAD.
+	return fmt.Sprintf("https://storage.googleapis.com/minikube-builds/v1.7.3/%s", binaryName)


Should this be v1.7.3?
Also does kubeadm bootstrapper respect the k8s version flags?

aaron-prindle · 2017-09-14T23:04:15Z

LGTM, just tried this and it works great!
Might need some additional PRs to make sure all flag options work properly with it but it looks good to merge as.

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 31, 2017

r2d4 mentioned this pull request Aug 31, 2017

[WIP] Kubeadm bootstrapper #1825

Closed

8 tasks

r2d4 changed the title ~~Kubeadm v2~~ Kubeadm bootstrapper Aug 31, 2017

r2d4 mentioned this pull request Aug 31, 2017

Setting authorization mode to RBAC by default #1904

Closed

r2d4 force-pushed the kubeadm-v2 branch from 93e9ba5 to c27f52e Compare September 6, 2017 23:55

MHBauer approved these changes Sep 7, 2017

View reviewed changes

r2d4 force-pushed the kubeadm-v2 branch 2 times, most recently from c7484f5 to b92a23e Compare September 8, 2017 22:10

arschles approved these changes Sep 8, 2017

View reviewed changes

r2d4 mentioned this pull request Sep 9, 2017

Integration test cleanup #1938

Merged

r2d4 force-pushed the kubeadm-v2 branch from b92a23e to 1d54365 Compare September 14, 2017 17:39

r2d4 added 5 commits September 14, 2017 13:51

Add experimental kubeadm bootstrapper

49d471f

Disable provisioning test for kubeadm for now.

5ba262d

Until we get caching and can run an external storage provisioner.

Add jenkins integration tests for kubeadm

53e5806

Add status for kubeadm integration tests

Add caching for kubeadm

72f7628

Add restart for kube-proxy

b291b0f

r2d4 force-pushed the kubeadm-v2 branch from 1d54365 to b291b0f Compare September 14, 2017 20:51

aaron-prindle reviewed Sep 14, 2017

View reviewed changes

aaron-prindle approved these changes Sep 14, 2017

View reviewed changes

aaron-prindle merged commit e654879 into kubernetes:master Sep 14, 2017

r2d4 deleted the kubeadm-v2 branch September 15, 2017 00:18

ijl mentioned this pull request Sep 18, 2017

kubeadm should enable apiVersion batch/v2alpha1 and batch/v1beta1 for CronJob #1977

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Kubeadm bootstrapper #1903

Kubeadm bootstrapper #1903

r2d4 commented Aug 31, 2017

codecov-io commented Aug 31, 2017 •

edited

Loading

MHBauer commented Aug 31, 2017

r2d4 commented Aug 31, 2017

MHBauer commented Sep 6, 2017

r2d4 commented Sep 6, 2017

r2d4 commented Sep 7, 2017 •

edited

Loading

MHBauer commented Sep 7, 2017

r2d4 commented Sep 7, 2017

MHBauer commented Sep 7, 2017

MHBauer commented Sep 7, 2017

arschles left a comment

arschles Sep 8, 2017

r2d4 Sep 8, 2017

arschles Sep 8, 2017

r2d4 Sep 8, 2017

arschles Sep 8, 2017

r2d4 Sep 8, 2017

arschles Sep 8, 2017

r2d4 Sep 8, 2017

aaron-prindle Sep 14, 2017

aaron-prindle commented Sep 14, 2017

Kubeadm bootstrapper #1903

Kubeadm bootstrapper #1903

Conversation

r2d4 commented Aug 31, 2017

codecov-io commented Aug 31, 2017 • edited Loading

Codecov Report

MHBauer commented Aug 31, 2017

r2d4 commented Aug 31, 2017

MHBauer commented Sep 6, 2017

r2d4 commented Sep 6, 2017

r2d4 commented Sep 7, 2017 • edited Loading

MHBauer commented Sep 7, 2017

r2d4 commented Sep 7, 2017

MHBauer commented Sep 7, 2017

MHBauer commented Sep 7, 2017

arschles left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaron-prindle commented Sep 14, 2017

codecov-io commented Aug 31, 2017 •

edited

Loading

r2d4 commented Sep 7, 2017 •

edited

Loading