Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP (Early): kic (improved VM-free: deploy Kubernetes to containers) #5188

Closed
wants to merge 20 commits into from
+1,015 −30
Closed

WIP (Early): kic (improved VM-free: deploy Kubernetes to containers) #5188

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
01b8f30
add boilerplate kic bootstrapper
medyagh Aug 20, 2019
f1d3ea9
Added boilerplate for kic pkg/driver
medyagh Aug 20, 2019
009a815
added pkg/minikube/drivers/kics boilerplate
medyagh Aug 20, 2019
066877a
goimports
medyagh Aug 20, 2019
186d2d7
added to supported os
medyagh Aug 20, 2019
e770a66
Added command runner boilerplate
medyagh Aug 20, 2019
8901818
Added Oci Runner
medyagh Aug 20, 2019
d019109
create OciRunner based on SSHRunner
medyagh Aug 22, 2019
f61a419
Add KickImage and OciClient to driver config
medyagh Aug 22, 2019
67913d1
Add auto select image
medyagh Aug 22, 2019
1538d99
rename to a shorter var name
medyagh Aug 22, 2019
d4bdd76
added start flags for ocilient and kicimg
medyagh Aug 22, 2019
9b4b041
skip download iso for kic and move helper func
medyagh Aug 22, 2019
11e8ba0
Add cache boiler funcs for kic and rename saveConfig to be more obvious
medyagh Aug 22, 2019
b004228
Add OciCommandRunner boilerplate
medyagh Aug 22, 2019
4e42176
add boilet plate bootstrapper for kic
medyagh Aug 23, 2019
25450b7
StartCluster for kic
medyagh Aug 23, 2019
e810f52
add boilerplate for installCNI
medyagh Aug 23, 2019
2e687b3
Added SetStdin to Runner interface
medyagh Aug 23, 2019
52b94c1
add SetSetdOut to command runner
medyagh Aug 23, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 33 additions & 11 deletions cmd/minikube/cmd/start.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,13 @@ import (
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/medyagh/kic/pkg/image"
gopshost "github.com/shirou/gopsutil/host"
"github.com/spf13/cobra"
"github.com/spf13/viper"
"golang.org/x/sync/errgroup"
cmdcfg "k8s.io/minikube/cmd/minikube/cmd/config"
pkgdrivers "k8s.io/minikube/pkg/drivers"
"k8s.io/minikube/pkg/minikube/bootstrapper"
"k8s.io/minikube/pkg/minikube/bootstrapper/kubeadm"
"k8s.io/minikube/pkg/minikube/cluster"
Expand Down Expand Up @@ -104,6 +106,8 @@ const (
waitUntilHealthy = "wait"
force = "force"
waitTimeout = "wait-timeout"
ociClient = "oci-client"
kicImg = "kic-img"
)

var (
Expand Down Expand Up @@ -197,6 +201,11 @@ func initDriverFlags() {

// hyperv
startCmd.Flags().String(hypervVirtualSwitch, "", "The hyperv virtual switch name. Defaults to first found. (only supported with HyperV driver)")

// kic
startCmd.Flags().String(kicImg, "", "override the kic image. (Only to be used with on with kic driver) ")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use --image? I would prefer not to have kic as a second brandname that users need to learn about.

startCmd.Flags().String(ociClient, "docker", "the oci client to be used. (Only to be used with on with kic driver)")
medyagh marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid introducing an additional level of confusing concepts, I think this should use --driver=docker (and rename --vm-driver to --driver)


}

// initNetworkingFlags inits the commandline flags for connectivity related flags for start
Expand Down Expand Up @@ -294,8 +303,8 @@ func runStart(cmd *cobra.Command, args []string) {
beginCacheImages(&cacheGroup, config.KubernetesConfig.ImageRepository, k8sVersion)

// Abstraction leakage alert: startHost requires the config to be saved, to satistfy pkg/provision/buildroot.
// Hence, saveConfig must be called before startHost, and again afterwards when we know the IP.
if err := saveConfig(&config); err != nil {
// Hence, saveProfileConfig must be called before startHost, and again afterwards when we know the IP.
if err := saveProfileConfig(&config); err != nil {
exit.WithError("Failed to save config", err)
}

Expand All @@ -308,7 +317,7 @@ func runStart(cmd *cobra.Command, args []string) {
showVersionInfo(k8sVersion, cr)
waitCacheImages(&cacheGroup)

// setup kube adm and certs and return bootstrapperx
// setup kube adm and certs and return bootstrapper
bs := setupKubeAdm(machineAPI, config.KubernetesConfig)

// The kube config must be update must come before bootstrapping, otherwise health checks may use a stale IP
Expand Down Expand Up @@ -393,7 +402,7 @@ func startMachine(config *cfg.Config) (runner command.Runner, preExists bool, ma
}
// Save IP to configuration file for subsequent use
config.KubernetesConfig.NodeIP = ip
if err := saveConfig(config); err != nil {
if err := saveProfileConfig(config); err != nil {
exit.WithError("Failed to save config", err)
}
runner, err = machine.CommandRunner(host)
Expand All @@ -413,7 +422,7 @@ func getKubernetesVersion() (k8sVersion string, isUpgrade bool) {
}

func downloadISO(config cfg.Config) {
if viper.GetString(vmDriver) != constants.DriverNone {
if !pkgdrivers.IsLocal(viper.GetString(vmDriver)) {
if err := cluster.CacheISO(config.MachineConfig); err != nil {
exit.WithError("Failed to cache ISO", err)
}
Expand Down Expand Up @@ -670,6 +679,15 @@ func generateCfgFromFlags(cmd *cobra.Command, k8sVersion string) (cfg.Config, er
out.T(out.SuccessType, "Using image repository {{.name}}", out.V{"name": repository})
}

kimg := viper.GetString(kicImg)
if kimg == "" {
kimg, err = image.NameForVersion(k8sVersion)
if err != nil {
glog.Errorf("error selecting kic image for %s : %v", k8sVersion, err)
}
glog.Infof("auto selected kic image: %s", kimg)
}

cfg := cfg.Config{
MachineConfig: cfg.MachineConfig{
KeepContext: viper.GetBool(keepContext),
Expand Down Expand Up @@ -700,6 +718,8 @@ func generateCfgFromFlags(cmd *cobra.Command, k8sVersion string) (cfg.Config, er
NoVTXCheck: viper.GetBool(noVTXCheck),
DNSProxy: viper.GetBool(dnsProxy),
HostDNSResolver: viper.GetBool(hostDNSResolver),
KicImage: kimg,
OciClient: viper.GetString(ociClient),
},
KubernetesConfig: cfg.KubernetesConfig{
KubernetesVersion: k8sVersion,
Expand Down Expand Up @@ -775,15 +795,15 @@ func startHost(api libmachine.API, mc cfg.MachineConfig) (*host.Host, bool) {
}

var host *host.Host
start := func() (err error) {
s := func() (err error) {
host, err = cluster.StartHost(api, mc)
if err != nil {
glog.Errorf("StartHost: %v", err)
}
return err
}

if err = retry.Expo(start, 2*time.Second, 3*time.Minute, 5); err != nil {
if err = retry.Expo(s, 2*time.Second, 3*time.Minute, 5); err != nil {
exit.WithError("Unable to start VM", err)
}
return host, exists
Expand Down Expand Up @@ -882,8 +902,8 @@ func setupKubeAdm(mAPI libmachine.API, kc cfg.KubernetesConfig) bootstrapper.Boo

// configureRuntimes does what needs to happen to get a runtime going.
func configureRuntimes(runner cruntime.CommandRunner) cruntime.Manager {
config := cruntime.Config{Type: viper.GetString(containerRuntime), Runner: runner}
cr, err := cruntime.New(config)
crCfg := cruntime.Config{Type: viper.GetString(containerRuntime), Runner: runner}
cr, err := cruntime.New(crCfg)
if err != nil {
exit.WithError("Failed runtime", err)
}
Expand All @@ -892,6 +912,8 @@ func configureRuntimes(runner cruntime.CommandRunner) cruntime.Manager {
if viper.GetString(vmDriver) == constants.DriverNone {
disableOthers = false
}

// MEDYA:TODO handle kic runtime
err = cr.Enable(disableOthers)
if err != nil {
exit.WithError("Failed to enable container runtime", err)
Expand Down Expand Up @@ -952,8 +974,8 @@ func configureMounts() {
}
}

// saveConfig saves profile cluster configuration in $MINIKUBE_HOME/profiles/<profilename>/config.json
func saveConfig(clusterCfg *cfg.Config) error {
// saveProfileConfig saves profile cluster configuration in $MINIKUBE_HOME/profiles/<profilename>/config.json
func saveProfileConfig(clusterCfg *cfg.Config) error {
return cfg.CreateProfile(viper.GetString(cfg.MachineProfile), clusterCfg)
}

Expand Down
5 changes: 3 additions & 2 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ require (
github.com/libvirt/libvirt-go v3.4.0+incompatible
github.com/machine-drivers/docker-machine-driver-vmware v0.1.1
github.com/mattn/go-isatty v0.0.8
github.com/medyagh/kic v0.0.3
github.com/mitchellh/go-ps v0.0.0-20170309133038-4fdf99ab2936
github.com/moby/hyperkit v0.0.0-20171020124204-a12cd7250bcd
github.com/olekukonko/tablewriter v0.0.0-20160923125401-bdcc175572fd
Expand All @@ -60,7 +61,7 @@ require (
github.com/xeipuuv/gojsonreference v0.0.0-20150808065054-e02fc20de94c // indirect
github.com/xeipuuv/gojsonschema v0.0.0-20160623135812-c539bca196be
github.com/zchee/go-vmnet v0.0.0-20161021174912-97ebf9174097
golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4
golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a
golang.org/x/sync v0.0.0-20190423024810-112230192c58
golang.org/x/sys v0.0.0-20190626221950-04f50cda93cb
Expand All @@ -69,7 +70,7 @@ require (
gopkg.in/gemnasium/logrus-airbrake-hook.v2 v2.1.2 // indirect
k8s.io/api v0.0.0
k8s.io/apimachinery v0.0.0
k8s.io/client-go v0.0.0
k8s.io/client-go v11.0.0+incompatible
k8s.io/kubectl v0.0.0-00010101000000-000000000000
sigs.k8s.io/sig-storage-lib-external-provisioner v4.0.0+incompatible
)
Expand Down
6 changes: 6 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,8 @@ github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzp
github.com/mattn/go-shellwords v0.0.0-20180605041737-f8471b0a71de/go.mod h1:3xCvwCdWdlDJUrvuMn7Wuy9eWs4pE8vqg+NOMyg4B2o=
github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/medyagh/kic v0.0.3 h1:V+kXN7cz9s4csIT9neiI1VPp7p21TEhnSa3WDxTEGN4=
github.com/medyagh/kic v0.0.3/go.mod h1:chkceU2bnXqVDVp/9DfHMXlZvyB3lqpzoQpSpN1DKIU=
github.com/mesos/mesos-go v0.0.9/go.mod h1:kPYCMQ9gsOXVAle1OsoY4I1+9kPu8GHkf88aV59fDr4=
github.com/mholt/caddy v0.0.0-20180213163048-2de495001514/go.mod h1:Wb1PlT4DAYSqOEd03MsqkdkXnTxA8v9pKjdpxbqM1kY=
github.com/microcosm-cc/bluemonday v1.0.1/go.mod h1:hsXNsILzKxV+sX77C5b8FSuKF00vh2OMYv+xgHpAMF4=
Expand Down Expand Up @@ -511,6 +513,8 @@ golang.org/x/crypto v0.0.0-20190228161510-8dd112bcdc25/go.mod h1:djNgcEr1/C05ACk
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8 h1:1wopBVtVdWnn03fZelqdXTqk7U7zPQCb+T4rbU9ZEoU=
golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4 h1:HuIa8hRrWRSrqYzx1qI49NNxhdi2PrY7gxVSq1JjLDc=
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190125153040-c74c464bbbf2/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190312203227-4b39c73a6495/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
Expand Down Expand Up @@ -635,6 +639,8 @@ k8s.io/gengo v0.0.0-20190116091435-f8a0810f38af/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8
k8s.io/heapster v1.2.0-beta.1/go.mod h1:h1uhptVXMwC8xtZBYsPXKVi8fpdlYkTs6k949KozGrM=
k8s.io/klog v0.3.1 h1:RVgyDHY/kFKtLqh67NvEWIgkMneNoIrdkN0CxDSQc68=
k8s.io/klog v0.3.1/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
k8s.io/klog v0.3.3 h1:niceAagH1tzskmaie/icWd7ci1wbG7Bf2c6YGcQv+3c=
k8s.io/klog v0.3.3/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
k8s.io/kube-openapi v0.0.0-20190228160746-b3a7cee44a30 h1:TRb4wNWoBVrH9plmkp2q86FIDppkbrEXdXlxU3a3BMI=
k8s.io/kube-openapi v0.0.0-20190228160746-b3a7cee44a30/go.mod h1:BXM9ceUBTj2QnfH2MK1odQs778ajze1RxcmP6S8RVVc=
k8s.io/kubernetes v1.15.2 h1:RO9EuRw5vlN3oa/lnmPxmywOoJRtg9o40KcklHXNIAQ=
Expand Down
9 changes: 9 additions & 0 deletions pkg/drivers/drivers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,17 @@ import (
"github.com/docker/machine/libmachine/ssh"
"github.com/golang/glog"
"github.com/pkg/errors"
"k8s.io/minikube/pkg/minikube/constants"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this makes the docker-machine drivers depend on minikube again ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch ! will remove that

)

// IsLocal returns whether or not the driver should be considered local
func IsLocal(name string) bool {
if name == constants.DriverNone || name == constants.DriverMock || name == constants.DriverKic {
return true
}
return false
}

// GetDiskPath returns the path of the machine disk image
func GetDiskPath(d *drivers.BaseDriver) string {
return filepath.Join(d.ResolveStorePath("."), d.GetMachineName()+".rawdisk")
Expand Down
Loading