Make the provisioner interface support more container runtimes

[afbjorklund]

Currently the libmachine interface is quite targetted towards Docker:

        // Do the actual provisioning piece:
        //     1. Set the hostname on the instance.
        //     2. Install Docker if it is not present.
        //     3. Configure the daemon to accept connections over TLS.
        //     4. Copy the needed certificates to the server and local config dir.
        //     5. Configure / activate swarm if applicable.
        Provision(swarmOptions swarm.Options, authOptions auth.Options, engineOptions engine.Options) error

It needs to at least know which runtime to install, during provisioning ?

And the swarmOptions and engineOptions need a generic replacement...

---

Currently the provisioner in minikube is broken, in that it doesn't provision.

It just assumes that all container runtimes are always present on the node.

The original provisioner would install Docker, if it was not already available:

const (
	DefaultEngineInstallURL = "https://get.docker.com"
)

func installDockerGeneric(p Provisioner, baseURL string) error {
        // install docker - until cloudinit we use ubuntu everywhere so we
        // just install it using the docker repos
        if output, err := p.SSHCommand(fmt.Sprintf("if ! type docker; then curl -sSL %s | sh -; fi", baseURL)); err != nil {
                return fmt.Errorf("error installing docker: %s", output)
        }

        return nil
}

[afbjorklund]

It should be possible to start with a pretty "blank" machine.

And the Provisioner installs the container runtime.
Then the Bootstrapper installs kubernetes on that.

Currently there are some "grayzones" like CRI* and CNI*.

* mostly because Docker didn't use these by default
So it was left to the "other" runtimes, to install them

[afbjorklund]

This is a prerequisite for removing the runtimes: #9989

It's still possible to provide images with pre-installed runtimes.

[medyagh]

thanks for noting this @afbjorklund I think this is important, since we making containerd as Default as possible. I

[afbjorklund]

The main downside of this at the moment, is that you have to install and configure your own container runtime.

[medyagh]

this would allow us to install specific version of container-runtime, is there a reason you prioritize this as soon ? how about backlog ?

[medyagh]

one challenge would be how we learn to install Containerd or Docker or Crio on different OS (for example in case of SSH driver)
but in KIC and ISO it would be easy

[afbjorklund]

Currently Kubernetes defaults to using Docker to install containerd (?), rather than using the CNCF project https://containerd.io/

https://kubernetes.io/docs/setup/production-environment/container-runtimes/#containerd

Install the containerd.io package from the official Docker repositories.

I started on making an install script for CRI-O, but I'm not sure if it is available from the home page yet ? https://cri-o.io/

https://kubernetes.io/docs/setup/production-environment/container-runtimes/#cri-o

Install package devel:kubic:libcontainers:stable:cri-o:1.20 / cri-o

[afbjorklund]

We would also need some way to install CRI for Docker.

https://github.com/Mirantis/cri-dockerd

And to install crictl, since it is not provided by kubeadm

https://github.com/kubernetes-sigs/cri-tools

[afbjorklund]

is there a reason you prioritize this as soon ? how about backlog ?

@medyagh : actually I did put this in the backlog

The workaround is that the none or generic user would have to their own provisioning of the container runtime packages.

This issue is part of the "removing libmachine legacy", where Docker was the only provisioner that could ever be imagined

[sharifelgamal]

Freezing this because it's still relevant

[brandonros]

https://minikube.sigs.k8s.io/docs/drivers/ssh/

Might want to update the docs to say that only Docker container runtime is supported at the moment, and/or throw an error when trying to pass --container-runtime='containerd' with --driver=ssh

[brandonros]

even if the box you are sshing to already has containerd on it, it seems like minikube doesn't support this because it's looking for a docker libmachine profile to provision, and Flatcar Linux is not supported, and the libmachine repo is end-of-lifed because it got moved into Docker Desktop?

[afbjorklund]

because it's looking for a docker libmachine profile to provision

It is supposed to work with --container-runtime=containerd, but there are registry bugs requring "docker" to be present

the libmachine repo is end-of-lifed because it got moved into Docker Desktop?

the repo didn't get moved anywhere, but the previous Docker Toolbox product got replaced by the Docker Desktop product...

[afbjorklund]

@brandonros

and Flatcar Linux is not supported

As far as I know, Flatcar Linux (née CoreOS) should come with the container runtime pre-installed ?

So provisioning should be quite fast, basically which containerd and which ctr or some such.