minikube with docker driver run as root not allowed but requires root anyway

[FlorianLudwig]

Steps to reproduce the issue:

1. Run:

   $ sudo minikube start --driver=docker` 
   [...]
   The "docker" driver should not be used with root privileges.
   

2. The solution is to create a user and grant it access to the docker socket. Granting access to the docker socket grants privileges equivalent to the root user.

Users will end up reading comments like here: #7903 (comment) and setup access to the docker socket without realizing they are granting root access to those users. By doing so minikube encourages less secure systems in practise.

[afbjorklund]

Hi @FlorianLudwig !

Hmm, the current situation is a bit confusing and the above seems to be as well:

Your step 1) is about "none" driver and step 2) is about "docker" driver ?

Maybe this happened because of some misguided suggestions on minikube's part:

❗ 'none' driver reported an issue: the 'none' driver must be run as the root user
💡 Suggestion: For non-root usage, try the newer 'docker' driver

Currently running minikube requires root, this is (or should be) documented: #7963
(The situation on macOS and Windows is a bit different, I'm going to ignore that here.)

And yes this also has security implications - which should be clearly documented - like so:

https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user

If you don’t want to preface the docker command with sudo, create a Unix group called docker and add users to it. When the Docker daemon starts, it creates a Unix socket accessible by members of the docker group.

Warning

The docker group grants privileges equivalent to the root user. For details on how this impacts security in your system, see Docker Daemon Attack Surface.

The podman driver is a bit more straight-forward, in that it still uses sudo rather than a root group.

---

Using a VM also requires admin privileges to set up*, but then root is inside the VM.
So this might be a better option, rather than running privileged containers (as root) ?

* the libvirt group is also root-equivalent

The current check against not running as root (with sudo), is mostly for the config files.
The files under the user account ended up being owned by root, or even under /root...

Only the "none" driver currently runs as root, since it is not intended for users but for CI
https://minikube.sigs.k8s.io/docs/drivers/none/

The "docker" driver is somewhere half-way, so it runs minikube as the user but docker as root.
Running all of Kubernetes as the user is an interesting project, but currently outside the scope.

You can check out the "Usernetes" project, if that is what you are after here ?

Ultimately we probably want to run also the none driver as non-root: #3760

Currently there are bunch of hacks and workarounds to move the config files afterwards.
And most of them don't work, due to hardcoded /root paths in for instance the certificates.

So it is more straight-forward to do everything as the user, and then call "sudo" as needed.
Even CI systems should not run as root by default, but rather use some functional user.

https://help.ubuntu.com/community/RootSudo

By default, the root account password is locked in Ubuntu

[afbjorklund]

But we should try to avoid misleading statements like these:

6276c1c

return registry.State{Error: fmt.Errorf("the 'none' driver must be run as the root user"), Healthy: false, Fix: "For non-root usage, try the newer 'docker' driver", Installed: true}

#3760 (comment)

I encourage that users of --driver=none consider using the newer --driver=docker, which does not require root access.

They seem to imply that running docker does not require root ?

---

I'm also not sure that the "docker" driver was ever intended as a substitute for "none" driver ?
It was more available as a new alternative on Linux, to run KIND (or KIC) rather than a VM.

Then it spread to other platforms, in case you were already running a Docker Desktop VM...
It requires less resources to run one VM than to run two VMs, but you could also choose ours.

[FlorianLudwig]

Hi @afbjorklund Thank you very much for your long explanation!

First of all, I indeed messed up my bug report - I meant the docker driver, not none. I fixed the bug report.

Some background: I am looking for a development environment without compromising (too much) on security nor usability - always a hard thing to do.

Options:

• Using the none driver (setup as root) together with a restricted kubernetes account (usable from non-root account)
• Using the docker driver (setup as root) but used from non-root account
• Using the kvm2 driver (setup as root) but used from non-root account
• Using the podman driver - I had some issues with this one but that's another story I guess

Using a VM also requires admin privileges to set up*, but then root is inside the VM.
So this might be a better option, rather than running privileged containers (as root) ?

• the libvirt group is also root-equivalent

Actually, this is what I ended up using. Without the libvirt group though, setup asked me for my password like 100 times but using kubectl now works fine.

---

But back to my original question: Why isn't it allowed to use the docker driver as root?

[FlorianLudwig]

But we should try to avoid misleading statements like these:

6276c1c

return registry.State{Error: fmt.Errorf("the 'none' driver must be run as the root user"), Healthy: false, Fix: "For non-root usage, try the newer 'docker' driver", Installed: true}

#3760 (comment)

I encourage that users of --driver=none consider using the newer --driver=docker, which does not require root access.

Exactly! This is what I mean with misleading :)

---

I'm also not sure that the "docker" driver was ever intended as a substitute for "none" driver ?
It was more available as a new alternative on Linux, to run KIND (or KIC) rather than a VM.

I didn't know about KIND, might be interesting to look at, thanks!

Then it spread to other platforms, in case you were already running a Docker Desktop VM...
It requires less resources to run one VM than to run two VMs, but you could also choose ours.

Actually, I don't run a VM normally. I try to stick to podman as much as I can as it plays a lot more nice with permissions etc.

[afbjorklund]

But back to my original question: Why isn't it allowed to use the docker driver as root?

There is no particular technical reason why the root user wouldn't be allowed to run minikube.
Theoretically you could log in as the root user, and have files end up under /root and be fine.

But since it is regarded as a bad practice (logging in as root), we try to "encourage" avoiding it.

The practical issue was that people run sudo minikube and ended up with unreadable config.

i.e. you would have files created by root, under ~/.minikube

So the next command would be unable to read e.g. config

Ending up with the ugly situation of having to sudo everything, even dashboard and kubectl
We wanted the user to be able to work from their usual account, instead of doing "ssh root@"

---

I didn't know about KIND, might be interesting to look at, thanks!

KIC (the "docker" driver) is our adapation of KIND, it uses the same docker image etc.

The main difference is that KIC is for developers, while KIND is for testing kubernetes.

Both are from Kubernetetes SIGs...

And both use kubeadm for setup.

Then it spread to other platforms, in case you were already running a Docker Desktop VM...
It requires less resources to run one VM than to run two VMs, but you could also choose ours.

Actually, I don't run a VM normally. I try to stick to podman as much as I can as it plays a lot more nice with permissions etc.

But podman is only available for linux, so there are no "other platforms" to talk about

Same problem though, so we expose podman-env to not have to use podman-machine
If we didn't, the user would end up with one VM for docker/podman and one VM for k8s ?

Once there is a Podman Desktop product, that could be considered as an option as well.

[FlorianLudwig]

Ending up with the ugly situation of having to sudo everything, even dashboard and kubectl
We wanted the user to be able to work from their usual account, instead of doing "ssh root@"

Isn't that still better than adding the user to the docker or kvm groups?

So maybe:

1. allow using sudo minikube and warn about that the config will be stored in root's home - maybe with a flag --yes-i-know-what-i-am-doing? (quickfix)
2. Rethink the security best practices recommended to the user in more details. (long term discussion)

[afbjorklund]

I think the sudo setup varies a little between the distributions, as well. Some use the same $HOME for both the user and the sudo user (root), while others change to put the sudo files under /root instead.

Others (looking at you, CentOS) even have different $PATH for sudo and root so it won't find things. Basically it is a hassle to support it, and most users seem fine without running minikube as root ?

Security could be better documented...

It's not really one of the strong points.

[FlorianLudwig]

What is the reasoning not allowing to run minikube as root? At least behind a flag --yes-i-know-what-i-am-doing? Setups are quite different and as you pointed out there is no technical reason to not allow it.

[jaybirdshain]

This is disappointing. Finally, I just did sudo chmod 0777 /var/run/docker.sock. And then minikube start and it started working.

[afbjorklund]

@jaybirdshain : if you are running as root, then you don't need to chmod. And if you are not (root), then using a docker group is more fine-grained than letting every local user have docker access ?

[snyman]

most users seem fine without running minikube as root ?

This is probably because (when using the docker driver) they are running it as root, because as the OP mentions, the current recommended setup of adding the user to the docker group effectively grants that user root privileges. So the correct phrasing there would be "most users seem fine (or unaware) running minikube (and any other process they run as their logged-in user) as effectively-root".

Minikube is encouraging users to make their systems less secure while actively disabling (for a fairly silly reason, in my opinion) the most obvious option a more security-conscious user might try to work around it.

[afbjorklund]

We don't yet have a good way of running some commands (like stop and start) as root, and some commands (like dashboard or service) as the regular user. The previous setup with sudo minikube created some config files and caches in the wrong place with the wrong permissions. So that needs to be fixed, preferrably without something silly like the current suggestions from the "none" driver...

It would be nice if it could reach the state of "make && sudo make install", so that you could mix. (Then again that just might be a bad example, some Makefiles are poorly written and don't build everything the first time which means some stuff in the local dir ends up being owned by root.) There is also no good auth mechanism, asking for password every time isn't improving things (it makes it unusable).

The recommendations for docker and podman (and libvirt) are pretty much based on upstream.

But it does required escalated privileges (basically root-access) for the user running minikube...

This needs to be clear from the documentation, even if it is not something that minikube is doing. Currently we don't have a driver that works without it (maybe VirtualBox*?), at least not on Linux.

* VirtualBox does require privileges to install, but I'm not sure it exposes them when running ?

[medyagh]

I agree @afbjorklund that allowing user to run as root causes a lot of headaches such as folders and files that are created and will make profile management more difficult,

and I don't think it will be in scope of minikube to run as root user. since we minikube's focus is local kubernetes developement.

[AkihiroSuda]

kind is going to support rootless Docker

kubernetes-sigs/kind#1727

[zaourzag]

I had the same issue, a quick minikube start --force --driver=docker fixed it for me

[ghost]

I had the same issue, a quick minikube start --force --driver=docker fixed it for me

This is worked for me too.Thanks a lot.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[spowelljr]

I agree @afbjorklund that allowing user to run as root causes a lot of headaches such as folders and files that are created and will make profile management more difficult,

and I don't think it will be in scope of minikube to run as root user. since we minikube's focus is local kubernetes developement.

Due to this I am closing this issue, feel free to respond/reopen if desired.

[jamallmahmoudi]

minikube start --force --driver=docker
minikube start --force --driver=kvm2
minikube start --force --driver=virtualbox
minikube start --force --driver=vmware workstation

[jamartincelis]

First

sudo usermod -aG docker $USER && newgrp docker

and then

minikube start

[JasonkayZK]

I had the same issue, a quick minikube start --force --driver=docker fixed it for me

This works fine with me. 😄
minikube version: v1.20.0 btw.

[laith-leo]

--force will get the deal done. However, this is HIGHLY UNRECOMMENDED inPROD

[subodhkumar95340]

Follow this command
It will resolve your issue

sudo -E minikube start --driver=none

sudo minikube config set driver none

minikube start

[afbjorklund]

You don't need to run minikube with sudo.

[AvikantSrivastava]

sudo chmod 666 /var/run/docker.sock

Give permission to the docker socket, and minikube should work as expected

[FlorianLudwig]

@AvikantSrivastava

sudo chmod 666 /var/run/docker.sock

Give permission to the docker socket, and minikube should work as expected

Just log in as root. Would be more honest as setting mod 666 on the docker socket gives root privileges to all users and applications on your system.

[niconical]

I had the same issue, a quick minikube start --force --driver=docker fixed it for me

Thanks.