Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minikube on driver docker not running using docker desktop vm #14202

Closed
prakharporwal opened this issue May 21, 2022 · 23 comments · Fixed by #15126
Closed

minikube on driver docker not running using docker desktop vm #14202

prakharporwal opened this issue May 21, 2022 · 23 comments · Fixed by #15126
Assignees
Labels
co/docker-driver Issues related to kubernetes in container kind/bug Categorizes issue or PR as related to a bug. needs-faq-entry Things that could use documentation in a FAQ os/linux priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done.

Comments

@prakharporwal
Copy link

prakharporwal commented May 21, 2022

What Happened?

  • I run minikube start
    I get above logs.

image

I have tried multiple times
I tried with driver virtualbox but failes because virtual box and docker desktop cannot run together ( because 2 VM cannot run together on Ubuntu)

minikube status gives this

image

I have tried

  • reinstallng minikube
  • restarting the minikube container
  • tried minikube delete --all and minikube start
    Not working in all the above cases.

Attach the log file

logs.txt

Operating System

Ubuntu

Driver

Docker

@afbjorklund
Copy link
Collaborator

afbjorklund commented May 21, 2022

I don't think Docker Desktop (VM) has been tested on Ubuntu, only the open source (and native) Docker Engine...

https://docs.docker.com/engine/install/ubuntu/

There might be some hard-coded code paths, where only macOS and Windows expect to be using Docker Desktop.

// IsDockerDesktop checks if the driver is a Docker for Desktop (Docker on windows or MacOs)
// for linux and exotic archs, this will be false
func IsDockerDesktop(name string) bool {
        if IsDocker(name) {
                if runtime.GOOS == "darwin" || runtime.GOOS == "windows" {
                        return true
                }
        }
        return false
}

@afbjorklund afbjorklund added os/linux co/docker-driver Issues related to kubernetes in container labels May 21, 2022
@prakharporwal
Copy link
Author

Is it possible to give some fix for now from minikube end ?

@scoobhidu
Copy link

Yes I am also getting the same error

image

@afbjorklund afbjorklund changed the title Ubuntu 20.04 : minikube on driver docker not running Ubuntu 20.04 : minikube on driver docker not running using docker desktop vm May 24, 2022
@afbjorklund
Copy link
Collaborator

Please use text instead of images.

@afbjorklund afbjorklund added kind/bug Categorizes issue or PR as related to a bug. priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. labels May 24, 2022
@afbjorklund afbjorklund added the needs-faq-entry Things that could use documentation in a FAQ label Jun 27, 2022
@afbjorklund afbjorklund changed the title Ubuntu 20.04 : minikube on driver docker not running using docker desktop vm minikube on driver docker not running using docker desktop vm Aug 21, 2022
@mesketh
Copy link

mesketh commented Sep 12, 2022

Same issue:

---8<-----

  • ==> Last Start <==
  • Log file created at: 2022/09/12 12:08:55
    Running on machine: markx1
    Binary: Built with gc go1.18.3 for linux/amd64
    Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
    I0912 12:08:55.207032 1552433 out.go:296] Setting OutFile to fd 1 ...
    I0912 12:08:55.207148 1552433 out.go:348] isatty.IsTerminal(1) = true
    I0912 12:08:55.207151 1552433 out.go:309] Setting ErrFile to fd 2...
    I0912 12:08:55.207155 1552433 out.go:348] isatty.IsTerminal(2) = true
    I0912 12:08:55.207240 1552433 root.go:333] Updating PATH: /home/mark/.minikube/bin
    I0912 12:08:55.207553 1552433 out.go:303] Set.ting JSON to false
    I0912 12:08:55.232156 1552433 start.go:115] hostinfo: {"hostname":"markx1","uptime":218779,"bootTime":1662729757,"procs":321,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"22.04","kernelVersion":"5.15.0-46-generic","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"host","hostId":"0cdb2f8c-ebdc-46f8-b26c-f3aaf240514a"}
    I0912 12:08:55.232223 1552433 start.go:125] virtualization: kvm host
    I0912 12:08:55.235875 1552433 out.go:177] 😄 minikube v1.26.1 on Ubuntu 22.04
    I0912 12:08:55.238100 1552433 notify.go:193] Checking for updates...
    I0912 12:08:55.239057 1552433 config.go:180] Loaded profile config "minikube": Driver=none, ContainerRuntime=containerd, KubernetesVersion=v1.24.3
    I0912 12:08:55.239176 1552433 driver.go:365] Setting default libvirt URI to qemu:///system
    I0912 12:08:55.240159 1552433 exec_runner.go:51] Run: systemctl --version
    I0912 12:08:55.271983 1552433 out.go:177] ✨ Using the none driver based on existing profile
    I0912 12:08:55.273427 1552433 start.go:284] selected driver: none
    I0912 12:08:55.273451 1552433 start.go:808] validating driver "none" against &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.33@sha256:73b259e144d926189cf169ae5b46bbec4e08e4e2f2bd87296054c3244f70feb8 Memory:3900 CPUs:2 DiskSize:20000 VMDriver: Driver:none HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.3 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:resolv-conf Value:/run/systemd/resolve/resolv.conf}] ShouldLoadCachedImages:false EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name:m01 IP:192.168.86.235 Port:8443 KubernetesVersion:v1.24.3 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/mark:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
    I0912 12:08:55.273621 1552433 start.go:819] status for none: {Installed:true Healthy:true Running:false NeedsImprovement:false Error: Reason: Fix: Doc: Version:}
    I0912 12:08:55.273658 1552433 start.go:1544] auto setting extra-config to "kubelet.resolv-conf=/run/systemd/resolve/resolv.conf".
    I0912 12:08:55.292763 1552433 cni.go:95] Creating CNI manager for ""
    I0912 12:08:55.292771 1552433 cni.go:149] Driver none used, CNI unnecessary in this configuration, recommending no CNI
    I0912 12:08:55.292791 1552433 start_flags.go:310] config:
    {Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.33@sha256:73b259e144d926189cf169ae5b46bbec4e08e4e2f2bd87296054c3244f70feb8 Memory:3900 CPUs:2 DiskSize:20000 VMDriver: Driver:none HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.3 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:resolv-conf Value:/run/systemd/resolve/resolv.conf}] ShouldLoadCachedImages:false EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name:m01 IP:192.168.86.235 Port:8443 KubernetesVersion:v1.24.3 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/mark:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
    I0912 12:08:55.294588 1552433 out.go:177] 👍 Starting control plane node minikube in cluster minikube
    I0912 12:08:55.296179 1552433 profile.go:148] Saving config to /home/mark/.minikube/profiles/minikube/config.json ...
    I0912 12:08:55.296398 1552433 cache.go:208] Successfully downloaded all kic artifacts
    I0912 12:08:55.296419 1552433 start.go:371] acquiring machines lock for minikube: {Name:mk5f9b86c28b826cc11ad7f97be3d412517886cb Clock:{} Delay:500ms Timeout:13m0s Cancel:}
    I0912 12:08:55.296599 1552433 start.go:375] acquired machines lock for "minikube" in 152.922µs
    I0912 12:08:55.296617 1552433 start.go:95] Skipping create...Using existing machine configuration
    I0912 12:08:55.296623 1552433 fix.go:55] fixHost starting: m01
    W0912 12:08:55.297376 1552433 none.go:130] unable to get port: "minikube" does not appear in /home/mark/.kube/config
    I0912 12:08:55.297398 1552433 api_server.go:165] Checking apiserver status ...
    I0912 12:08:55.297457 1552433 exec_runner.go:51] Run: sudo pgrep -xnf kube-apiserver.minikube.
    W0912 12:08:55.319931 1552433 api_server.go:169] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.minikube.: exit status 1
    stdout:

stderr:
I0912 12:08:55.319988 1552433 exec_runner.go:51] Run: sudo systemctl is-active --quiet service kubelet
I0912 12:08:55.332543 1552433 fix.go:103] recreateIfNeeded on minikube: state=Stopped err=
W0912 12:08:55.332566 1552433 fix.go:129] unexpected machine state, will restart:
I0912 12:08:55.334407 1552433 out.go:177] 🔄 Restarting existing none bare metal machine for "minikube" ...
I0912 12:08:55.339368 1552433 profile.go:148] Saving config to /home/mark/.minikube/profiles/minikube/config.json ...
I0912 12:08:55.339788 1552433 start.go:307] post-start starting for "minikube" (driver="none")
I0912 12:08:55.339810 1552433 start.go:335] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0912 12:08:55.340009 1552433 exec_runner.go:51] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0912 12:08:55.358979 1552433 main.go:134] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0912 12:08:55.359057 1552433 main.go:134] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0912 12:08:55.359075 1552433 main.go:134] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0912 12:08:55.362343 1552433 out.go:177] ℹ️ OS release is Ubuntu 22.04.1 LTS
I0912 12:08:55.363769 1552433 filesync.go:126] Scanning /home/mark/.minikube/addons for local assets ...
I0912 12:08:55.363833 1552433 filesync.go:126] Scanning /home/mark/.minikube/files for local assets ...
I0912 12:08:55.363863 1552433 start.go:310] post-start completed in 24.059413ms
I0912 12:08:55.363873 1552433 fix.go:57] fixHost completed within 67.250199ms
I0912 12:08:55.363880 1552433 start.go:82] releasing machines lock for "minikube", held for 67.271011ms
I0912 12:08:55.364759 1552433 exec_runner.go:51] Run: curl -sS -m 2 https://k8s.gcr.io/
I0912 12:08:55.366595 1552433 out.go:177]
W0912 12:08:55.368185 1552433 out.go:239] ❌ Exiting due to RUNTIME_ENABLE: unknown network plugin:
W0912 12:08:55.368228 1552433 out.go:239]

@dishkakrauch
Copy link

Same issue. Do we have any change to fix it this year?

@jodusan
Copy link

jodusan commented Sep 24, 2022

Same issue here

@zidanomar
Copy link

Up~ same issue here

@Arun-N
Copy link

Arun-N commented Oct 1, 2022

Facing the same issue. Completely removed minikube and tried again, but no luck.

OS
Ubuntu 22.04.1 LTS

Docker
Docker Desktop
version: 20.10.18
build: b40c2f6

Minikube
version: v1.27.0
commit: 4243041

minikube start                                                                                                arun@arun-MS-7C56
😄  minikube v1.27.0 on Ubuntu 22.04
❗  Kubernetes 1.25.0 has a known issue with resolv.conf. minikube is using a workaround that should work for most use cases.
❗  For more information, see: https://github.com/kubernetes/kubernetes/issues/112135
✨  Automatically selected the docker driver. Other choices: ssh, qemu2 (experimental)
💨  For improved Docker performance, enable the overlay Linux kernel module using 'modprobe overlay'
📌  Using Docker driver with root privileges
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.25.0 preload ...
    > preloaded-images-k8s-v18-v1...:  385.37 MiB / 385.37 MiB  100.00% 5.63 Mi
    > gcr.io/k8s-minikube/kicbase:  386.75 MiB / 386.76 MiB  100.00% 5.18 MiB p
    > gcr.io/k8s-minikube/kicbase:  0 B [_________________________] ?% ? p/s 0s
    > index.docker.io/kicbase/sta...:  386.75 MiB / 386.76 MiB  100.00% 8.17 Mi
    > index.docker.io/kicbase/sta...:  0 B [______________________] ?% ? p/s 0s
    > gcr.io/k8s-minikube/kicbase...:  386.75 MiB / 386.76 MiB  100.00% 7.93 Mi
    > gcr.io/k8s-minikube/kicbase...:  0 B [______________________] ?% ? p/s 0s
    > index.docker.io/kicbase/sta...:  386.75 MiB / 386.76 MiB  100.00% 8.15 Mi
    > index.docker.io/kicbase/sta...:  0 B [______________________] ?% ? p/s 0s
❗  minikube was unable to download gcr.io/k8s-minikube/kicbase:v0.0.34, but successfully downloaded docker.io/kicbase/stable:v0.0.34 as a fallback image
E1001 10:56:37.366171    6263 cache.go:203] Error downloading kic artifacts:  failed to download kic base image or any fallback image
🔥  Creating docker container (CPUs=2, Memory=3769MB) ...
🐳  Preparing Kubernetes v1.25.0 on Docker 20.10.17 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...\ E1001 11:00:02.808660    6263 start.go:267] Unable to scale down deployment "coredns" in namespace "kube-system" to 1 replica: timed out waiting for the condition

🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
❗  Enabling 'default-storageclass' returned an error: running callbacks: [Error making standard the default storage class: Error listing StorageClasses: Get "https://192.168.49.2:8443/apis/storage.k8s.io/v1/storageclasses": dial tcp 192.168.49.2:8443: i/o timeout]
🌟  Enabled addons: storage-provisioner

❌  Exiting due to GUEST_START: wait 6m0s for node: wait for healthy API server: apiserver healthz never reported healthy: timed out waiting for the condition

╭───────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                           │
│    😿  If the above advice does not help, please let us know:                             │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose                           │
│                                                                                           │
│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │
│                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────╯

@gio-shara-code
Copy link

gio-shara-code commented Oct 9, 2022

Same issue here guys.

OS
os name: Pop!_OS 22.04 LTS (64-bit)

Executing minikube start

gioshara@pop-os ~> minikube start
😄  minikube v1.27.1 on Debian bookworm/sid
✨  Using the docker driver based on user configuration
📌  Using Docker driver with root privileges
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
    > gcr.io/k8s-minikube/kicbase:  0 B [______________________] ?% ? p/s 1m15s
🔥  Creating docker container (CPUs=2, Memory=3597MB) ...
🐳  Preparing Kubernetes v1.25.2 on Docker 20.10.18 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...| E1009 13:23:34.119267   85786 start.go:268] Unable to scale down deployment "coredns" in namespace "kube-system" to 1 replica: timed out waiting for the condition

🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
❗  Enabling 'default-storageclass' returned an error: running callbacks: [Error making standard the default storage class: Error listing StorageClasses: Get "https://192.168.49.2:8443/apis/storage.k8s.io/v1/storageclasses": dial tcp 192.168.49.2:8443: i/o timeout]
🌟  Enabled addons: storage-provisioner

❌  Exiting due to GUEST_START: wait 6m0s for node: wait for healthy API server: apiserver healthz never reported healthy: timed out waiting for the condition

╭───────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                           │
│    😿  If the above advice does not help, please let us know:                             │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose                           │
│                                                                                           │
│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │
│                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
gioshara@pop-os ~ [14]> minikube config get driver
docker

Do we have some updates regarding this issue?

@afbjorklund
Copy link
Collaborator

afbjorklund commented Oct 9, 2022

It will need code changes, before it will work (with Docker Desktop on Linux). There should be a warning, meanwhile.

The supported environment is Docker Engine on Linux (without the VM), as in https://docs.docker.com/engine/install/

It is documented here: https://minikube.sigs.k8s.io/docs/drivers/docker/

You can also use any other --vm driver, besides the docker driver.

@ComradeProgrammer
Copy link
Member

By the way, is there anybody working on this issue? If not, maybe I am willing to give a try to support linux docker desktop.(Although I am not sure whether I can solve it)

@klaases
Copy link
Contributor

klaases commented Oct 11, 2022

Hi @ComradeProgrammer, there is no one currently assigned to this issue and we would be happy to review your contribution.

Would you like to be assigned to the issue?

@ComradeProgrammer
Copy link
Member

/assign ComradeProgrammer

@Akritesom
Copy link

What Happened?

  • I run minikube start
    I get above logs.

image

I have tried multiple times I tried with driver virtualbox but failes because virtual box and docker desktop cannot run together ( because 2 VM cannot run together on Ubuntu)

minikube status gives this

image

I have tried

  • reinstallng minikube
  • restarting the minikube container
  • tried minikube delete --all and minikube start
    Not working in all the above cases.

Attach the log file

logs.txt

Operating System

Ubuntu

Driver

Docker
The kubernetes needs to be enabled in Docker Desktop
Capture1

@spowelljr
Copy link
Member

@prakharporwal @loyaltytrooper @mesketh @dishkakrauch @jodusan @zidanomar @Arun-N @gio-shara-code

Here's a binary that has a fix I implemented (#15126) that should make minikube work with Docker Desktop Linux. Could some of you test it and confirm it works on your end, thanks.

curl -LO https://storage.googleapis.com/minikube-builds/15126/minikube-linux-amd64
chmod +x ./minikube-linux-amd64
./minikube-linux-amd64 delete
./minikube-linux-amd64 start

@tikarammardi
Copy link

@spowelljr I have tested with the above binary. minikube is working with Docker Desktop Linux
image
image

@spowelljr
Copy link
Member

@tikarammardi I'm not sure why your apiserver is stopped, it's running when I test it on my machine.

@Arun-N
Copy link

Arun-N commented Oct 25, 2022

@spowelljr Tested. Everything looks fine but I haven't tried running any pods on it yet.

@tikarammardi I'm not sure why your apiserver is stopped, it's running when I test it on my machine.

They are using minikube status, not ./minikube-linux-amd64 status

~/minikube-test » ./minikube-linux-amd64 delete
🙄  "minikube" profile does not exist, trying anyways.
💀  Removed all traces of the "minikube" cluster.
~/minikube-test » ./minikube-linux-amd64 start
😄  minikube v1.27.1 on Ubuntu 22.04
✨  Automatically selected the docker driver. Other choices: ssh, qemu2 (experimental)
📌  Using Docker driver with root privileges
❗  For an improved experience it's recommended to use Docker Engine instead of Docker Desktop.
Docker Engine installation instructions: https://docs.docker.com/engine/install/#server
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.25.2 preload ...
    > preloaded-images-k8s-v18-v1...:  385.41 MiB / 385.41 MiB  100.00% 7.42 Mi
    > gcr.io/k8s-minikube/kicbase...:  386.72 MiB / 386.72 MiB  100.00% 3.59 Mi
    > gcr.io/k8s-minikube/kicbase...:  0 B [_____________________] ?% ? p/s 49s
🔥  Creating docker container (CPUs=2, Memory=3769MB) ...
🐳  Preparing Kubernetes v1.25.2 on Docker 20.10.18 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
~/minikube-test » ./minikube-linux-amd64 status
E1025 12:55:18.949249   14810 status.go:415] kubeconfig endpoint: got: 192.168.49.2:8443, want: 127.0.0.1:42825
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Misconfigured


WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
~/minikube-test » ./minikube-linux-amd64 update-context
🎉  "minikube" context has been updated to point to 127.0.0.1:42825
💗  Current context is "minikube"
~/minikube-test » ./minikube-linux-amd64 status
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
~/minikube-test » ./minikube-linux-amd64 dashboard --port 3456 --url
🔌  Enabling dashboard ...
    ▪ Using image docker.io/kubernetesui/metrics-scraper:v1.0.8
    ▪ Using image docker.io/kubernetesui/dashboard:v2.7.0
🤔  Verifying dashboard health ...
🚀  Launching proxy ...
🤔  Verifying proxy health ...
http://127.0.0.1:3456/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/

@Vickydev95
Copy link

@prakharporwal @loyaltytrooper @mesketh @dishkakrauch @jodusan @zidanomar @Arun-N @gio-shara-code

Here's a binary that has a fix I implemented (#15126) that should make minikube work with Docker Desktop Linux. Could some of you test it and confirm it works on your end, thanks.

curl -LO https://storage.googleapis.com/minikube-builds/15126/minikube-linux-amd64
chmod +x ./minikube-linux-amd64
./minikube-linux-amd64 delete
./minikube-linux-amd64 start

this worked for me thanks

@spowelljr
Copy link
Member

Change is merged and will be included in the next release

@imflash217
Copy link

Hi,
I am having the same issue with Docker Desktop on MacOS 13.0
I tried all above approaches but none is working either with --driver=docker or --driver=virtualbox
Stack:

  1. MacOS 13.0
  2. Docker Desktop
  3. Virtualbox 7.0
  4. ❯ minikube version
    minikube version: v1.27.1
    commit: fe869b5d4da11ba318eb84a3ac00f336411de7ba
  5. Logs:
❯ minikube start
😄  minikube v1.27.1 on Darwin 13.0
✨  Automatically selected the docker driver. Other choices: virtualbox, ssh
❗  docker is currently using the stargz storage driver, consider switching to overlay2 for better performance
📌  Using Docker Desktop driver with root privileges
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
    > gcr.io/k8s-minikube/kicbase:  0 B [________________________] ?% ? p/s 23s
🔥  Creating docker container (CPUs=2, Memory=2200MB) ...
✋  Stopping node "minikube"  ...
🛑  Powering off "minikube" via SSH ...
🔥  Deleting "minikube" in docker ...
🤦  StartHost failed, but will try again: creating host: create: provisioning: ssh command error:
command : sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
err     : Process exited with status 1
output  : [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target
tee: /lib/systemd/system/docker.service.new: No such file or directory

🔥  Creating docker container (CPUs=2, Memory=2200MB) ...
😿  Failed to start docker container. Running "minikube delete" may fix it: creating host: create: provisioning: ssh command error:
command : sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
err     : Process exited with status 1
output  : tee: /lib/systemd/system/docker.service.new: No such file or directory
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target

❗  Startup with docker driver failed, trying with alternate driver virtualbox: Failed to start host: creating host: create: provisioning: ssh command error:
command : sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
err     : Process exited with status 1
output  : tee: /lib/systemd/system/docker.service.new: No such file or directory
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target

🔥  Deleting "minikube" in docker ...
🔥  Deleting container "minikube" ...
🔥  Removing /Users/imflash217/.minikube/machines/minikube ...
💀  Removed all traces of the "minikube" cluster.
👍  Starting control plane node minikube in cluster minikube
🔥  Creating virtualbox VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
🔥  Deleting "minikube" in virtualbox ...
🤦  StartHost failed, but will try again: creating host: create: creating: Error setting up host only network on machine start: The host-only adapter we just created is not visible. This is a well known VirtualBox bug. You might want to uninstall it and reinstall at least version 5.0.12 that is is supposed to fix this issue
🔥  Creating virtualbox VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
😿  Failed to start virtualbox VM. Running "minikube delete" may fix it: creating host: create: creating: Error setting up host only network on machine start: The host-only adapter we just created is not visible. This is a well known VirtualBox bug. You might want to uninstall it and reinstall at least version 5.0.12 that is is supposed to fix this issue

❌  Exiting due to GUEST_PROVISION: Failed to start host: creating host: create: provisioning: ssh command error:
command : sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
err     : Process exited with status 1
output  : tee: /lib/systemd/system/docker.service.new: No such file or directory
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure



# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target


╭───────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                           │
│    😿  If the above advice does not help, please let us know:                             │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose                           │
│                                                                                           │
│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │
│                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────╯

Any help on this!
Thanks

@spowelljr
Copy link
Member

Hi @imflash217, this issue is specifically for Docker Desktop Linux. Since you're using Docker Desktop macOS I'd recommend opening a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
co/docker-driver Issues related to kubernetes in container kind/bug Categorizes issue or PR as related to a bug. needs-faq-entry Things that could use documentation in a FAQ os/linux priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done.
Projects
None yet