Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker minikube network not automatically removed at minikube stop or start #10065

Closed
srilumpa opened this issue Dec 30, 2020 · 2 comments
Closed
Labels
area/networking networking issues co/docker-driver Issues related to kubernetes in container kind/bug Categorizes issue or PR as related to a bug. triage/duplicate Indicates an issue is a duplicate of other open issue.

Comments

@srilumpa
Copy link

Hi,

I upgraded this morning my minikube cluster from version 1.14.0 to 1.16.0 and it seems the docker network usage is not fully handled. I was unable to restart my minikube cluster without manually deleting the docker network created by minikube.

Steps to reproduce the issue:

  1. Having a running minikube clusterwith minikube in version 1.14.0
  2. minikube stop && minikube start

Full output of failed command:

✋  Stopping node "minikube"  ...
🛑  Powering off "minikube" via SSH ...
🛑  1 nodes stopped.
😄  minikube v1.16.0 on Ubuntu 20.04
🆕  Kubernetes 1.20.0 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.20.0
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🔄  Restarting existing docker container for "minikube" ...
🐳  Preparing Kubernetes v1.18.3 on Docker 19.03.8 ...| E1230 09:47:23.082385   13400 start.go:99] Unable to get host IP: network inspect: error parsing network inspect output: "{\"Name\": \"minikube\",\"Driver\": \"bridge\",\"Subnet\": \"192.168.49.0/24\",\"Gateway\": \"192.168.49.1\",\"MTU\": , \"ContainerIPs\": [\"192.168.49.2/24\"]}\n"


❌  Exiting due to GUEST_START: Failed to setup kubeconfig: network inspect: error parsing network inspect output: "{\"Name\": \"minikube\",\"Driver\": \"bridge\",\"Subnet\": \"192.168.49.0/24\",\"Gateway\": \"192.168.49.1\",\"MTU\": , \"ContainerIPs\": [\"192.168.49.2/24\"]}\n"

😿  If the above advice does not help, please let us know: 
👉  https://github.com/kubernetes/minikube/issues/new/choose

At that step, I simply deleted my cluster and recreated it:

  1. minikube delete && minikube start --cpus=4 --memory=8g --disk-size=40g

And it failed outputing the following:

😄  minikube v1.16.0 on Ubuntu 20.04
✨  Automatically selected the docker driver. Other choices: virtualbox, none
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.20.0 preload ...
    > preloaded-images-k8s-v8-v1....: 491.00 MiB / 491.00 MiB  100.00% 25.00 Mi
🔥  Creating docker container (CPUs=4, Memory=8192MB) ...
E1230 09:49:23.788509   15496 network_create.go:81] error while trying to create network create network minikube 192.168.49.0/24: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true minikube: exit status 1
stdout:

stderr:
Error response from daemon: network with name minikube already exists
❗  Unable to create dedicated network, this might result in cluster IP change after restart: un-retryable: create network minikube 192.168.49.0/24: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true minikube: exit status 1
stdout:

stderr:
Error response from daemon: network with name minikube already exists

🐳  Preparing Kubernetes v1.20.0 on Docker 20.10.0 ...| E1230 09:49:33.959739   15496 start.go:99] Unable to get host IP: network inspect: error parsing network inspect output: "{\"Name\": \"minikube\",\"Driver\": \"bridge\",\"Subnet\": \"192.168.49.0/24\",\"Gateway\": \"192.168.49.1\",\"MTU\": , \"ContainerIPs\": []}\n"


❌  Exiting due to GUEST_START: Failed to setup kubeconfig: network inspect: error parsing network inspect output: "{\"Name\": \"minikube\",\"Driver\": \"bridge\",\"Subnet\": \"192.168.49.0/24\",\"Gateway\": \"192.168.49.1\",\"MTU\": , \"ContainerIPs\": []}\n"

😿  If the above advice does not help, please let us know: 
👉  https://github.com/kubernetes/minikube/issues/new/choose

I looked into my docker networks and found the minikube network still existing. I removed it and was able to successfully start my new cluster:

  1. docker network rm minikube
  2. minikube start
😄  minikube v1.16.0 on Ubuntu 20.04
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🏃  Updating the running docker "minikube" container ...
🐳  Preparing Kubernetes v1.20.0 on Docker 20.10.0 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

I don't know if it is only linked to the upgrade and if it is still relevant with future upgrades but, in doubt, I wanted to report this issue.

@afbjorklund
Copy link
Collaborator

afbjorklund commented Dec 30, 2020

This is a side-effect of #10033, it is trying to inspect the old network - and then fails to parse the results.

Your workaround is the suggested one.

@afbjorklund afbjorklund added co/docker-driver Issues related to kubernetes in container kind/bug Categorizes issue or PR as related to a bug. triage/duplicate Indicates an issue is a duplicate of other open issue. labels Dec 30, 2020
@tstromberg tstromberg added triage/discuss Items for discussion priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. area/networking networking issues labels Jan 6, 2021
@medyagh
Copy link
Member

medyagh commented Jan 6, 2021

I belive this was fixed by this PR #10053
yet deleting minikube and starting it agian with old version should still work.

please feel free to open if u have more questions

@medyagh medyagh closed this as completed Jan 6, 2021
@tstromberg tstromberg removed priority/awaiting-more-evidence Lowest priority. Possibly useful, but not yet enough support to actually get it done. triage/discuss Items for discussion labels Jan 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/networking networking issues co/docker-driver Issues related to kubernetes in container kind/bug Categorizes issue or PR as related to a bug. triage/duplicate Indicates an issue is a duplicate of other open issue.
Projects
None yet
Development

No branches or pull requests

4 participants