Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix minikube dashboard failing on macOS #15037

Merged
merged 1 commit into from
Sep 30, 2022

Conversation

spowelljr
Copy link
Member

@spowelljr spowelljr commented Sep 29, 2022

Fixes #15040

Problem:
Sometimes minikube dashboard would fail on macOS with the failing error:

$ minikube dashboard --alsologtostderr
I0929 13:34:36.468440   68393 out.go:296] Setting OutFile to fd 1 ...
I0929 13:34:36.468612   68393 out.go:348] isatty.IsTerminal(1) = true
I0929 13:34:36.468616   68393 out.go:309] Setting ErrFile to fd 2...
I0929 13:34:36.468620   68393 out.go:348] isatty.IsTerminal(2) = true
I0929 13:34:36.468683   68393 root.go:333] Updating PATH: /Users/powellsteven/.minikube/bin
I0929 13:34:36.468934   68393 mustload.go:65] Loading cluster: minikube
I0929 13:34:36.469326   68393 config.go:180] Loaded profile config "minikube": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.25.2
I0929 13:34:36.469839   68393 cli_runner.go:164] Run: docker container inspect minikube --format={{.State.Status}}
I0929 13:34:36.542243   68393 host.go:66] Checking if "minikube" exists ...
I0929 13:34:36.542568   68393 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" minikube
I0929 13:34:36.602713   68393 api_server.go:165] Checking apiserver status ...
I0929 13:34:36.602871   68393 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0929 13:34:36.602971   68393 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0929 13:34:36.666393   68393 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59958 SSHKeyPath:/Users/powellsteven/.minikube/machines/minikube/id_rsa Username:docker}
I0929 13:34:36.748542   68393 ssh_runner.go:195] Run: sudo egrep ^[0-9]+:freezer: /proc/4517/cgroup
W0929 13:34:36.756995   68393 api_server.go:176] unable to find freezer cgroup: sudo egrep ^[0-9]+:freezer: /proc/4517/cgroup: Process exited with status 1
stdout:

stderr:
I0929 13:34:36.757017   68393 api_server.go:240] Checking apiserver healthz at https://127.0.0.1:59962/healthz ...
I0929 13:34:36.763443   68393 api_server.go:266] https://127.0.0.1:59962/healthz returned 200:
ok
W0929 13:34:36.763514   68393 out.go:239] 🤔  Verifying dashboard health ...
🤔  Verifying dashboard health ...
I0929 13:34:36.774050   68393 service.go:214] Found service: &Service{ObjectMeta:{kubernetes-dashboard  kubernetes-dashboard  1a50361b-b019-4f47-bcb7-0b642cb221d6 429 0 2022-09-29 12:40:24 -0700 PDT <nil> <nil> map[addonmanager.kubernetes.io/mode:Reconcile k8s-app:kubernetes-dashboard kubernetes.io/minikube-addons:dashboard] map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"addonmanager.kubernetes.io/mode":"Reconcile","k8s-app":"kubernetes-dashboard","kubernetes.io/minikube-addons":"dashboard"},"name":"kubernetes-dashboard","namespace":"kubernetes-dashboard"},"spec":{"ports":[{"port":80,"targetPort":9090}],"selector":{"k8s-app":"kubernetes-dashboard"}}}
] [] [] [{kubectl-client-side-apply Update v1 2022-09-29 12:40:24 -0700 PDT FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}},"f:labels":{".":{},"f:addonmanager.kubernetes.io/mode":{},"f:k8s-app":{},"f:kubernetes.io/minikube-addons":{}}},"f:spec":{"f:internalTrafficPolicy":{},"f:ports":{".":{},"k:{\"port\":80,\"protocol\":\"TCP\"}":{".":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{},"f:sessionAffinity":{},"f:type":{}}} }]},Spec:ServiceSpec{Ports:[]ServicePort{ServicePort{Name:,Protocol:TCP,Port:80,TargetPort:{0 9090 },NodePort:0,AppProtocol:nil,},},Selector:map[string]string{k8s-app: kubernetes-dashboard,},ClusterIP:10.105.27.235,Type:ClusterIP,ExternalIPs:[],SessionAffinity:None,LoadBalancerIP:,LoadBalancerSourceRanges:[],ExternalName:,ExternalTrafficPolicy:,HealthCheckNodePort:0,PublishNotReadyAddresses:false,SessionAffinityConfig:nil,IPFamilyPolicy:*SingleStack,ClusterIPs:[10.105.27.235],IPFamilies:[IPv4],AllocateLoadBalancerNodePorts:nil,LoadBalancerClass:nil,InternalTrafficPolicy:*Cluster,},Status:ServiceStatus{LoadBalancer:LoadBalancerStatus{Ingress:[]LoadBalancerIngress{},},Conditions:[]Condition{},},}
W0929 13:34:36.774227   68393 out.go:239] 🚀  Launching proxy ...
🚀  Launching proxy ...
I0929 13:34:36.774319   68393 dashboard.go:152] Executing: /usr/local/bin/kubectl [/usr/local/bin/kubectl --context minikube proxy --port 0]
I0929 13:34:36.775983   68393 dashboard.go:157] Waiting for kubectl to output host:port ...
I0929 13:34:36.809478   68393 dashboard.go:175] proxy stdout: Starting to serve on 1 7.0.0.1:61546
W0929 13:34:36.809534   68393 out.go:239] 🤔  Verifying proxy health ...
🤔  Verifying proxy health ...
I0929 13:34:36.809574   68393 dashboard.go:212] http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ response: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL <nil>
I0929 13:34:36.809620   68393 retry.go:31] will retry after 110.466µs: checkURL: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL
I0929 13:34:36.809769   68393 dashboard.go:212] http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ response: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL <nil>
I0929 13:34:36.809784   68393 retry.go:31] will retry after 216.077µs: checkURL: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL
I0929 13:34:36.810075   68393 dashboard.go:212] http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ response: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL <nil>
I0929 13:34:36.810109   68393 retry.go:31] will retry after 262.026µs: checkURL: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL
I0929 13:34:36.810420   68393 dashboard.go:212] http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ response: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL <nil>
I0929 13:34:36.810439   68393 retry.go:31] will retry after 316.478µs: checkURL: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL
I0929 13:34:36.810815   68393 dashboard.go:212] http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ response: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL <nil>
I0929 13:34:36.810824   68393 retry.go:31] will retry after 468.098µs: checkURL: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL
I0929 13:34:36.811368   68393 dashboard.go:212] http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ response: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL <nil>
I0929 13:34:36.811380   68393 retry.go:31] will retry after 901.244µs: checkURL: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL

The root of the problem is in the line:

I0929 13:34:36.809478   68393 dashboard.go:175] proxy stdout: Starting to serve on 1 7.0.0.1:61546

The 2 is missing in 127.0.0.1 and is replaced with a space. This then fails the regex regexp.MustCompile(127.0.0.1(:| )\d{4,}) and results in an empty host address that can be seen in the following log:

I0929 13:34:36.811368   68393 dashboard.go:212] http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ response: Get "http:///api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/": http: no Host in request URL <nil>

Solution:

This seems to stem from closing the channels too early. Resulting in the error channel getting called and running return byte(' '), false, err, hence the space. By closing the channels in the defer the output is consistently the expected value.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Sep 29, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: spowelljr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 29, 2022
@medyagh medyagh merged commit b672a15 into kubernetes:master Sep 30, 2022
@spowelljr spowelljr deleted the fixDashboardCmd branch September 30, 2022 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

minikube dashboard failing on macOS
3 participants