Make it possible to install addons from alternate image repositories

[jgoeres]

I was just trying to start my minikube (after it again became unresponsive to kubectl for some reason, which I finally reported as #8300) when I noticed that the ingress-controller add on was in ImagePullBackOff:

$ kubectl get pods -A
NAMESPACE       NAME                                        READY   STATUS             RESTARTS   AGE
kube-system     coredns-5d4dd4b4db-jrjgv                    1/1     Running            0          15m
[...]
kube-system     nginx-ingress-controller-657fd58d97-rf859   0/1     ImagePullBackOff   0          9m39s
kube-system     storage-provisioner                         1/1     Running            0          15m


$ kubectl describe pod -n kube-system nginx-ingress-controller-657fd58d97-rf859
Name:           nginx-ingress-controller-657fd58d97-rf859
[...]
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  3m43s                 default-scheduler  Successfully assigned kube-system/nginx-ingress-controller-657fd58d97-rf859 to m01
  Normal   Pulling    2m9s (x4 over 3m43s)  kubelet, m01       Pulling image "quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.26.1"
  Warning  Failed     2m9s (x4 over 3m42s)  kubelet, m01       Failed to pull image "quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.26.1": rpc error: code = Unknown desc = Error response from daemon: Get https://quay.io/v2/: EOF
  Warning  Failed     2m9s (x4 over 3m42s)  kubelet, m01       Error: ErrImagePull
  Warning  Failed     117s (x6 over 3m42s)  kubelet, m01       Error: ImagePullBackOff
  Normal   BackOff    104s (x7 over 3m42s)  kubelet, m01       Back-off pulling image "quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.26.1"

This problem was already reported here as well #8153

In general I think it should be possible for any addon that pulls images to specify an alternative image (not so much to use an alternative implementation, but to have backup registry) and an image pull secret. This seems currently not possible, at least for the ingress addon, I tried the addon configure command:

$ minikube addons configure ingress
* ingress has no available configuration options

Is there any way how this can be done?

That would allow people that use minikube in their internal process to host such images on internal registries, to not have a dependency on external systems.

[tstromberg]

There is technically an --image-repository flag to minikube start, but not every addon supports it. Ingress is one of these.

The other issue here is that there is no way to specify the image repository after minikube has started up.

[jgoeres]

THX for pointing out the --image-repository flag.
Alas, our internal registry does not allow anonymous pull access.
I just went over the various switches of minikube start - I didn't see an obvious way to pass in a registry secret/docker login credentials. Is there any way to do this?

[medyagh]

THX for pointing out the --image-repository flag.
Alas, our internal registry does not allow anonymous pull access.
I just went over the various switches of minikube start - I didn't see an obvious way to pass in a registry secret/docker login credentials. Is there any way to do this?

@jgoeres if you are using a private registery that needs creds, you would need to add that as a kuberentes secret so kuberentes can talk to it.

does this docs on using private registery help ?
https://minikube.sigs.k8s.io/docs/handbook/registry/

or this example in aws https://minikube.sigs.k8s.io/docs/tutorials/configuring_creds_for_aws_ecr/

[clementDele]

+1

I run my Minikube 1.11.0 on an offline machine. I was able to pull control-plane images from my private Docker registry with the --image-repository flag but it seems impossible to change the repository for the dashboard:v2.0.0 and metrics-scraper:v1.0.2. Despite the flag, they keep trying to pull from registry1.docker.io...

A dirty workaround would be to docker load those images directly on the Minikube VM i guess. This is obviously not a viable long term solution ....

Is there a better solution at the moment ? Thanks

[medyagh]

I would accept any PR that makes this possible

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[yehiyam]

I'm creating a PR for that
/assign