Fix --base-image caching for images specified by name:tag #11603

ilya-zuyev · 2021-06-08T00:48:47Z

This PR fixes an issue when the image set by --base-image is downloaded on start even if it's already present in minikube cache.

Before:

➜  minikube git:(master) ✗ time m start  --base-image=docker.io/ilyazuyev/kicbase:jun4_0 
😄  minikube v1.21.0-beta.0 on Darwin 11.3.1
✨  Automatically selected the docker driver. Other choices: virtualbox, ssh
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
    > index.docker.io/ilyazuyev/k...: 359.08 MiB / 359.08 MiB  100.00% 5.63 MiB
🔥  Creating docker container (CPUs=2, Memory=4000MB) ...
🐳  Preparing Kubernetes v1.20.7 on Docker 20.10.7 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
out/minikube start --base-image=docker.io/ilyazuyev/kicbase:jun4_0  11.13s user 7.60s system 13% cpu 2:23.21 total

➜  minikube git:(master) ✗ m delete --all
🔥  Deleting "minikube" in docker ...
🔥  Removing /Users/izuyev/.minikube/machines/minikube ...
💀  Removed all traces of the "minikube" cluster.
🔥  Successfully deleted all profiles

➜  minikube git:(master) ✗ time m start  --base-image=docker.io/ilyazuyev/kicbase:jun4_0 
😄  minikube v1.21.0-beta.0 on Darwin 11.3.1
✨  Automatically selected the docker driver. Other choices: virtualbox, ssh
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
    > index.docker.io/ilyazuyev/k...: 359.08 MiB / 359.08 MiB  100.00% 5.73 MiB
🔥  Creating docker container (CPUs=2, Memory=4000MB) ...
🐳  Preparing Kubernetes v1.20.7 on Docker 20.10.7 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
out/minikube start --base-image=docker.io/ilyazuyev/kicbase:jun4_0  13.43s user 10.21s system 15% cpu 2:36.20 total

After:

➜  minikube git:(ilyaz/fix_no_digest_caching) ✗ time m start  --base-image=docker.io/ilyazuyev/kicbase:jun4_0                              
😄  minikube v1.21.0-beta.0 on Darwin 11.3.1
✨  Automatically selected the docker driver. Other choices: virtualbox, ssh
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.20.7 preload ...
    > preloaded-images-k8s-v11-v1...: 492.20 MiB / 492.20 MiB  100.00% 6.57 MiB
    > index.docker.io/ilyazuyev/k...: 359.08 MiB / 359.08 MiB  100.00% 2.91 MiB
🔥  Creating docker container (CPUs=2, Memory=4000MB) ...
🐳  Preparing Kubernetes v1.20.7 on Docker 20.10.7 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
out/minikube start --base-image=docker.io/ilyazuyev/kicbase:jun4_0  16.40s user 13.19s system 13% cpu 3:41.00 total


➜  minikube git:(ilyaz/fix_no_digest_caching) ✗ m delete --all
🔥  Deleting "minikube" in docker ...
🔥  Removing /Users/izuyev/.minikube/machines/minikube ...
💀  Removed all traces of the "minikube" cluster.
🔥  Successfully deleted all profiles


➜  minikube git:(ilyaz/fix_no_digest_caching) ✗ time m start  --base-image=docker.io/ilyazuyev/kicbase:jun4_0 
😄  minikube v1.21.0-beta.0 on Darwin 11.3.1
✨  Automatically selected the docker driver. Other choices: virtualbox, ssh
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=4000MB) ...
🐳  Preparing Kubernetes v1.20.7 on Docker 20.10.7 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
out/minikube start --base-image=docker.io/ilyazuyev/kicbase:jun4_0  9.32s user 5.43s system 15% cpu 1:36.86 total

Note, after the fix, the base image is not downloaded.

There is an exception for latest, when --base-image=<img>:latest format is used the image is always pulled

ilya-zuyev · 2021-06-08T00:49:05Z

/ok-to-test

minikube-pr-bot · 2021-06-08T02:18:56Z

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 11603) |
+----------------+----------+---------------------+
| minikube start | 48.6s    | 46.8s               |
| enable ingress | 34.1s    | 35.0s               |
+----------------+----------+---------------------+

Times for minikube start: 47.7s 50.7s 47.3s 46.2s 51.1s
Times for minikube (PR 11603) start: 46.7s 45.8s 47.6s 47.5s 46.6s

Times for minikube ingress: 34.7s 33.8s 33.9s 35.7s 32.3s
Times for minikube (PR 11603) ingress: 37.1s 34.2s 34.4s 34.8s 34.7s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 11603) |
+----------------+----------+---------------------+
| minikube start | 22.0s    | 21.6s               |
| enable ingress | 29.8s    | 32.3s               |
+----------------+----------+---------------------+

Times for minikube (PR 11603) start: 20.9s 22.4s 21.3s 21.1s 22.3s
Times for minikube start: 23.0s 21.5s 22.5s 21.8s 21.3s

Times for minikube ingress: 29.0s 37.0s 27.5s 26.5s 29.0s
Times for minikube (PR 11603) ingress: 36.5s 32.0s 30.0s 34.0s 29.0s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 11603) |
+----------------+----------+---------------------+
| minikube start | 41.5s    | 45.7s               |
| enable ingress |          |                     |
+----------------+----------+---------------------+

Times for minikube start: 31.2s 43.3s 43.2s 46.2s 43.4s
Times for minikube (PR 11603) start: 47.1s 47.0s 42.7s 43.4s 48.0s

medyagh

thank you for this PR this seems interesting !

pkg/minikube/download/image.go

medyagh · 2021-06-21T18:57:08Z

@ilya-zuyev please pull upstream on this PR, there was a another PR that touched same part of this code might need to test with that

minikube-bot · 2021-06-24T20:24:12Z

These are the flake rates of all failed tests on Docker_Linux.

Failed Tests	Flake Rate (%)
TestScheduledStopUnix	4.00 (chart)

minikube-pr-bot · 2021-06-24T20:39:47Z

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 11603) |
+----------------+----------+---------------------+
| minikube start | 47.5s    | 47.5s               |
| enable ingress | 35.3s    | 34.6s               |
+----------------+----------+---------------------+

Times for minikube ingress: 36.8s 33.8s 36.7s 34.3s 34.7s
Times for minikube (PR 11603) ingress: 34.2s 34.3s 35.2s 33.8s 35.8s

Times for minikube (PR 11603) start: 46.2s 48.0s 45.9s 46.8s 50.6s
Times for minikube start: 48.3s 47.3s 47.0s 47.9s 47.1s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 11603) |
+----------------+----------+---------------------+
| minikube start | 21.6s    | 21.6s               |
| enable ingress | 29.5s    | 33.3s               |
+----------------+----------+---------------------+

Times for minikube start: 21.7s 20.8s 22.4s 21.4s 21.4s
Times for minikube (PR 11603) start: 20.4s 22.1s 21.6s 22.1s 21.6s

Times for minikube ingress: 29.0s 29.6s 33.0s 26.5s 29.5s
Times for minikube (PR 11603) ingress: 36.5s 27.5s 29.0s 37.0s 36.5s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 11603) |
+----------------+----------+---------------------+
| minikube start | 45.9s    | 44.0s               |
| enable ingress |          |                     |
+----------------+----------+---------------------+

Times for minikube start: 48.3s 43.5s 47.4s 43.1s 47.3s
Times for minikube (PR 11603) start: 43.0s 43.0s 47.8s 43.3s 42.7s

ilya-zuyev · 2021-06-25T18:01:23Z

@ilya-zuyev please pull upstream on this PR, there was a another PR that touched same part of this code might need to test with that

done!

medyagh

lets fix the review comments and hopefully cloud shell test will pass this time too (seems like it didnt get started)

pkg/minikube/download/image.go

minikube-bot · 2021-06-25T22:26:11Z

These are the flake rates of all failed tests on Docker_Linux.

Failed Tests	Flake Rate (%)
TestRunningBinaryUpgrade	0.92 (chart)
TestScheduledStopUnix	4.59 (chart)

minikube-pr-bot · 2021-06-25T22:33:18Z

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 11603) |
+----------------+----------+---------------------+
| minikube start | 49.1s    | 47.3s               |
| enable ingress | 39.7s    | 38.3s               |
+----------------+----------+---------------------+

Times for minikube start: 51.0s 48.5s 47.1s 51.2s 47.4s
Times for minikube (PR 11603) start: 47.5s 47.1s 47.3s 46.7s 47.9s

Times for minikube (PR 11603) ingress: 43.2s 34.3s 43.2s 34.7s 36.3s
Times for minikube ingress: 42.3s 41.8s 36.7s 35.3s 42.2s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 11603) |
+----------------+----------+---------------------+
| minikube start | 22.2s    | 22.1s               |
| enable ingress | 34.5s    | 33.8s               |
+----------------+----------+---------------------+

Times for minikube start: 22.0s 21.8s 22.1s 22.3s 22.7s
Times for minikube (PR 11603) start: 22.4s 21.4s 21.5s 22.2s 22.8s

Times for minikube ingress: 36.0s 31.5s 34.5s 35.0s 35.5s
Times for minikube (PR 11603) ingress: 35.0s 32.0s 33.0s 37.0s 32.0s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 11603) |
+----------------+----------+---------------------+
| minikube start | 44.3s    | 43.4s               |
| enable ingress |          |                     |
+----------------+----------+---------------------+

Times for minikube start: 42.9s 43.8s 43.7s 43.8s 47.2s
Times for minikube (PR 11603) start: 44.0s 43.4s 43.8s 43.3s 42.7s

k8s-ci-robot · 2021-06-25T22:37:48Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ilya-zuyev, medyagh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [ilya-zuyev,medyagh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ilya-zuyev added 2 commits June 7, 2021 16:37

fix kicbase cache check

557d3a5

Add an exception for 'latest' images

6261f9d

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 8, 2021

k8s-ci-robot requested review from medyagh and RA489 June 8, 2021 00:48

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 8, 2021

k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Jun 8, 2021

medyagh requested changes Jun 18, 2021

View reviewed changes

pkg/minikube/download/image.go Show resolved Hide resolved

medyagh requested changes Jun 18, 2021

View reviewed changes

pkg/minikube/download/image.go Show resolved Hide resolved

Merge branch 'master' into ilyaz/fix_no_digest_caching

e910260

medyagh requested changes Jun 25, 2021

View reviewed changes

pkg/minikube/download/image.go Show resolved Hide resolved

pkg/minikube/download/image.go Show resolved Hide resolved

ilya-zuyev added 2 commits June 25, 2021 13:28

add comments

049c426

fix error message

7875965

medyagh approved these changes Jun 25, 2021

View reviewed changes

ilya-zuyev merged commit 6ac71f1 into kubernetes:master Jun 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix --base-image caching for images specified by name:tag #11603

Fix --base-image caching for images specified by name:tag #11603

ilya-zuyev commented Jun 8, 2021

ilya-zuyev commented Jun 8, 2021

minikube-pr-bot commented Jun 8, 2021

medyagh left a comment

medyagh commented Jun 21, 2021

minikube-bot commented Jun 24, 2021

minikube-pr-bot commented Jun 24, 2021

ilya-zuyev commented Jun 25, 2021

medyagh left a comment

minikube-bot commented Jun 25, 2021

minikube-pr-bot commented Jun 25, 2021

k8s-ci-robot commented Jun 25, 2021

Fix --base-image caching for images specified by name:tag #11603

Fix --base-image caching for images specified by name:tag #11603

Conversation

ilya-zuyev commented Jun 8, 2021

Before:

After:

ilya-zuyev commented Jun 8, 2021

minikube-pr-bot commented Jun 8, 2021

medyagh left a comment

Choose a reason for hiding this comment

medyagh commented Jun 21, 2021

minikube-bot commented Jun 24, 2021

minikube-pr-bot commented Jun 24, 2021

ilya-zuyev commented Jun 25, 2021

medyagh left a comment

Choose a reason for hiding this comment

minikube-bot commented Jun 25, 2021

minikube-pr-bot commented Jun 25, 2021

k8s-ci-robot commented Jun 25, 2021