Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multi-node: Add support for volumes using CSI addon #15829

Merged
merged 4 commits into from
Feb 14, 2023

Conversation

denisok
Copy link
Contributor

@denisok denisok commented Feb 10, 2023

fixes #12360

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Feb 10, 2023
@k8s-ci-robot
Copy link
Contributor

Welcome @denisok!

It looks like this is your first PR to kubernetes/minikube 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/minikube has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Feb 10, 2023
@k8s-ci-robot
Copy link
Contributor

Hi @denisok. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Feb 10, 2023
@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@denisok denisok marked this pull request as ready for review February 10, 2023 23:00
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 10, 2023
containers:
- name: csi-external-health-monitor-agent
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -92,12 +96,24 @@ metadata:
rules:
# Only one of the following rules for endpoints or leases is required based on
# what is set for `--leader-election-type`. Endpoints are deprecated in favor of Leases.
- apiGroups: [""]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mind adding output of minikube Before / After this PR?

also, it would be nice to add a section on multinode use example in our website

https://minikube.sigs.k8s.io/docs/tutorials/volume_snapshots_and_csi/

@medyagh
Copy link
Member

medyagh commented Feb 12, 2023

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Feb 12, 2023
@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15829) |
+----------------+----------+---------------------+
| minikube start | 51.1s    | 53.2s               |
| enable ingress | 26.0s    | 25.1s               |
+----------------+----------+---------------------+

Times for minikube ingress: 27.7s 26.7s 24.7s 26.7s 24.2s
Times for minikube (PR 15829) ingress: 28.2s 23.7s 24.2s 25.2s 24.2s

Times for minikube (PR 15829) start: 55.0s 53.9s 53.3s 49.9s 53.7s
Times for minikube start: 49.8s 50.1s 55.1s 50.8s 49.6s

docker driver with docker runtime

+-------------------+----------+---------------------+
|      COMMAND      | MINIKUBE | MINIKUBE (PR 15829) |
+-------------------+----------+---------------------+
| minikube start    | 26.2s    | 26.2s               |
| ⚠️  enable ingress | 33.7s    | 44.6s ⚠️             |
+-------------------+----------+---------------------+

Times for minikube start: 25.1s 25.6s 26.2s 27.4s 26.9s
Times for minikube (PR 15829) start: 26.8s 26.2s 26.9s 26.7s 24.5s

Times for minikube ingress: 20.0s 22.0s 23.0s 21.0s 82.6s
Times for minikube (PR 15829) ingress: 19.6s 20.0s 81.6s 81.6s 20.1s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15829) |
+----------------+----------+---------------------+
| minikube start | 21.4s    | 21.4s               |
| enable ingress | 38.7s    | 32.1s               |
+----------------+----------+---------------------+

Times for minikube (PR 15829) start: 20.8s 21.1s 21.6s 21.7s 22.0s
Times for minikube start: 21.7s 21.4s 22.3s 20.9s 20.9s

Times for minikube ingress: 80.0s 18.6s 31.6s 31.6s 31.6s
Times for minikube (PR 15829) ingress: 33.0s 32.5s 31.6s 31.6s 31.6s

@minikube-pr-bot
Copy link

These are the flake rates of all failed tests.

Environment Failed Tests Flake Rate (%)
Hyperkit_macOS TestStartStop/group/embed-certs/serial/AddonExistsAfterStop (gopogh) 2.10 (chart)
Hyperkit_macOS TestStartStop/group/embed-certs/serial/Pause (gopogh) 2.10 (chart)
Hyperkit_macOS TestStartStop/group/embed-certs/serial/SecondStart (gopogh) 2.10 (chart)
Hyperkit_macOS TestStartStop/group/embed-certs/serial/VerifyKubernetesImages (gopogh) 2.10 (chart)
Hyperkit_macOS TestStartStop/group/embed-certs/serial/UserAppExistsAfterStop (gopogh) 2.80 (chart)
KVM_Linux TestMultiNode/serial/StartAfterStop (gopogh) 6.25 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressDNSAddonActivation (gopogh) 98.03 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressAddonActivation (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressAddons (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/StartLegacyK8sCluster (gopogh) 100.00 (chart)
Docker_macOS TestKubernetesUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestMissingContainerUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestRunningBinaryUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/AddonExistsAfterStop (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/DeployApp (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/EnableAddonWhileActive (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/FirstStart (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/SecondStart (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/UserAppExistsAfterStop (gopogh) 100.00 (chart)
Docker_macOS TestStoppedBinaryUpgrade/Upgrade (gopogh) 100.00 (chart)
KVM_Linux_containerd TestPreload (gopogh) 100.00 (chart)

To see the flake rates of all tests by environment, click here.

@denisok
Copy link
Contributor Author

denisok commented Feb 12, 2023

do you mind adding output of minikube Before / After this PR?

sure, here evertyhing with ./out/ is a new version:

[dkondratenko@denlen minikube]$ minikube addons enable volumesnapshots 
💡  volumesnapshots is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
    ▪ Using image k8s.gcr.io/sig-storage/snapshot-controller:v4.0.0
🌟  The 'volumesnapshots' addon is enabled
[dkondratenko@denlen minikube]$ minikube addons disable volumesnapshots 
🌑  "The 'volumesnapshots' addon is disabled

[dkondratenko@denlen minikube]$ ./out/minikube addons enable volumesnapshots 
💡  volumesnapshots is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
    ▪ Using image registry.k8s.io/sig-storage/snapshot-controller:v6.1.0
🌟  The 'volumesnapshots' addon is enabled
[dkondratenko@denlen minikube]$ ./out/minikube addons disable volumesnapshots 
🌑  "The 'volumesnapshots' addon is disabled

[dkondratenko@denlen minikube]$ minikube addons enable csi-hostpath-driver
💡  csi-hostpath-driver is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
❗  [WARNING] For full functionality, the 'csi-hostpath-driver' addon requires the 'volumesnapshots' addon to be enabled.

You can enable 'volumesnapshots' addon by running: 'minikube addons enable volumesnapshots'

    ▪ Using image k8s.gcr.io/sig-storage/csi-snapshotter:v4.0.0
    ▪ Using image k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.0.1
    ▪ Using image k8s.gcr.io/sig-storage/livenessprobe:v2.2.0
    ▪ Using image k8s.gcr.io/sig-storage/csi-provisioner:v2.1.0
    ▪ Using image k8s.gcr.io/sig-storage/csi-external-health-monitor-controller:v0.2.0
    ▪ Using image k8s.gcr.io/sig-storage/hostpathplugin:v1.6.0
    ▪ Using image k8s.gcr.io/sig-storage/csi-resizer:v1.1.0
    ▪ Using image k8s.gcr.io/sig-storage/csi-attacher:v3.1.0
    ▪ Using image k8s.gcr.io/sig-storage/csi-external-health-monitor-agent:v0.2.0
🔎  Verifying csi-hostpath-driver addon...
🌟  The 'csi-hostpath-driver' addon is enabled
[dkondratenko@denlen minikube]$ minikube addons disable csi-hostpath-driver
🌑  "The 'csi-hostpath-driver' addon is disabled

[dkondratenko@denlen minikube]$ ./out/minikube addons enable csi-hostpath-driver
💡  csi-hostpath-driver is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
❗  [WARNING] For full functionality, the 'csi-hostpath-driver' addon requires the 'volumesnapshots' addon to be enabled.

You can enable 'volumesnapshots' addon by running: 'minikube addons enable volumesnapshots'

    ▪ Using image registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.6.0
    ▪ Using image registry.k8s.io/sig-storage/hostpathplugin:v1.9.0
    ▪ Using image registry.k8s.io/sig-storage/livenessprobe:v2.8.0
    ▪ Using image registry.k8s.io/sig-storage/csi-resizer:v1.6.0
    ▪ Using image registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0
    ▪ Using image registry.k8s.io/sig-storage/csi-provisioner:v3.3.0
    ▪ Using image registry.k8s.io/sig-storage/csi-attacher:v4.0.0
    ▪ Using image registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.7.0
🔎  Verifying csi-hostpath-driver addon...
🌟  The 'csi-hostpath-driver' addon is enabled
[dkondratenko@denlen minikube]$ ./out/minikube addons disable csi-hostpath-driver
🌑  "The 'csi-hostpath-driver' addon is disabled

also, it would be nice to add a section on multinode use example in our website

https://minikube.sigs.k8s.io/docs/tutorials/volume_snapshots_and_csi/
yes, will do later.

@denisok
Copy link
Contributor Author

denisok commented Feb 12, 2023

I see some failures related to csi, will check it.

@denisok
Copy link
Contributor Author

denisok commented Feb 13, 2023

OK, I looked to some of the tests, and those failures related to current state: Pending for many containers not just csi drivers. Same in other PRs currently open.

I tried different confirurations and don't see any issues with kvm or podman driver,

Once I seen the issue minikube starting with podman and cri-o - Node wasn't ready and thus containers were in Pending state, looks like #15676 .

Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Mon, 13 Feb 2023 19:39:17 +0100   Mon, 13 Feb 2023 19:39:12 +0100   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Mon, 13 Feb 2023 19:39:17 +0100   Mon, 13 Feb 2023 19:39:12 +0100   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Mon, 13 Feb 2023 19:39:17 +0100   Mon, 13 Feb 2023 19:39:12 +0100   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            False   Mon, 13 Feb 2023 19:39:17 +0100   Mon, 13 Feb 2023 19:39:12 +0100   KubeletNotReady              container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/cni/net.d/. Has your network provider started?

But it was only once and I can't reproduce it.

PR / functional_podman_ubuntu (pull_request) fails due to network issue with GH or network:

10s
Run curl -LO https://github.com/medyagh/gopogh/releases/download/v0.13.0/gopogh-linux-amd64
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

  0 3848k    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 73 3848k   73 2831k    0     0  1985k      0  0:00:01  0:00:01 --:--:-- 2686k
 73 3848k   73 2831k    0     0  1166k      0  0:00:03  0:00:02  0:00:01 1377k
 73 3848k   73 2831k    0     0   825k      0  0:00:04  0:00:03  0:00:01  926k
 73 3848k   73 2831k    0     0   639k      0  0:00:06  0:00:04  0:00:02  697k
 73 3848k   73 2831k    0     0   521k      0  0:00:07  0:00:05  0:00:02  559k
 73 3848k   73 2831k    0     0   440k      0  0:00:08  0:00:06  0:00:02     0
 73 3848k   73 2831k    0     0   380k      0  0:00:10  0:00:07  0:00:03     0
 73 3848k   73 2831k    0     0   335k      0  0:00:11  0:00:08  0:00:03     0
 73 3848k   73 2831k    0     0   300k      0  0:00:12  0:00:09  0:00:03     0
 73 3848k   73 2831k    0     0   272k      0  0:00:14  0:00:10  0:00:04     0
 73 3848k   73 2831k    0     0   272k      0  0:00:14  0:00:10  0:00:04     0
curl: (18) transfer closed with 1041185 bytes remaining to read
Error: Process completed with exit code 18.

needs re-run.

@denisok
Copy link
Contributor Author

denisok commented Feb 13, 2023

Hi @medyagh,

I added docs and checked tests. So far I found no regressions.

Thanks,
Denys

@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15829) |
+----------------+----------+---------------------+
| minikube start | 53.4s    | 54.1s               |
| enable ingress | 26.6s    | 27.1s               |
+----------------+----------+---------------------+

Times for minikube start: 54.0s 53.6s 53.0s 52.9s 53.3s
Times for minikube (PR 15829) start: 55.4s 53.4s 55.4s 52.8s 53.3s

Times for minikube ingress: 25.7s 27.7s 29.2s 24.2s 26.4s
Times for minikube (PR 15829) ingress: 28.2s 25.2s 25.7s 27.7s 28.7s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15829) |
+----------------+----------+---------------------+
| minikube start | 26.3s    | 27.0s               |
| enable ingress | 21.3s    | 21.1s               |
+----------------+----------+---------------------+

Times for minikube start: 26.4s 26.0s 25.9s 26.6s 26.6s
Times for minikube (PR 15829) start: 27.2s 26.0s 26.6s 27.8s 27.2s

Times for minikube ingress: 21.1s 20.1s 22.1s 21.1s 22.1s
Times for minikube (PR 15829) ingress: 20.1s 22.1s 21.1s 20.1s 22.1s

docker driver with containerd runtime

+-------------------+----------+---------------------+
|      COMMAND      | MINIKUBE | MINIKUBE (PR 15829) |
+-------------------+----------+---------------------+
| minikube start    | 22.2s    | 22.1s               |
| ⚠️  enable ingress | 30.0s    | 46.8s ⚠️             |
+-------------------+----------+---------------------+

Times for minikube start: 22.3s 21.8s 23.1s 21.9s 22.0s
Times for minikube (PR 15829) start: 21.5s 22.0s 22.3s 22.4s 22.3s

Times for minikube ingress: 32.6s 32.6s 20.6s 31.6s 32.6s
Times for minikube (PR 15829) ingress: 31.6s 20.1s 81.1s 80.6s 20.6s

@denisok denisok requested review from medyagh and removed request for prezha, afbjorklund and sharifelgamal February 13, 2023 21:46
@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15829) |
+----------------+----------+---------------------+
| minikube start | 56.2s    | 55.8s               |
| enable ingress | 27.5s    | 27.7s               |
+----------------+----------+---------------------+

Times for minikube (PR 15829) start: 57.1s 55.0s 54.5s 56.7s 56.0s
Times for minikube start: 55.2s 56.2s 54.9s 58.5s 56.3s

Times for minikube ingress: 28.4s 29.8s 28.9s 24.8s 25.8s
Times for minikube (PR 15829) ingress: 28.4s 28.9s 24.3s 27.9s 28.9s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15829) |
+----------------+----------+---------------------+
| minikube start | 28.3s    | 27.7s               |
| enable ingress | 33.5s    | 33.3s               |
+----------------+----------+---------------------+

Times for minikube start: 28.5s 28.9s 28.7s 27.0s 28.4s
Times for minikube (PR 15829) start: 27.2s 27.7s 28.9s 27.1s 27.5s

Times for minikube ingress: 21.1s 22.6s 82.6s 22.2s 19.1s
Times for minikube (PR 15829) ingress: 82.2s 20.2s 21.1s 21.2s 22.1s

docker driver with containerd runtime

+-------------------+----------+---------------------+
|      COMMAND      | MINIKUBE | MINIKUBE (PR 15829) |
+-------------------+----------+---------------------+
| minikube start    | 22.6s    | 23.3s               |
| ⚠️  enable ingress | 29.8s    | 37.2s ⚠️             |
+-------------------+----------+---------------------+

Times for minikube (PR 15829) ingress: 79.6s 21.6s 31.6s 20.6s 32.7s
Times for minikube ingress: 31.6s 32.6s 32.7s 31.6s 20.6s

Times for minikube (PR 15829) start: 23.2s 23.1s 23.4s 23.5s 23.1s
Times for minikube start: 22.5s 22.7s 23.6s 21.9s 22.5s

@minikube-pr-bot
Copy link

These are the flake rates of all failed tests.

Environment Failed Tests Flake Rate (%)
Docker_Windows TestRunningBinaryUpgrade (gopogh) 0.00 (chart)
Docker_Linux TestPause/serial/SecondStartNoReconfiguration (gopogh) 7.02 (chart)
Hyper-V_Windows TestPause/serial/SecondStartNoReconfiguration (gopogh) 56.25 (chart)
Hyper-V_Windows TestNoKubernetes/serial/StartWithK8s (gopogh) 62.89 (chart)
Hyper-V_Windows TestRunningBinaryUpgrade (gopogh) 64.95 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressDNSAddonActivation (gopogh) 98.15 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressAddonActivation (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressAddons (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/StartLegacyK8sCluster (gopogh) 100.00 (chart)
Docker_macOS TestKubernetesUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestMissingContainerUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestRunningBinaryUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/AddonExistsAfterStop (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/DeployApp (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/EnableAddonWhileActive (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/FirstStart (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/SecondStart (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/UserAppExistsAfterStop (gopogh) 100.00 (chart)
Docker_macOS TestStoppedBinaryUpgrade/Upgrade (gopogh) 100.00 (chart)
Docker_Windows TestFunctional/parallel/ServiceCmd (gopogh) 100.00 (chart)
Hyper-V_Windows TestMultiNode/serial/PingHostFrom2Pods (gopogh) 100.00 (chart)
Hyper-V_Windows TestMultiNode/serial/RestartKeepsNodes (gopogh) 100.00 (chart)
Hyper-V_Windows TestStoppedBinaryUpgrade/Upgrade (gopogh) 100.00 (chart)
KVM_Linux_containerd TestPreload (gopogh) 100.00 (chart)

To see the flake rates of all tests by environment, click here.

@minikube-pr-bot
Copy link

These are the flake rates of all failed tests.

Environment Failed Tests Flake Rate (%)
Docker_Windows TestRunningBinaryUpgrade (gopogh) 0.00 (chart)
Docker_Linux_containerd TestFunctional/serial/LogsFileCmd (gopogh) 1.75 (chart)
Docker_macOS TestPause/serial/SecondStartNoReconfiguration (gopogh) 3.97 (chart)
KVM_Linux TestMultiNode/serial/RestartMultiNode (gopogh) 8.82 (chart)
Hyperkit_macOS TestRunningBinaryUpgrade (gopogh) 10.46 (chart)
Hyperkit_macOS TestFunctional/parallel/ConfigCmd (gopogh) 15.03 (chart)
Hyper-V_Windows TestPause/serial/SecondStartNoReconfiguration (gopogh) 56.25 (chart)
Hyper-V_Windows TestNoKubernetes/serial/StartWithK8s (gopogh) 62.89 (chart)
Hyper-V_Windows TestRunningBinaryUpgrade (gopogh) 64.95 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressDNSAddonActivation (gopogh) 98.15 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressAddonActivation (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressAddons (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/StartLegacyK8sCluster (gopogh) 100.00 (chart)
Docker_macOS TestKubernetesUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestMissingContainerUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestRunningBinaryUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/AddonExistsAfterStop (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/DeployApp (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/EnableAddonWhileActive (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/FirstStart (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/SecondStart (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/old-k8s-version/serial/UserAppExistsAfterStop (gopogh) 100.00 (chart)
Docker_macOS TestStoppedBinaryUpgrade/Upgrade (gopogh) 100.00 (chart)
Docker_Windows TestFunctional/parallel/ServiceCmd (gopogh) 100.00 (chart)
Hyper-V_Windows TestMultiNode/serial/PingHostFrom2Pods (gopogh) 100.00 (chart)
Hyper-V_Windows TestMultiNode/serial/RestartKeepsNodes (gopogh) 100.00 (chart)
Hyper-V_Windows TestStoppedBinaryUpgrade/Upgrade (gopogh) 100.00 (chart)
KVM_Linux_containerd TestPreload (gopogh) 100.00 (chart)
none_Linux TestAddons/Setup (gopogh) 100.00 (chart)
none_Linux TestCertExpiration (gopogh) 100.00 (chart)
More tests... Continued...

Too many tests failed - See test logs for more details.

To see the flake rates of all tests by environment, click here.

@denisok
Copy link
Contributor Author

denisok commented Feb 14, 2023

I can't make any sense from test results.

could someone help me to see where should I look into? Should be they all green?
What should I pay attention to ?

@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15829) |
+----------------+----------+---------------------+
| minikube start | 58.9s    | 59.3s               |
| enable ingress | 30.5s    | 30.5s               |
+----------------+----------+---------------------+

Times for minikube start: 57.9s 59.9s 57.6s 59.6s 59.3s
Times for minikube (PR 15829) start: 60.7s 58.7s 59.3s 59.7s 58.0s

Times for minikube ingress: 31.4s 30.5s 29.4s 30.9s 30.4s
Times for minikube (PR 15829) ingress: 29.4s 30.5s 31.5s 30.4s 30.5s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15829) |
+----------------+----------+---------------------+
| minikube start | 29.1s    | 29.0s               |
| enable ingress | 34.4s    | 22.4s               |
+----------------+----------+---------------------+

Times for minikube start: 29.1s 28.6s 28.8s 29.6s 29.3s
Times for minikube (PR 15829) start: 29.3s 29.6s 29.3s 28.4s 28.3s

Times for minikube ingress: 21.2s 25.7s 22.2s 81.7s 21.3s
Times for minikube (PR 15829) ingress: 22.2s 21.2s 24.2s 22.2s 22.2s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15829) |
+----------------+----------+---------------------+
| minikube start | 24.5s    | 24.4s               |
| enable ingress | 37.5s    | 29.6s               |
+----------------+----------+---------------------+

Times for minikube start: 23.6s 24.8s 24.6s 24.2s 25.2s
Times for minikube (PR 15829) start: 24.5s 24.2s 23.9s 24.2s 25.2s

Times for minikube ingress: 47.8s 47.7s 32.3s 28.2s 31.7s
Times for minikube (PR 15829) ingress: 31.7s 31.7s 32.7s 19.2s 32.8s

@minikube-pr-bot
Copy link

These are the flake rates of all failed tests.

Environment Failed Tests Flake Rate (%)
Docker_Windows TestStartStop/group/default-k8s-diff-port/serial/Pause (gopogh) 0.00 (chart)
Hyperkit_macOS TestStartStop/group/embed-certs/serial/DeployApp (gopogh) 0.00 (chart)
Hyperkit_macOS TestStartStop/group/embed-certs/serial/EnableAddonWhileActive (gopogh) 0.00 (chart)
Hyperkit_macOS TestStartStop/group/embed-certs/serial/FirstStart (gopogh) 0.00 (chart)
Docker_Linux_containerd TestMultiNode/serial/RestartMultiNode (gopogh) 0.60 (chart)
KVM_Linux_containerd TestFunctional/parallel/ConfigCmd (gopogh) 0.61 (chart)
Hyper-V_Windows TestAddons/parallel/CloudSpanner (gopogh) 9.86 (chart)
Docker_Windows TestStartStop/group/old-k8s-version/serial/Pause (gopogh) 10.00 (chart)
Docker_Windows TestFunctional/parallel/ServiceCmd (gopogh) 100.00 (chart)
Hyper-V_Windows TestMultiNode/serial/PingHostFrom2Pods (gopogh) 100.00 (chart)
Hyper-V_Windows TestMultiNode/serial/RestartKeepsNodes (gopogh) 100.00 (chart)
Hyper-V_Windows TestNoKubernetes/serial/StartWithStopK8s (gopogh) 100.00 (chart)
Hyper-V_Windows TestStoppedBinaryUpgrade/Upgrade (gopogh) 100.00 (chart)
KVM_Linux_containerd TestPreload (gopogh) 100.00 (chart)

To see the flake rates of all tests by environment, click here.

Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this PR, please consider making a follow UP PR that when users use multi-node, minikube automatically enable this addon for them or suggests them to do so

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: denisok, medyagh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 14, 2023
@medyagh medyagh merged commit 5ea80d0 into kubernetes:master Feb 14, 2023
@medyagh medyagh changed the title update CSI driver, enable multi-node multi-node: Add support for volumes using CSI addon Feb 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Persistent Volumes permission issue for 2+ node clutsters
7 participants