WIP: Add external Kubernetes tests #473

jsafrane · 2020-03-24T16:08:40Z

Fixes #470

What is this PR about? / Why do we need it?
Kubernetes offers so called external tests, where it can run e2e tests against an installed CSI drivers.
See https://github.com/kubernetes/kubernetes/tree/master/test/e2e/storage/external

As consequence, I renamed tests/e2e-migration to tests/e2e-kubernetes, because both external & migration tests are there now.

Work in progress:

Update to Kubernetes 1.17

Tests imported from Kubernetes will be used for more than migration tests.

k8s-ci-robot · 2020-03-24T16:08:51Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jsafrane

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jsafrane]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

coveralls · 2020-03-24T16:12:02Z

Pull Request Test Coverage Report for Build 1052

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 77.43%

Totals
Change from base Build 1045:	0.0%
Covered Lines:	1386
Relevant Lines:	1790

💛 - Coveralls

jsafrane · 2020-03-24T16:14:23Z

I tested it in my AWS account - it installs 1.15 cluster, runs AWS EBS driver there and runs 1.16 tests against the driver.

The test don't pass - as it runs 1.16 tests, expecting 1.16 cluster. I haven't been able to install Kubernetes 1.16 with k8s-e2e-tester. In addition, 1.17 or even 1.18 cluster + tests would be very helpful, 1.16 is pretty old.

Looking at https://github.com/aws/aws-k8s-tester, it hardcodes kops 1.14.alpha. Is there any plan to move to 1.17?

Whole k8s-e2e-tester is quite cryptic, it does not log anything and I can't even get Kubernetes logs from it.

cc @leakingtapan @wongma7

leakingtapan · 2020-03-25T01:31:54Z

tester/external-config.yaml

@@ -0,0 +1,75 @@
+cluster:
+  kops:
+    stateFile: s3://jsafrane-k8s-kops-csi-e2e


Fix the s3 path

leakingtapan · 2020-03-25T01:32:26Z

tester/external-config.yaml

+  echo "Deploying driver"
+  # install helm
+  OS_ARCH=$(go env GOOS)-amd64
+  helm_name=helm-v2.14.1-$OS_ARCH.tar.gz


I'd prefer all config.yamls to be the same and the rest uses 2.14.

leakingtapan · 2020-03-25T01:33:46Z

tests/e2e-kubernetes/manifest.yaml

+    Max: 16Ti
+  SupportedFsType:
+    xfs: {}
+    ext4: {}


add ext2 and ext3 since they are supported too

I am not sure it brings any real value. It only doubles number of tests. It's served by ext4 kernel module anyway.

leakingtapan · 2020-03-25T01:34:59Z

tests/e2e-kubernetes/README.md

+
+# Update dependencies
+```sh
+go mod edit -require=k8s.io/kubernetes@v1.15.3


Which k/k version supports external tests? Update this accordingly?

The version of installed kubernetes and tests should be the same. Running newer version of the tests against older kubernetes will test features that are not enabled by default or run regression tests for bugs that were not fixed yet. That's why 1.15.

As I wrote above, we need 1.17 cluster or even 1.18 to run some real tests. But I don't know how to install it with k8s-e2e-tester.

leakingtapan · 2020-03-26T20:30:06Z

To make sure migration test also works
/test pull-aws-ebs-csi-driver-migration-test-latest

leakingtapan · 2020-04-05T06:51:13Z

/test pull-aws-ebs-csi-driver-migration-test-latest

fejta-bot · 2020-07-04T07:35:50Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

k8s-ci-robot · 2020-07-04T07:35:57Z

@jsafrane: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

fejta-bot · 2020-08-03T08:16:47Z

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

leakingtapan · 2020-08-08T22:30:40Z

/remove-lifecycle stale

leakingtapan · 2020-08-08T22:31:03Z

/remove-lifecycle rotten

leakingtapan · 2020-08-08T22:31:32Z

/remove-lifecycle stale

fejta-bot · 2020-11-06T22:32:15Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot · 2020-12-06T23:17:28Z

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

fejta-bot · 2021-01-06T00:02:52Z

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

k8s-ci-robot · 2021-01-06T00:02:59Z

@fejta-bot: Closed this PR.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ayberk · 2021-01-06T01:22:03Z

/reopen
/remove-lifecycle rotten

I think we still need this? cc @wongma7 @jsafrane

k8s-ci-robot · 2021-01-06T01:22:10Z

@ayberk: Reopened this PR.

In response to this:

/reopen
/remove-lifecycle rotten

I think we still need this? cc @wongma7 @jsafrane

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot · 2021-01-06T01:22:25Z

@jsafrane: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-aws-ebs-csi-driver-migration-test-latest	`725c41a`	link	`/test pull-aws-ebs-csi-driver-migration-test-latest`
pull-aws-ebs-csi-driver-e2e-multi-az	`725c41a`	link	`/test pull-aws-ebs-csi-driver-e2e-multi-az`
pull-aws-ebs-csi-driver-unit	`725c41a`	link	`/test pull-aws-ebs-csi-driver-unit`
pull-aws-ebs-csi-driver-verify	`725c41a`	link	`/test pull-aws-ebs-csi-driver-verify`
pull-aws-ebs-csi-driver-e2e-single-az	`725c41a`	link	`/test pull-aws-ebs-csi-driver-e2e-single-az`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

wongma7 · 2021-01-06T04:10:17Z

Yes we do. The e2e-migration test setup is not good enough, it's failing since inline migration fix is not in yet, so can't be presubmit, and anyway we should run the same tests with migration on AND off.

I can rebase and pick this up from jsafrane, I revamped the run-e2e-test script recently and it doesn't rely on the tester binary so it should be easier to keep up with k/k test/e2e

AndyXiangLi · 2021-02-02T18:55:21Z

Hi @wongma7, are you still actively working on this issue?

wongma7 · 2021-04-21T21:26:34Z

/close

thanks all, we picked this up in #726

jsafrane added 2 commits March 24, 2020 14:26

Rename e2e-migration to e2e-kubernetes

2847025

Tests imported from Kubernetes will be used for more than migration tests.

Add Kubernetes external test

725c41a

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Mar 24, 2020

k8s-ci-robot requested review from bertinatto and d-nishi March 24, 2020 16:08

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 24, 2020

jsafrane force-pushed the e2e branch 2 times, most recently from 0c3ea19 to 3e5da35 Compare March 24, 2020 16:19

leakingtapan reviewed Mar 25, 2020

View reviewed changes

jsafrane force-pushed the e2e branch from 3e5da35 to 725c41a Compare March 25, 2020 09:37

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 4, 2020

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 4, 2020

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Aug 3, 2020

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Aug 8, 2020

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 6, 2020

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 6, 2020

k8s-ci-robot closed this Jan 6, 2021

k8s-ci-robot reopened this Jan 6, 2021

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jan 6, 2021

This was referenced Jan 13, 2021

Refactor e2e testing scripts to be more reusable and use them instead… #694

Merged

Prow test jobs status #696

Closed

wongma7 closed this Apr 21, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Add external Kubernetes tests #473

WIP: Add external Kubernetes tests #473

jsafrane commented Mar 24, 2020

k8s-ci-robot commented Mar 24, 2020

coveralls commented Mar 24, 2020 •

edited

Loading

jsafrane commented Mar 24, 2020

leakingtapan Mar 25, 2020

jsafrane Mar 25, 2020

leakingtapan Mar 25, 2020

jsafrane Mar 25, 2020

leakingtapan Mar 25, 2020

jsafrane Mar 25, 2020

leakingtapan Mar 25, 2020

jsafrane Mar 25, 2020

leakingtapan commented Mar 26, 2020

leakingtapan commented Apr 5, 2020

fejta-bot commented Jul 4, 2020

k8s-ci-robot commented Jul 4, 2020

fejta-bot commented Aug 3, 2020

leakingtapan commented Aug 8, 2020

leakingtapan commented Aug 8, 2020

leakingtapan commented Aug 8, 2020

fejta-bot commented Nov 6, 2020

fejta-bot commented Dec 6, 2020

fejta-bot commented Jan 6, 2021

k8s-ci-robot commented Jan 6, 2021

ayberk commented Jan 6, 2021

k8s-ci-robot commented Jan 6, 2021

k8s-ci-robot commented Jan 6, 2021

wongma7 commented Jan 6, 2021

AndyXiangLi commented Feb 2, 2021

wongma7 commented Apr 21, 2021

WIP: Add external Kubernetes tests #473

WIP: Add external Kubernetes tests #473

Conversation

jsafrane commented Mar 24, 2020

k8s-ci-robot commented Mar 24, 2020

coveralls commented Mar 24, 2020 • edited Loading

Pull Request Test Coverage Report for Build 1052

💛 - Coveralls

jsafrane commented Mar 24, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

leakingtapan commented Mar 26, 2020

leakingtapan commented Apr 5, 2020

fejta-bot commented Jul 4, 2020

k8s-ci-robot commented Jul 4, 2020

fejta-bot commented Aug 3, 2020

leakingtapan commented Aug 8, 2020

leakingtapan commented Aug 8, 2020

leakingtapan commented Aug 8, 2020

fejta-bot commented Nov 6, 2020

fejta-bot commented Dec 6, 2020

fejta-bot commented Jan 6, 2021

k8s-ci-robot commented Jan 6, 2021

ayberk commented Jan 6, 2021

k8s-ci-robot commented Jan 6, 2021

k8s-ci-robot commented Jan 6, 2021

wongma7 commented Jan 6, 2021

AndyXiangLi commented Feb 2, 2021

wongma7 commented Apr 21, 2021

coveralls commented Mar 24, 2020 •

edited

Loading