run e2e tests in parallel on OpenShift CI

[kadel]

Right now e2e tests are run in series on OpenShift CI.
As we add more and more tests it gets slower.

It is a good time to pause and figure out how we can run our tests in parallel before adding more.

It would be great if we could run all tests against the same cluster, to cut down time to spin up new clusters.

[girishramnani]

all the tests that not depend on service catalog should be run on openshift CI

[kadel]

GinkGo can run tests in parallel - https://onsi.github.io/ginkgo/#parallel-specs

This would be ideal for us. as we can execute all our tests in one pod.
We just need to make sure that our tests are written in a way where one spec doesn't depend on the other.

ping @amitkrout

[amitkrout]

This is awesome!!!

Thanks for the reference. Will have a look

[amitkrout]

This would be ideal for us. as we can execute all our tests in one pod.
We just need to make sure that our tests are written in a way where one spec doesn't depend on the other.

As of now i don't think test scenarios will be overlapped to other test file scenario because each test are running in their own namespace. For example: java test running in odo-java name space. So the only task will be a implementation of GonkGo Parallel Specs in our make file.

Will check and test locally. If test looks fine then will send the pr

[kadel]

@amitkrout problem is that structure of our tests is wrong.
As I understand it GinkGo can run specs in parallel. But that means that each Spec (It() has to be independent, But that is not true in our e2e. This needs to be fixed first

[amitkrout]

@amitkrout problem is that structure of our tests is wrong.
As I understand it GinkGo can run specs in parallel. But that means that each Spec (It() has to be independent, But that is not true in our e2e. This needs to be fixed first

Forgot to update the issue with my findings which I got few days back. Sorry for the delay

IMHO in our case it will be very difficult to to isolate and maintain each spec i mean the It(…, func() {} clause. Even there will be a high chance of race condition arises in operation like creating/deleting and possible in intermediate operation. Also isolation increases the duplication of some steps in each spec.

There is an open issue on how to run parallel spec/testsuite in smart way for environment like us - onsi/ginkgo#280.

But yes I can give a try on single test file with currently ginkgo supported parallel spec run. If this fit for our requirement then will apply to all test file depending on the use of scenario.

[kadel]

IMHO in our case it will be very difficult to to isolate and maintain each spec i mean the It(…, func() {} clause. Even there will be a high chance of race condition arises in operation like creating/deleting and possible in intermediate operation. Also isolation increases the duplication of some steps in each spec.

there are nice patterns in ginkgo docs explaining how to avoid duplication https://onsi.github.io/ginkgo/#shared-example-patterns

[amitkrout]

IMHO in our case it will be very difficult to to isolate and maintain each spec i mean the It(…, func() {} clause. Even there will be a high chance of race condition arises in operation like creating/deleting and possible in intermediate operation. Also isolation increases the duplication of some steps in each spec.

there are nice patterns in ginkgo docs explaining how to avoid duplication https://onsi.github.io/ginkgo/#shared-example-patterns

Applying on json_test.go file. will send the pr once done

[amitkrout]

IMHO in our case it will be very difficult to to isolate and maintain each spec i mean the It(…, func() {} clause. Even there will be a high chance of race condition arises in operation like creating/deleting and possible in intermediate operation. Also isolation increases the duplication of some steps in each spec.

there are nice patterns in ginkgo docs explaining how to avoid duplication https://onsi.github.io/ginkgo/#shared-example-patterns

Applying on json_test.go file. will send the pr once done

Waiting parallel test template via #1535 to be merged first.

[kadel]

IMHO in our case it will be very difficult to to isolate and maintain each spec i mean the It(…, func() {} clause. Even there will be a high chance of race condition arises in operation like creating/deleting and possible in intermediate operation. Also isolation increases the duplication of some steps in each spec.

there are nice patterns in ginkgo docs explaining how to avoid duplication https://onsi.github.io/ginkgo/#shared-example-patterns

Applying on json_test.go file. will send the pr once done

Waiting parallel test template via #1535 to be merged first.

merged

[amitkrout]

Integration test file

• generic test

• component test

• config test

• java test

• json format test

• link test

• login test

• service test

• source test

• storage test

• watch

e2e test file

• core beta flow test

test template

• clean test

All integration test

• test-integration

[kadel]

/close

[openshift-ci-robot]

@kadel: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.