Clean e2e test example #1535

kadel · 2019-03-28T15:07:19Z

This is an example of how to write e2e tests using ginkgo that perform automatic setup and teardown.

To run this do

go test -v github.com/openshift/odo/tests/e2e -ginkgo.focus="Example of a clean test"  -ginkgo.slowSpecThreshold=120 -ginkgo.randomizeAllSpecs

-ginkgo.randomizeAllSpecs runs all test in random order. This will make sure that we don't create dependencies between tests specs (It( functions) even accidntaly, as the order of in which tests are run can't be prediceted.

You can also run tests in parallel:

ginkgo -nodes=2 -focus="Example of a clean test" slowSpecThreshold=120 -randomizeAllSpecs  tests/e2e/

nodes sets number of parallel executions

Note that you have to use ginkgo (go get github.com/onsi/ginkgo/ginkgo than $GOPATH/bin/ginkgo) to tests in parallel

I had to updated Ginkgo as the version that we were using doesn't support JustAfterEach

kadel · 2019-03-28T15:07:44Z

ping @amitkrout @girishramnani

amitkrout · 2019-03-28T15:35:45Z

@kadel Thanks for the pr in a quick time. Tomorrow will have a look

girishramnani · 2019-03-28T19:22:57Z

great thinking of using the latest version of ginkgo 👍

girishramnani · 2019-03-28T19:24:47Z

tests/e2e/helper/helper_run.go

+
+// and returns the stdout, stderr and exitcode
+func cmdRunner(cmdS string) (string, string, int) {
+	//TODO: this needs to be os independent


lets document this in an issue so that @amitkrout can work on making this os independent

@girishramnani We have issue for it #1131

girishramnani · 2019-03-28T19:26:40Z

I am not seeing any helpers related to waitForCmdOut @kadel are we planning to add those or use the one in the utils?

mohammedzee1000 · 2019-03-29T06:57:04Z

tests/e2e/cleantest_test.go

+		helper.Chdir(originalDir)
+	}
+	if originalProject != "" {
+		helper.OcSwitchProject(originalProject)


Could this not cause an overlap? if oc active project is depended on. What happens if one of the parallel runs gets here first and sets a different active project, while another is working with a different project (in a single cluster context)

good point.
It looks like we could use SynchronizedBeforeSuite and SynchronizedAfterSuite

https://onsi.github.io/ginkgo/#managing-singleton-external-processes-in-parallel-test-suites

Fixed by removing those function all together, as I realized that there is a better way to do this

mohammedzee1000 · 2019-03-29T06:57:18Z

glide.yaml

 - package: github.com/onsi/ginkgo
-  version: v1.4.0
+  version: v1.8.0


mohammedzee1000 · 2019-03-29T06:59:07Z

tests/e2e/helper/helper_oc.go

+
+// CreateRandProject create new project with random name (10 letters)
+// without writing to the config file (without switching project)
+func OcCreateRandProject() string {


amitkrout · 2019-03-29T07:07:32Z

tests/e2e/e2e_test.go

@@ -44,11 +44,11 @@ func TestOdo(t *testing.T) {
 	RunSpecs(t, "odo test suite")


Remove the suite function in a separate file to isolate it form the actual test

For example:

Create a file odo_suite_test.go

package e2e import ( "testing" . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" ) func TestOdo(t *testing.T) { RegisterFailHandler(Fail) RunSpecs(t, "odo test suite") }

This PR is intended to provide just a clean direction and template how we can structure our tests.
This should be part of a cleanup effort in a separate PR.

amitkrout · 2019-03-29T07:15:51Z

I am not seeing any helpers related to waitForCmdOut @kadel are we planning to add those or use the one in the utils?

@girishramnani I think its just a templates for our reference to write the test. Those functions are in utils can be moved under helper package in the subsequent improvement of this pr template.

amitkrout

It make more sense if you rename file name as template_cleantest_test.go.

amitkrout · 2019-03-29T07:46:11Z

Test went fine locally and the interesting part is that with four node parallel execution the total run time reduces to half compare to serial execution of the same template.

Test structure looks good to me.

kadel · 2019-03-29T12:34:56Z

/retest
aws error

cdrage · 2019-03-29T15:33:53Z

tests/e2e/cleantest_test.go

+	Expect(err).NotTo(HaveOccurred())
+
+	originalProject = helper.OcCurrentProject()
+})


If these will be ran for each and every test, would it make sense to try to put as much information as possible into the utility functions / helper files?

amitkrout · 2019-04-01T06:17:19Z

@kadel Rebase and fix the formatting issue

$ make validate
./scripts/check-gofmt.sh
gofmt ERROR - These files are not formated by gofmt:
./tests/e2e/template_cleantest_test.go
make: *** [gofmt] Error 1

kadel · 2019-04-01T12:06:22Z

ping @amitkrout @girishramnani

amitkrout

Test structure is quite impressive. Tested the template locally without any issue. @girishramnani We need to push this pr ASAP.

amitkrout · 2019-04-01T13:06:45Z

/approve

openshift-ci-robot · 2019-04-01T13:06:53Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amitkrout

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [amitkrout]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2019-04-01T13:06:53Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amitkrout

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [amitkrout]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

girishramnani · 2019-04-01T13:08:43Z

/lgtm

openshift-ci-robot added the size/XXL label Mar 28, 2019

openshift-ci-robot requested review from childsb, danwinship, amitkrout and anmolbabu March 28, 2019 15:07

kadel removed request for danwinship and childsb March 28, 2019 18:03

girishramnani reviewed Mar 28, 2019

View reviewed changes

mohammedzee1000 reviewed Mar 29, 2019

View reviewed changes

amitkrout suggested changes Mar 29, 2019

View reviewed changes

cdrage reviewed Mar 29, 2019

View reviewed changes

This was referenced Mar 29, 2019

Implement new odo workflow for: odo watch #1446

Merged

uncomment link tests #1531

Closed

amitkrout mentioned this pull request Apr 1, 2019

run e2e tests in parallel on OpenShift CI #1473

Closed

kadel added 4 commits April 1, 2019 11:05

a template for a clean tests

d14b3b2

vendor: update ginkgo,gomega,testify

94c8e82

e2e template: Remove Before/AfterSuite and rename file

63ffb61

gofmt

a99ec11

kadel force-pushed the clean-test-example branch from 64cd127 to a99ec11 Compare April 1, 2019 09:05

don't ignore .odo

a2931d9

amitkrout approved these changes Apr 1, 2019

View reviewed changes

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. Required by Prow. label Apr 1, 2019

openshift-ci-robot assigned girishramnani Apr 1, 2019

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. Required by Prow. label Apr 1, 2019

openshift-merge-robot merged commit 7c81a21 into redhat-developer:master Apr 1, 2019

girishramnani mentioned this pull request Jan 2, 2020

Templates for integration tests #1179

Closed

rm3l added the estimated-size/XXL (60 - ??) Rough sizing for Epics. More than 3 sprints of work for one person label Jun 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean e2e test example #1535

Clean e2e test example #1535

kadel commented Mar 28, 2019

kadel commented Mar 28, 2019

amitkrout commented Mar 28, 2019 •

edited

Loading

girishramnani commented Mar 28, 2019

girishramnani Mar 28, 2019

amitkrout Mar 29, 2019

girishramnani commented Mar 28, 2019

mohammedzee1000 Mar 29, 2019

kadel Mar 29, 2019

kadel Mar 29, 2019

mohammedzee1000 Mar 29, 2019

mohammedzee1000 Mar 29, 2019

amitkrout Mar 29, 2019

kadel Mar 29, 2019

amitkrout commented Mar 29, 2019

amitkrout left a comment

amitkrout commented Mar 29, 2019

kadel commented Mar 29, 2019

cdrage Mar 29, 2019

amitkrout commented Apr 1, 2019

kadel commented Apr 1, 2019

amitkrout left a comment

amitkrout commented Apr 1, 2019

openshift-ci-robot commented Apr 1, 2019

openshift-ci-robot commented Apr 1, 2019

girishramnani commented Apr 1, 2019

		@@ -44,11 +44,11 @@ func TestOdo(t *testing.T) {
		RunSpecs(t, "odo test suite")

Clean e2e test example #1535

Clean e2e test example #1535

Conversation

kadel commented Mar 28, 2019

kadel commented Mar 28, 2019

amitkrout commented Mar 28, 2019 • edited Loading

girishramnani commented Mar 28, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

girishramnani commented Mar 28, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amitkrout commented Mar 29, 2019

amitkrout left a comment

Choose a reason for hiding this comment

amitkrout commented Mar 29, 2019

kadel commented Mar 29, 2019

Choose a reason for hiding this comment

amitkrout commented Apr 1, 2019

kadel commented Apr 1, 2019

amitkrout left a comment

Choose a reason for hiding this comment

amitkrout commented Apr 1, 2019

openshift-ci-robot commented Apr 1, 2019

openshift-ci-robot commented Apr 1, 2019

girishramnani commented Apr 1, 2019

amitkrout commented Mar 28, 2019 •

edited

Loading