add canary test entrypoint script #1717
Conversation
test/README.md
Outdated
| @@ -0,0 +1,8 @@ | |||
| ## Available Ginkgo Focuses | |||
|
|
|||
| ### [CANARY] | |||
There was a problem hiding this comment.
A note on why do we separate it, and how it is used in testing infrastructure might guide the next maintainer.
Is the idea that
- SMOKE will be run always against any environment and change?
- CANARY will be run before the release or nightly?
There was a problem hiding this comment.
Here are my initial thoughts,
- The nightly tests will run all integration tests which could run for hours.
- Canary tests will be run quite frequently multiple times in a day so we won't be able to run all of our integration tests. So we run a selected set of tests of most important features that need to be tested. And probably cross dependencies like ALB creation as part of Service creation etc..
- Smoke tests can be utilized to fail a workflow early, for example PR Integration workflow can run Smoke tests as prerequisite before running the lengthy integration tests. But in case of this PR we are also using it to run on latest addon given running the entire Canary test could lead to the script running for 20 more minutes.
I don't have much information about how upstream does this, so open to feedback to be more aligned with upstream testing or to improve this PR. Also I think as you suggested we need more clear guidance on types of tests and what should run in each type of test.
There was a problem hiding this comment.
Hi @abhipth - This information is sufficient. You could please add this in the readme itself.
When we realize how upstream or other projects do any different for their benefits, we can evaluate and change it then.
But the above schedule idea and separation looks good to me.
scripts/run-canary-test.sh
Outdated
| INTEGRATION_TEST_DIR="$SCRIPT_DIR/../test/integration-new" | ||
| VPC_CNI_ADDON_NAME="vpc-cni" | ||
|
|
||
| echo "Running Canary tests for amazon-vpc-cni-k8s with the following varialbes |
There was a problem hiding this comment.
nit: varialbes->variables
scripts/run-canary-test.sh
Outdated
| echo "addon status matches expected status" | ||
| return | ||
| fi | ||
| echo "addon status is not euqal to $expected_status" |
* add canary test entrypoint script * only run the linux tests on linux nodes * add details about the type of tests in README * run tests only on the latest addon version
What type of PR is this?
Script for adding Canary tests entrypoint script. The script does the following
Currently the total time for the entire canary script executions is around 25minutes.
Testing done on this change:
Results from test execution.
Will this break upgrades or downgrades. Has updating a running cluster been tested?:
No
Does this change require updates to the CNI daemonset config files to work?:
No
Does this PR introduce any user-facing change?:
No
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.