Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ST] Upgrade tests methodsource + generated test name #3557

Merged
merged 1 commit into from
Aug 28, 2020

Conversation

kornys
Copy link
Contributor

@kornys kornys commented Aug 27, 2020

Signed-off-by: David Kornel kornys@outlook.com

Type of change

  • Refactoring

Description

  1. Set up surefire/failsafe to use displayname in xunit files
  2. Using junit jupiter methodsource for parametrized upgrade tests instead of 3rd party lib
  3. Generating names for upgrade parametrized test

@kornys kornys requested review from Frawless and a team August 27, 2020 09:33
@kornys kornys self-assigned this Aug 27, 2020
@kornys kornys added this to the 0.20.0 milestone Aug 27, 2020
Signed-off-by: David Kornel <kornys@outlook.com>
@kornys kornys force-pushed the st-junit-displayname branch from 8fd5b0b to d9b35ff Compare August 27, 2020 12:59
@kornys
Copy link
Contributor Author

kornys commented Aug 27, 2020

@strimzi-ci run tests profile=upgrade test_only

@strimzi-ci
Copy link

❌ Test Summary ❌

TEST_PROFILE: upgrade
EXCLUDED_GROUPS: networkpolicies,flaky
TEST_CASE: *ST
TOTAL: 9
PASS: 6
FAIL: 3
SKIP: 0
BUILD_NUMBER: 1470
BUILD_ENV: oc cluster up

❗ Test Failures ❗

  • testUpgradeKafkaWithoutVersion in io.strimzi.systemtest.upgrade.StrimziUpgradeST
  • testChainUpgrade in io.strimzi.systemtest.upgrade.StrimziUpgradeST

Re-run command:
@strimzi-ci run tests true profile=upgrade testcase=io.strimzi.systemtest.upgrade.StrimziUpgradeST#testUpgradeKafkaWithoutVersion,io.strimzi.systemtest.upgrade.StrimziUpgradeST#testChainUpgrade,io.strimzi.systemtest.upgrade.StrimziUpgradeST#testUpgradeStrimziVersion-0.15.0-0.16.2,io.strimzi.systemtest.upgrade.StrimziUpgradeST#testUpgradeStrimziVersion-0.16.2-0.17.0,io.strimzi.systemtest.upgrade.StrimziUpgradeST#testUpgradeStrimziVersion-0.17.0-0.18.0,io.strimzi.systemtest.upgrade.StrimziUpgradeST#testUpgradeStrimziVersion-0.18.0-0.19.0,io.strimzi.systemtest.upgrade.StrimziUpgradeST#testUpgradeStrimziVersion-0.19.0-HEAD

@kornys kornys added the ready for merge Label for PRs which are ready for merge label Aug 28, 2020
@kornys
Copy link
Contributor Author

kornys commented Aug 28, 2020

Issues are not caused by this PR but jenkins env issues.

@kornys
Copy link
Contributor Author

kornys commented Aug 28, 2020

Also there is parf of xunit file generated by failsafe and tests are correctly named so we can easily identify upgrade test failure for specific version from testname

<testcase name="testUpgradeStrimziVersion-0.12.1-0.13.0" classname="io.strimzi.systemtest.upgrade.StrimziUpgradeST" time="2,715.949"/>
  <testcase name="testUpgradeStrimziVersion-0.13.0-0.14.0" classname="io.strimzi.systemtest.upgrade.StrimziUpgradeST" time="1,242.575"/>
  <testcase name="testUpgradeStrimziVersion-0.14.0-0.15.0" classname="io.strimzi.systemtest.upgrade.StrimziUpgradeST" time="1,629.396"/>
  <testcase name="testUpgradeStrimziVersion-0.15.0-0.16.2" classname="io.strimzi.systemtest.upgrade.StrimziUpgradeST" time="2,485.258">

@scholzj scholzj merged commit c3f81be into strimzi:master Aug 28, 2020
@scholzj
Copy link
Member

scholzj commented Aug 28, 2020

Thanks for the PR.

@kornys kornys deleted the st-junit-displayname branch August 28, 2020 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for merge Label for PRs which are ready for merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants