Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate tests to JUnit5 #634

Merged
merged 3 commits into from
Feb 6, 2025

Conversation

strangelookingnerd
Copy link
Contributor

@strangelookingnerd strangelookingnerd commented Jan 16, 2025

This PR aims to migrate all tests to JUnit5. Changes include:

  • Use JUnit5 annotations and imports
  • Cleanup assertions
  • Remove public visibility from test classes and methods

I am well aware that this is a quite large changeset however I hope that there is still interest in this PR and it will be reviewd.
If there are any questions, please do not hesitate to ping me.

  • Make sure you are requesting to pull a topic/feature/bugfix branch (right side) and not your master branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests - that demonstrates feature works or fixes the issue

@strangelookingnerd strangelookingnerd requested a review from a team as a code owner January 16, 2025 09:18
@strangelookingnerd strangelookingnerd marked this pull request as draft January 21, 2025 13:48
@rantoniuk rantoniuk removed the request for review from a team January 28, 2025 12:11
@strangelookingnerd strangelookingnerd marked this pull request as ready for review February 4, 2025 10:29
@rantoniuk
Copy link
Contributor

I noticed a 30 second duration increase for test execution phase:

https://ci.jenkins.io/job/Plugins/job/jira-plugin/job/PR-634/5/pipeline-console/log?nodeId=51

[2025-02-04T10:14:45.025Z] Running tests for org.jenkins-ci.plugins:jira:3.14-rc1562.f6b_411a_64290
[2025-02-04T10:17:34.611Z] [INFO] Tests run: 222, Failures: 0, Errors: 0, Skipped: 0

3m3s

https://ci.jenkins.io/job/Plugins/job/jira-plugin/job/master/451/pipeline-console/log?nodeId=112

[2025-02-03T11:41:06.451Z] Running tests for org.jenkins-ci.plugins:jira:3.14-rc1557.80a_9999b_67c0
[2025-02-03T11:43:33.767Z] [INFO] Tests run: 222, Failures: 0, Errors: 0, Skipped: 0

2m27s

Do you think it might be related to the changes in any way?

strangelookingnerd and others added 3 commits February 6, 2025 09:40
* Use JUnit5 annotations and imports
* Cleanup assertions
* Remove public visibility from test classes and methods
* Use JUnit5 annotations and imports
* Cleanup assertions
* Remove public visibility from test classes and methods
@strangelookingnerd
Copy link
Contributor Author

I took a closer look at it and compared the execution times of tests before and after my changes on my local machine.
The overall execution time varies by +/- 30 seconds in both cases. I had multiple runs and took the median values:

Test Case JUnit4 Runtime JUnit5 Runtime
ApacheAsyncHttpClientTest 24.23 s 19.14 s
CompletableFuturePromiseHttpPromiseAsyncClientTest 1.254 s 1.127 s
ChangingWorkflowTest 1.273 s 1.254 s
CliParameterTest 18.70 s 17.36 s
ConfigAsCodeTest 18.76 s 17.93 s
CredentialsHelperTest 21.80 s 17.98 s
DescriptorImplTest 22.23 s 18.58 s
EmptyFriendlyURLConverterTest 15.52 s 14.56 s
EnvironmentExpanderTest 1.356 s 1.353 s
JiraBuildActionTest 17.05 s 16.01 s
JiraChangeLogAnnotatorTest 1.434 s 1.483 s
JiraCreateIssueNotifierTest 18.15 s 17.50 s
JiraCreateReleaseNotesTest 1.893 s 1.937 s
JiraEnvironmentContributingActionTest 1.388 s 1.370 s
JiraEnvironmentVariableBuilderTest 1.827 s 1.848 s
JiraFolderPropertyTest 23.70 s 23.53 s
JiraGlobalConfigurationSaveTest 16.91 s 15.35 s
JiraGlobalConfigurationTest 15.16 s 15.04 s
JiraIssueMigratorTest 1.721 s 1.770 s
JiraIssueParameterDefResultTest 0.904 s 0.925 s
JiraIssueUpdateBuilderTest 1.765 s 1.947 s
JiraIssueUpdaterTest 0.069 s 0.081 s
JiraJobActionTest 22.41 s 19.86 s
JiraProjectPropertyTest 31.03 s 26.34 s
JiraReplaceFixVersionByRegExTest 0.984 s 1.020 s
JiraRestServiceProxyTest 19.70 s 18.47 s
JiraRestServiceTest 1.202 s 1.180 s
JiraSiteSecurity1029Test 20.02 s 19.95 s
JiraSiteTest 35.26 s 27.98 s
MailResolverDisabledTest 15.85 s 14.67 s
MailResolverWithExtensionTest 21.12 s 21.41 s
UnmaskMailTest 0.020 s 0.035 s
UpdaterTest 23.35 s 21.81 s
VersionCreatorTest 1.781 s 1.770 s
VersionReleaserTest 1.781 s 1.973 s
BearerHttpAuthenticationHandlerTest 0.825 s 0.873 s
JiraRestServiceBearerAuthTest 1.189 s 1.232 s
CommentStepTest 46.06 s 49.04 s
IssueFieldUpdateStepTest 19.98 s 19.92 s
IssueSelectorStepTest 18.39 s 18.50 s
SearchIssuesStepTest 71.25 s 75.99 s
DefaultIssueSelectorTest 1.629 s 1.432 s
ExplicitIssueSelectorTest 0.019 s 0.032 s
JqlIssueSelectorTest 1.682 s 1.635 s
P4JobIssueSelectorTest 1.655 s 1.604 s
JiraVersionParameterDefinitionTest 1.179 s 1.384 s
VersionComparatorTest 0.022 s 0.037 s
InjectedTest 16.78 s 18.67 s
- - -
Total 578.032 s 555.752 s

In my runs, the JUnit5 tests were faster compared to JUnit4. There is little reliable data available on comparison of performance between these two frameworks but looking at the numbers I'd say they are equal.

@rantoniuk
Copy link
Contributor

Great comparison, thanks a lot for spending additional time on this!

@rantoniuk rantoniuk merged commit b6eef6c into jenkinsci:master Feb 6, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants