Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Document test execution order #134

Closed
h-arlt opened this issue Aug 24, 2020 · 7 comments
Closed

Document test execution order #134

h-arlt opened this issue Aug 24, 2020 · 7 comments
Assignees
@occupant23
Labels
codeDone The issue is rechecked for development branch codeReadyForRecheck The issue is fixed/implemented AND merged into development, but needs to be rechecked there docsReadyForMerge Documentation was created for this issue in a separate branch which is ready to merge to develop Low Priority
Milestone
v4.1.0

Comments

@h-arlt
Copy link
Contributor

h-arlt commented Aug 24, 2020

The JUnit runner used by Neodymium executes the tests in a specific and fix order.
This fact and what execution order is used in detail are not document anywhere.

Also, we need to determine the effect of annotating a test class with @FixMethodOrder as described here.
In case there is no effect, fine. But that is something the user should be aware of.
On the other hand, if the annotation has an effect on the test execution order, the user should be aware of that fact too.
@h-arlt h-arlt added the docsReadyForMerge Documentation was created for this issue in a separate branch which is ready to merge to develop label Aug 24, 2020
@occupant23 occupant23 modified the milestones: backlog, v4.1.0 Sep 1, 2020
@occupant23 occupant23 self-assigned this Oct 8, 2020
@occupant23
Copy link
Contributor

Added a new documentation page for JUnit within the develop branch.

@occupant23 occupant23 added the codeReadyForRecheck The issue is fixed/implemented AND merged into development, but needs to be rechecked there label Oct 8, 2020
@occupant23
Copy link
Contributor

@h-arlt Please check if this suits your need.

@occupant23
Copy link
Contributor

@rkl-xc Please also recheck it.

@h-arlt
Copy link
Contributor Author

h-arlt commented Oct 12, 2020

Some remarks:

  • IMO page title is too specific; what about to use 'Test Execution' instead?

  • For Neodymium this results in a mixture of ordered and unordered.

    An example would be nice, e.g. two methods (testA and testB) that are to be run with 3 data sets and two browsers.

Q1: Does Neodymium's test multiplication apply after JUnit's method ordering or before?
Q2: When using one of JUnit's method sorters that work on the test method's name, has the method's data set (or browser) any impact on the execution order?

@occupant23
Copy link
Contributor

added more information and extended the example like recommended

@h-arlt
Copy link
Contributor Author

h-arlt commented Oct 15, 2020

Looks good to me.

@h-arlt h-arlt added the codeDone The issue is rechecked for development branch label Oct 15, 2020
@rkl-xc
Copy link
Contributor

rkl-xc commented Oct 16, 2020

review done, added some suggestions, please have a look at it again

@rkl-xc rkl-xc removed their assignment Oct 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@occupant23 occupant23

Labels
codeDone The issue is rechecked for development branch codeReadyForRecheck The issue is fixed/implemented AND merged into development, but needs to be rechecked there docsReadyForMerge Documentation was created for this issue in a separate branch which is ready to merge to develop Low Priority
Projects
None yet
Milestone
v4.1.0
Development

No branches or pull requests
3 participants
@h-arlt @occupant23 @rkl-xc