Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pitest uses the default jqwik configuration #245

Open
martinmladenov opened this issue Jul 17, 2023 · 1 comment
Open

Pitest uses the default jqwik configuration #245

martinmladenov opened this issue Jul 17, 2023 · 1 comment
Labels
bug Something isn't working Medium priority

Comments

@martinmladenov
Copy link
Collaborator

PR #181 sets certain configuration options to improve jqwik performance. However, those options don't seem to apply when running Pitest mutants. This should be investigated.

@fm2003
Copy link
Collaborator

fm2003 commented Mar 10, 2025

We should look at the configuration options anyway. The following line does not make sense for normal test runs:

.configurationParameter("jqwik.shrinking.default", "OFF")

Using the shrinker is important to make sure error messages are readable. If we think the shrinker takes too long, we should set this parameter instead: "jqwik.shrinking.bounded.seconds" (the default is 10).

For pitest and meta test runs, it's better if the shrinker is off (because we do not display the error messages in those cases). Therefore, we should make sure we can change the configuration depending on the type of test run.

As for this one:

.configurationParameter("jqwik.tries.default", "100")

It would be nice to have a few more tries if we can afford this without reaching time-outs, because the tests are sometimes flaky (whereas they aren't with the default of 1000). Most reference solutions for the PBT assignments take 2 seconds for "Only tests" and around 10 seconds for "Assess", so it seems like we have some room to increase the number of tries (for both normal test runs and pitest / meta test runs).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Medium priority
Projects
None yet
Development

No branches or pull requests

2 participants