Skip to content

Commit

Permalink
A few more clarifications in README.
Browse files Browse the repository at this point in the history
  • Loading branch information
rjust committed Nov 3, 2024
1 parent 1c27463 commit 2a10307
Showing 1 changed file with 7 additions and 6 deletions.
13 changes: 7 additions & 6 deletions framework/test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,11 @@ parallel processes):
```
./jobs_cmd.pl ./test_verify_bugs.sh -A | shuf | parallel -j20 --progress
```
Reproducing all bugs (20 jobs in parallel) takes ~3h.
Reproducing all bugs (20 jobs in parallel) takes ~90min.

(When upgrading Defects4J it is helpful to drop the `-A` flag at first for
efficiency. After fixing any issues, add the `-A` flag to test for full
reproducibility with all tests.)
When upgrading Defects4J it is helpful to drop the `-A` flag at first for
efficiency. (Reduces runtime to ~50min). After fixing any issues, add the
`-A` flag back in to test for full reproducibility with all tests.

### Code coverage analysis for all bugs (parallel)
```
Expand All @@ -99,8 +99,9 @@ Obtaining code-level stats for all bugs (20 jobs in parallel) takes ~45min.
```
Running mutation analysis on all bugs (20 jobs in parallel) takes up to ~24h.

TODO: A few (Closure) bugs have a very long analysis time; we should prioritize
these jobs in the shuffled job list to make sure they run first.
TODO: A few (Closure) jobs have a very long analysis time. (All but a hand full
of jobs finish within 14 hours.) We should prioritize these long jobs in the
shuffled job list to make sure they run early in parallel to many smaller jobs.

### Export command and exported properties
```
Expand Down

0 comments on commit 2a10307

Please sign in to comment.