WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.
Gnuplot not found, using plotters backend
Benchmarking overhead_iter
Benchmarking overhead_iter: Warming up for 3.0000 s
Benchmarking overhead_iter: Collecting 100 samples in estimated 5.0000 s (18446744074B iterations)
Benchmarking overhead_iter: Analyzing
Criterion.rs ERROR: At least one measurement of benchmark overhead_iter took zero time per iteration. This should not be possible. If using iter_custom, please verify that your routine is correctly measured.
Benchmarking overhead_par_iter/threads/1
Benchmarking overhead_par_iter/threads/1: Warming up for 3.0000 s
Benchmarking overhead_par_iter/threads/1: Collecting 100 samples in estimated 5.1233 s (141k iterations)
Benchmarking overhead_par_iter/threads/1: Analyzing
overhead_par_iter/threads/1
time: [36.417 us 36.530 us 36.684 us]
change: [-1.7357% -1.3932% -1.0280%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
2 (2.00%) low mild
5 (5.00%) high mild
2 (2.00%) high severe
Benchmarking overhead_par_iter/threads/2
Benchmarking overhead_par_iter/threads/2: Warming up for 3.0000 s
Benchmarking overhead_par_iter/threads/2: Collecting 100 samples in estimated 5.2374 s (106k iterations)
Benchmarking overhead_par_iter/threads/2: Analyzing
overhead_par_iter/threads/2
time: [49.396 us 49.500 us 49.603 us]
change: [+1.8579% +2.3542% +2.8331%] (p = 0.00 < 0.05)
Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) low mild
1 (1.00%) high severe
Benchmarking overhead_par_iter/threads/4
Benchmarking overhead_par_iter/threads/4: Warming up for 3.0000 s
Benchmarking overhead_par_iter/threads/4: Collecting 100 samples in estimated 5.2682 s (91k iterations)
Benchmarking overhead_par_iter/threads/4: Analyzing
overhead_par_iter/threads/4
time: [57.821 us 58.004 us 58.182 us]
change: [-4.4366% -4.1229% -3.7942%] (p = 0.00 < 0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) low mild
2 (2.00%) high mild
Benchmarking overhead_par_iter/threads/8
Benchmarking overhead_par_iter/threads/8: Warming up for 3.0000 s
Benchmarking overhead_par_iter/threads/8: Collecting 100 samples in estimated 5.1527 s (76k iterations)
Benchmarking overhead_par_iter/threads/8: Analyzing
overhead_par_iter/threads/8
time: [68.481 us 68.643 us 68.788 us]
change: [-6.4654% -6.1010% -5.7737%] (p = 0.00 < 0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low severe
2 (2.00%) low mild
Benchmarking overhead_par_iter/threads/16
Benchmarking overhead_par_iter/threads/16: Warming up for 3.0000 s
Benchmarking overhead_par_iter/threads/16: Collecting 100 samples in estimated 5.0760 s (76k iterations)
Benchmarking overhead_par_iter/threads/16: Analyzing
overhead_par_iter/threads/16
time: [66.683 us 66.906 us 67.126 us]
change: [-7.4531% -7.0526% -6.6423%] (p = 0.00 < 0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) low mild
1 (1.00%) high mild
Benchmarking overhead_par_iter/threads/32
Benchmarking overhead_par_iter/threads/32: Warming up for 3.0000 s
Benchmarking overhead_par_iter/threads/32: Collecting 100 samples in estimated 5.0010 s (76k iterations)
Benchmarking overhead_par_iter/threads/32: Analyzing
overhead_par_iter/threads/32
time: [66.328 us 66.522 us 66.729 us]
change: [-8.0958% -7.6359% -7.1725%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking for_each_iter
Benchmarking for_each_iter: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 10.2s, or reduce sample count to 40.
Benchmarking for_each_iter: Collecting 100 samples in estimated 10.238 s (100 iterations)
Benchmarking for_each_iter: Analyzing
for_each_iter time: [101.86 ms 102.02 ms 102.20 ms]
change: [-14.963% -14.707% -14.445%] (p = 0.00 < 0.05)
Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
3 (3.00%) high mild
11 (11.00%) high severe
Benchmarking for_each_par_iter/threads/1
Benchmarking for_each_par_iter/threads/1: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.3s, or reduce sample count to 70.
Benchmarking for_each_par_iter/threads/1: Collecting 100 samples in estimated 6.2608 s (100 iterations)
Benchmarking for_each_par_iter/threads/1: Analyzing
for_each_par_iter/threads/1
time: [62.917 ms 64.526 ms 66.114 ms]
change: [-15.805% -12.861% -9.9089%] (p = 0.00 < 0.05)
Performance has improved.
Benchmarking for_each_par_iter/threads/2
Benchmarking for_each_par_iter/threads/2: Warming up for 3.0000 s
Benchmarking for_each_par_iter/threads/2: Collecting 100 samples in estimated 7.8981 s (200 iterations)
Benchmarking for_each_par_iter/threads/2: Analyzing
for_each_par_iter/threads/2
time: [39.001 ms 39.226 ms 39.454 ms]
change: [-16.478% -15.809% -15.132%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking for_each_par_iter/threads/4
Benchmarking for_each_par_iter/threads/4: Warming up for 3.0000 s
Benchmarking for_each_par_iter/threads/4: Collecting 100 samples in estimated 7.2607 s (300 iterations)
Benchmarking for_each_par_iter/threads/4: Analyzing
for_each_par_iter/threads/4
time: [24.271 ms 24.336 ms 24.401 ms]
change: [-14.880% -14.494% -14.086%] (p = 0.00 < 0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild
Benchmarking for_each_par_iter/threads/8
Benchmarking for_each_par_iter/threads/8: Warming up for 3.0000 s
Benchmarking for_each_par_iter/threads/8: Collecting 100 samples in estimated 5.6014 s (400 iterations)
Benchmarking for_each_par_iter/threads/8: Analyzing
for_each_par_iter/threads/8
time: [14.014 ms 14.016 ms 14.018 ms]
change: [-14.019% -13.897% -13.767%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low severe
2 (2.00%) low mild
2 (2.00%) high mild
4 (4.00%) high severe
Benchmarking for_each_par_iter/threads/16
Benchmarking for_each_par_iter/threads/16: Warming up for 3.0000 s
Benchmarking for_each_par_iter/threads/16: Collecting 100 samples in estimated 5.5001 s (700 iterations)
Benchmarking for_each_par_iter/threads/16: Analyzing
for_each_par_iter/threads/16
time: [7.8439 ms 7.8464 ms 7.8489 ms]
change: [-15.561% -15.337% -15.136%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high mild
Benchmarking for_each_par_iter/threads/32
Benchmarking for_each_par_iter/threads/32: Warming up for 3.0000 s
Benchmarking for_each_par_iter/threads/32: Collecting 100 samples in estimated 5.4805 s (700 iterations)
Benchmarking for_each_par_iter/threads/32: Analyzing
for_each_par_iter/threads/32
time: [7.8291 ms 7.8335 ms 7.8382 ms]
change: [-16.277% -15.982% -15.706%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high severe
Benchmarking many_maps_iter
Benchmarking many_maps_iter: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 11.1s, or reduce sample count to 40.
Benchmarking many_maps_iter: Collecting 100 samples in estimated 11.109 s (100 iterations)
Benchmarking many_maps_iter: Analyzing
many_maps_iter time: [110.76 ms 110.96 ms 111.18 ms]
change: [-18.390% -18.161% -17.937%] (p = 0.00 < 0.05)
Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
5 (5.00%) high mild
8 (8.00%) high severe
Benchmarking many_maps_par_iter/threads/1
Benchmarking many_maps_par_iter/threads/1: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.0s, or reduce sample count to 70.
Benchmarking many_maps_par_iter/threads/1: Collecting 100 samples in estimated 7.0427 s (100 iterations)
Benchmarking many_maps_par_iter/threads/1: Analyzing
many_maps_par_iter/threads/1
time: [70.629 ms 71.917 ms 73.152 ms]
change: [-16.133% -13.948% -11.638%] (p = 0.00 < 0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
10 (10.00%) low mild
Benchmarking many_maps_par_iter/threads/2
Benchmarking many_maps_par_iter/threads/2: Warming up for 3.0000 s
Benchmarking many_maps_par_iter/threads/2: Collecting 100 samples in estimated 8.6057 s (200 iterations)
Benchmarking many_maps_par_iter/threads/2: Analyzing
many_maps_par_iter/threads/2
time: [42.698 ms 42.954 ms 43.220 ms]
change: [-17.305% -16.275% -15.261%] (p = 0.00 < 0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild
Benchmarking many_maps_par_iter/threads/4
Benchmarking many_maps_par_iter/threads/4: Warming up for 3.0000 s
Benchmarking many_maps_par_iter/threads/4: Collecting 100 samples in estimated 5.2638 s (200 iterations)
Benchmarking many_maps_par_iter/threads/4: Analyzing
many_maps_par_iter/threads/4
time: [26.229 ms 26.317 ms 26.405 ms]
change: [-16.526% -16.115% -15.695%] (p = 0.00 < 0.05)
Performance has improved.
Benchmarking many_maps_par_iter/threads/8
Benchmarking many_maps_par_iter/threads/8: Warming up for 3.0000 s
Benchmarking many_maps_par_iter/threads/8: Collecting 100 samples in estimated 6.0870 s (400 iterations)
Benchmarking many_maps_par_iter/threads/8: Analyzing
many_maps_par_iter/threads/8
time: [15.212 ms 15.218 ms 15.225 ms]
change: [-15.460% -15.373% -15.288%] (p = 0.00 < 0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) high mild
9 (9.00%) high severe
Benchmarking many_maps_par_iter/threads/16
Benchmarking many_maps_par_iter/threads/16: Warming up for 3.0000 s
Benchmarking many_maps_par_iter/threads/16: Collecting 100 samples in estimated 5.2481 s (600 iterations)
Benchmarking many_maps_par_iter/threads/16: Analyzing
many_maps_par_iter/threads/16
time: [8.7423 ms 8.7466 ms 8.7511 ms]
change: [-18.317% -18.062% -17.829%] (p = 0.00 < 0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
Benchmarking many_maps_par_iter/threads/32
Benchmarking many_maps_par_iter/threads/32: Warming up for 3.0000 s
Benchmarking many_maps_par_iter/threads/32: Collecting 100 samples in estimated 5.2378 s (600 iterations)
Benchmarking many_maps_par_iter/threads/32: Analyzing
many_maps_par_iter/threads/32
time: [8.7344 ms 8.7467 ms 8.7628 ms]
change: [-18.591% -18.293% -17.999%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
1 (1.00%) high mild
3 (3.00%) high severe
WARNING: HTML report generation will become a non-default optional feature in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion (https://github.com/bheisler/cargo-criterion) and will be optional in a future version of Criterion.rs. To silence this warning, either switch to cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.
Gnuplot not found, using plotters backend
Benchmarking empty_systems/000_systems
Benchmarking empty_systems/000_systems: Warming up for 500.00 ms
Benchmarking empty_systems/000_systems: Collecting 100 samples in estimated 3.0074 s (2.0M iterations)
Benchmarking empty_systems/000_systems: Analyzing
empty_systems/000_systems
time: [1.7043 us 1.8221 us 1.9518 us]
change: [-22.084% -17.372% -12.272%] (p = 0.00 < 0.05)
Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
6 (6.00%) high mild
10 (10.00%) high severe
Benchmarking empty_systems/001_systems
Benchmarking empty_systems/001_systems: Warming up for 500.00 ms
Benchmarking empty_systems/001_systems: Collecting 100 samples in estimated 3.0138 s (833k iterations)
Benchmarking empty_systems/001_systems: Analyzing
empty_systems/001_systems
time: [3.6682 us 3.6776 us 3.6873 us]
change: [-0.9729% -0.2104% +0.5874%] (p = 0.60 > 0.05)
No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low severe
4 (4.00%) low mild
Benchmarking empty_systems/002_systems
Benchmarking empty_systems/002_systems: Warming up for 500.00 ms
Benchmarking empty_systems/002_systems: Collecting 100 samples in estimated 3.0070 s (656k iterations)
Benchmarking empty_systems/002_systems: Analyzing
empty_systems/002_systems
time: [4.5339 us 4.5448 us 4.5540 us]
change: [+0.2747% +1.0757% +1.8911%] (p = 0.01 < 0.05)
Change within noise threshold.
Found 15 outliers among 100 measurements (15.00%)
11 (11.00%) low severe
1 (1.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe
Benchmarking empty_systems/003_systems
Benchmarking empty_systems/003_systems: Warming up for 500.00 ms
Benchmarking empty_systems/003_systems: Collecting 100 samples in estimated 3.0224 s (586k iterations)
Benchmarking empty_systems/003_systems: Analyzing
empty_systems/003_systems
time: [5.1072 us 5.1210 us 5.1348 us]
change: [-5.5215% -4.9455% -4.3989%] (p = 0.00 < 0.05)
Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
2 (2.00%) low severe
8 (8.00%) low mild
5 (5.00%) high mild
Benchmarking empty_systems/004_systems
Benchmarking empty_systems/004_systems: Warming up for 500.00 ms
Benchmarking empty_systems/004_systems: Collecting 100 samples in estimated 3.0180 s (515k iterations)
Benchmarking empty_systems/004_systems: Analyzing
empty_systems/004_systems
time: [5.8494 us 5.8541 us 5.8588 us]
change: [-6.1130% -5.7811% -5.4419%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
2 (2.00%) low severe
3 (3.00%) low mild
2 (2.00%) high mild
Benchmarking empty_systems/005_systems
Benchmarking empty_systems/005_systems: Warming up for 500.00 ms
Benchmarking empty_systems/005_systems: Collecting 100 samples in estimated 3.0212 s (454k iterations)
Benchmarking empty_systems/005_systems: Analyzing
empty_systems/005_systems
time: [6.5900 us 6.6035 us 6.6175 us]
change: [-8.8161% -8.4196% -8.0282%] (p = 0.00 < 0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low severe
3 (3.00%) low mild
Benchmarking empty_systems/010_systems
Benchmarking empty_systems/010_systems: Warming up for 500.00 ms
Benchmarking empty_systems/010_systems: Collecting 100 samples in estimated 3.0426 s (293k iterations)
Benchmarking empty_systems/010_systems: Analyzing
empty_systems/010_systems
time: [10.375 us 10.388 us 10.401 us]
change: [-5.0393% -4.8362% -4.6118%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe
Benchmarking empty_systems/015_systems
Benchmarking empty_systems/015_systems: Warming up for 500.00 ms
Benchmarking empty_systems/015_systems: Collecting 100 samples in estimated 3.0232 s (207k iterations)
Benchmarking empty_systems/015_systems: Analyzing
empty_systems/015_systems
time: [14.428 us 14.447 us 14.467 us]
change: [-9.0771% -8.8480% -8.5936%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
Benchmarking empty_systems/020_systems
Benchmarking empty_systems/020_systems: Warming up for 500.00 ms
Benchmarking empty_systems/020_systems: Collecting 100 samples in estimated 3.0367 s (162k iterations)
Benchmarking empty_systems/020_systems: Analyzing
empty_systems/020_systems
time: [18.671 us 18.699 us 18.727 us]
change: [-5.6679% -5.4745% -5.2902%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
Benchmarking empty_systems/025_systems
Benchmarking empty_systems/025_systems: Warming up for 500.00 ms
Benchmarking empty_systems/025_systems: Collecting 100 samples in estimated 3.0415 s (136k iterations)
Benchmarking empty_systems/025_systems: Analyzing
empty_systems/025_systems
time: [22.359 us 22.414 us 22.461 us]
change: [-7.1029% -6.8566% -6.6290%] (p = 0.00 < 0.05)
Performance has improved.
Benchmarking empty_systems/030_systems
Benchmarking empty_systems/030_systems: Warming up for 500.00 ms
Benchmarking empty_systems/030_systems: Collecting 100 samples in estimated 3.1157 s (121k iterations)
Benchmarking empty_systems/030_systems: Analyzing
empty_systems/030_systems
time: [25.429 us 25.546 us 25.672 us]
change: [-6.3416% -6.0437% -5.7494%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
7 (7.00%) low mild
Benchmarking empty_systems/035_systems
Benchmarking empty_systems/035_systems: Warming up for 500.00 ms
Benchmarking empty_systems/035_systems: Collecting 100 samples in estimated 3.0205 s (106k iterations)
Benchmarking empty_systems/035_systems: Analyzing
empty_systems/035_systems
time: [28.804 us 28.929 us 29.049 us]
change: [-8.6419% -8.3394% -8.0143%] (p = 0.00 < 0.05)
Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
1 (1.00%) low mild
15 (15.00%) high mild
Benchmarking empty_systems/040_systems
Benchmarking empty_systems/040_systems: Warming up for 500.00 ms
Benchmarking empty_systems/040_systems: Collecting 100 samples in estimated 3.0748 s (96k iterations)
Benchmarking empty_systems/040_systems: Analyzing
empty_systems/040_systems
time: [32.120 us 32.152 us 32.182 us]
change: [-8.1865% -7.9840% -7.7842%] (p = 0.00 < 0.05)
Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
6 (6.00%) low severe
5 (5.00%) low mild
1 (1.00%) high mild
Benchmarking empty_systems/045_systems
Benchmarking empty_systems/045_systems: Warming up for 500.00 ms
Benchmarking empty_systems/045_systems: Collecting 100 samples in estimated 3.0445 s (86k iterations)
Benchmarking empty_systems/045_systems: Analyzing
empty_systems/045_systems
time: [35.693 us 35.758 us 35.826 us]
change: [-7.7204% -7.4439% -7.1753%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low severe
1 (1.00%) low mild
Benchmarking empty_systems/050_systems
Benchmarking empty_systems/050_systems: Warming up for 500.00 ms
Benchmarking empty_systems/050_systems: Collecting 100 samples in estimated 3.1263 s (81k iterations)
Benchmarking empty_systems/050_systems: Analyzing
empty_systems/050_systems
time: [38.667 us 38.724 us 38.777 us]
change: [-7.9611% -7.7146% -7.4415%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
Benchmarking empty_systems/055_systems
Benchmarking empty_systems/055_systems: Warming up for 500.00 ms
Benchmarking empty_systems/055_systems: Collecting 100 samples in estimated 3.1943 s (76k iterations)
Benchmarking empty_systems/055_systems: Analyzing
empty_systems/055_systems
time: [41.848 us 41.961 us 42.079 us]
change: [-8.7552% -8.5233% -8.2856%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low mild
7 (7.00%) high mild
Benchmarking empty_systems/060_systems
Benchmarking empty_systems/060_systems: Warming up for 500.00 ms
Benchmarking empty_systems/060_systems: Collecting 100 samples in estimated 3.0125 s (66k iterations)
Benchmarking empty_systems/060_systems: Analyzing
empty_systems/060_systems
time: [46.011 us 46.122 us 46.214 us]
change: [-6.2245% -6.0149% -5.8007%] (p = 0.00 < 0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) low mild
Benchmarking empty_systems/065_systems
Benchmarking empty_systems/065_systems: Warming up for 500.00 ms
Benchmarking empty_systems/065_systems: Collecting 100 samples in estimated 3.1754 s (66k iterations)
Benchmarking empty_systems/065_systems: Analyzing
empty_systems/065_systems
time: [48.413 us 48.469 us 48.523 us]
change: [-9.9712% -9.7502% -9.5391%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) low mild
2 (2.00%) high mild
Benchmarking empty_systems/070_systems
Benchmarking empty_systems/070_systems: Warming up for 500.00 ms
Benchmarking empty_systems/070_systems: Collecting 100 samples in estimated 3.1829 s (61k iterations)
Benchmarking empty_systems/070_systems: Analyzing
empty_systems/070_systems
time: [51.385 us 51.483 us 51.577 us]
change: [-9.5520% -9.3338% -9.1215%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking empty_systems/075_systems
Benchmarking empty_systems/075_systems: Warming up for 500.00 ms
Benchmarking empty_systems/075_systems: Collecting 100 samples in estimated 3.0217 s (56k iterations)
Benchmarking empty_systems/075_systems: Analyzing
empty_systems/075_systems
time: [54.665 us 54.814 us 54.956 us]
change: [-10.538% -10.231% -9.9209%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild
Benchmarking empty_systems/080_systems
Benchmarking empty_systems/080_systems: Warming up for 500.00 ms
Benchmarking empty_systems/080_systems: Collecting 100 samples in estimated 3.2512 s (56k iterations)
Benchmarking empty_systems/080_systems: Analyzing
empty_systems/080_systems
time: [57.672 us 57.761 us 57.854 us]
change: [-9.6153% -9.3481% -9.0961%] (p = 0.00 < 0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
Benchmarking empty_systems/085_systems
Benchmarking empty_systems/085_systems: Warming up for 500.00 ms
Benchmarking empty_systems/085_systems: Collecting 100 samples in estimated 3.1274 s (50k iterations)
Benchmarking empty_systems/085_systems: Analyzing
empty_systems/085_systems
time: [61.579 us 61.717 us 61.860 us]
change: [-8.7599% -8.5540% -8.3610%] (p = 0.00 < 0.05)
Performance has improved.
Benchmarking empty_systems/090_systems
Benchmarking empty_systems/090_systems: Warming up for 500.00 ms
Benchmarking empty_systems/090_systems: Collecting 100 samples in estimated 3.2842 s (50k iterations)
Benchmarking empty_systems/090_systems: Analyzing
empty_systems/090_systems
time: [64.641 us 64.723 us 64.806 us]
change: [-8.6350% -8.4094% -8.1593%] (p = 0.00 < 0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe
Benchmarking empty_systems/095_systems
Benchmarking empty_systems/095_systems: Warming up for 500.00 ms
Benchmarking empty_systems/095_systems: Collecting 100 samples in estimated 3.0770 s (45k iterations)
Benchmarking empty_systems/095_systems: Analyzing
empty_systems/095_systems
time: [67.014 us 67.106 us 67.201 us]
change: [-10.544% -10.322% -10.105%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
2 (2.00%) high mild
2 (2.00%) high severe
Benchmarking empty_systems/100_systems
Benchmarking empty_systems/100_systems: Warming up for 500.00 ms
Benchmarking empty_systems/100_systems: Collecting 100 samples in estimated 3.2127 s (45k iterations)
Benchmarking empty_systems/100_systems: Analyzing
empty_systems/100_systems
time: [70.419 us 70.534 us 70.648 us]
change: [-11.539% -11.311% -11.089%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
Benchmarking busy_systems/01x_entities_03_systems
Benchmarking busy_systems/01x_entities_03_systems: Warming up for 500.00 ms
Benchmarking busy_systems/01x_entities_03_systems: Collecting 100 samples in estimated 3.1465 s (40k iterations)
Benchmarking busy_systems/01x_entities_03_systems: Analyzing
busy_systems/01x_entities_03_systems
time: [77.479 us 77.646 us 77.830 us]
change: [+3.6572% +4.0595% +4.4642%] (p = 0.00 < 0.05)
Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
6 (6.00%) high mild
4 (4.00%) high severe
Benchmarking busy_systems/01x_entities_06_systems
Benchmarking busy_systems/01x_entities_06_systems: Warming up for 500.00 ms
Benchmarking busy_systems/01x_entities_06_systems: Collecting 100 samples in estimated 3.4090 s (20k iterations)
Benchmarking busy_systems/01x_entities_06_systems: Analyzing
busy_systems/01x_entities_06_systems
time: [168.48 us 169.60 us 170.73 us]
change: [+8.1967% +9.0865% +9.9979%] (p = 0.00 < 0.05)
Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
Benchmarking busy_systems/01x_entities_09_systems
Benchmarking busy_systems/01x_entities_09_systems: Warming up for 500.00 ms
Benchmarking busy_systems/01x_entities_09_systems: Collecting 100 samples in estimated 3.7241 s (15k iterations)
Benchmarking busy_systems/01x_entities_09_systems: Analyzing
busy_systems/01x_entities_09_systems
time: [246.17 us 247.69 us 249.25 us]
change: [+4.6941% +5.5947% +6.6011%] (p = 0.00 < 0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe
Benchmarking busy_systems/01x_entities_12_systems
Benchmarking busy_systems/01x_entities_12_systems: Warming up for 500.00 ms
Benchmarking busy_systems/01x_entities_12_systems: Collecting 100 samples in estimated 3.2918 s (10k iterations)
Benchmarking busy_systems/01x_entities_12_systems: Analyzing
busy_systems/01x_entities_12_systems
time: [320.17 us 322.19 us 324.28 us]
change: [+6.7997% +7.6338% +8.5892%] (p = 0.00 < 0.05)
Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low mild
6 (6.00%) high mild
1 (1.00%) high severe
Benchmarking busy_systems/01x_entities_15_systems
Benchmarking busy_systems/01x_entities_15_systems: Warming up for 500.00 ms
Benchmarking busy_systems/01x_entities_15_systems: Collecting 100 samples in estimated 3.9300 s (10k iterations)
Benchmarking busy_systems/01x_entities_15_systems: Analyzing
busy_systems/01x_entities_15_systems
time: [390.36 us 391.86 us 393.39 us]
change: [+1.8419% +2.5739% +3.3137%] (p = 0.00 < 0.05)
Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
Benchmarking busy_systems/02x_entities_03_systems
Benchmarking busy_systems/02x_entities_03_systems: Warming up for 500.00 ms
Benchmarking busy_systems/02x_entities_03_systems: Collecting 100 samples in estimated 3.4048 s (20k iterations)
Benchmarking busy_systems/02x_entities_03_systems: Analyzing
busy_systems/02x_entities_03_systems
time: [166.02 us 167.12 us 168.18 us]
change: [-5.5016% -4.3571% -3.3006%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
7 (7.00%) low mild
1 (1.00%) high mild
Benchmarking busy_systems/02x_entities_06_systems
Benchmarking busy_systems/02x_entities_06_systems: Warming up for 500.00 ms
Benchmarking busy_systems/02x_entities_06_systems: Collecting 100 samples in estimated 3.0983 s (10k iterations)
Benchmarking busy_systems/02x_entities_06_systems: Analyzing
busy_systems/02x_entities_06_systems
time: [307.12 us 309.48 us 312.41 us]
change: [-7.7938% -6.8467% -5.9620%] (p = 0.00 < 0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
Benchmarking busy_systems/02x_entities_09_systems
Benchmarking busy_systems/02x_entities_09_systems: Warming up for 500.00 ms
Benchmarking busy_systems/02x_entities_09_systems: Collecting 100 samples in estimated 4.4165 s (10k iterations)
Benchmarking busy_systems/02x_entities_09_systems: Analyzing
busy_systems/02x_entities_09_systems
time: [434.29 us 436.15 us 438.43 us]
change: [-5.3564% -4.8729% -4.3761%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
Benchmarking busy_systems/02x_entities_12_systems
Benchmarking busy_systems/02x_entities_12_systems: Warming up for 500.00 ms
Benchmarking busy_systems/02x_entities_12_systems: Collecting 100 samples in estimated 5.9657 s (10k iterations)
Benchmarking busy_systems/02x_entities_12_systems: Analyzing
busy_systems/02x_entities_12_systems
time: [589.00 us 591.44 us 594.32 us]
change: [-2.5357% -1.9148% -1.2170%] (p = 0.00 < 0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
4 (4.00%) high mild
6 (6.00%) high severe
Benchmarking busy_systems/02x_entities_15_systems
Benchmarking busy_systems/02x_entities_15_systems: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 3.0s. You may wish to increase target time to 3.7s, enable flat sampling, or reduce sample count to 60.
Benchmarking busy_systems/02x_entities_15_systems: Collecting 100 samples in estimated 3.7230 s (5050 iterations)
Benchmarking busy_systems/02x_entities_15_systems: Analyzing
busy_systems/02x_entities_15_systems
time: [732.79 us 737.39 us 742.26 us]
change: [-7.0343% -6.3843% -5.6171%] (p = 0.00 < 0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
5 (5.00%) high mild
1 (1.00%) high severe
Benchmarking busy_systems/03x_entities_03_systems
Benchmarking busy_systems/03x_entities_03_systems: Warming up for 500.00 ms
Benchmarking busy_systems/03x_entities_03_systems: Collecting 100 samples in estimated 3.1787 s (15k iterations)
Benchmarking busy_systems/03x_entities_03_systems: Analyzing
busy_systems/03x_entities_03_systems
time: [206.65 us 207.26 us 207.95 us]
change: [-16.728% -15.866% -15.031%] (p = 0.00 < 0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild
Benchmarking busy_systems/03x_entities_06_systems
Benchmarking busy_systems/03x_entities_06_systems: Warming up for 500.00 ms
Benchmarking busy_systems/03x_entities_06_systems: Collecting 100 samples in estimated 4.2416 s (10k iterations)
Benchmarking busy_systems/03x_entities_06_systems: Analyzing
busy_systems/03x_entities_06_systems
time: [418.63 us 420.40 us 422.58 us]
change: [-5.4998% -4.9536% -4.4319%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) high mild
5 (5.00%) high severe
Benchmarking busy_systems/03x_entities_09_systems
Benchmarking busy_systems/03x_entities_09_systems: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 3.0s. You may wish to increase target time to 3.0s, enable flat sampling, or reduce sample count to 70.
Benchmarking busy_systems/03x_entities_09_systems: Collecting 100 samples in estimated 3.0401 s (5050 iterations)
Benchmarking busy_systems/03x_entities_09_systems: Analyzing
busy_systems/03x_entities_09_systems
time: [590.83 us 593.33 us 596.68 us]
change: [-24.199% -23.593% -22.935%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
5 (5.00%) high mild
4 (4.00%) high severe
Benchmarking busy_systems/03x_entities_12_systems
Benchmarking busy_systems/03x_entities_12_systems: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 3.0s. You may wish to increase target time to 4.2s, enable flat sampling, or reduce sample count to 50.
Benchmarking busy_systems/03x_entities_12_systems: Collecting 100 samples in estimated 4.1674 s (5050 iterations)
Benchmarking busy_systems/03x_entities_12_systems: Analyzing
busy_systems/03x_entities_12_systems
time: [815.47 us 818.25 us 821.46 us]
change: [-8.3396% -7.7490% -7.0908%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
3 (3.00%) high mild
6 (6.00%) high severe
Benchmarking busy_systems/03x_entities_15_systems
Benchmarking busy_systems/03x_entities_15_systems: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 3.0s. You may wish to increase target time to 5.0s, enable flat sampling, or reduce sample count to 50.
Benchmarking busy_systems/03x_entities_15_systems: Collecting 100 samples in estimated 5.0352 s (5050 iterations)
Benchmarking busy_systems/03x_entities_15_systems: Analyzing
busy_systems/03x_entities_15_systems
time: [986.32 us 988.32 us 990.69 us]
change: [-12.655% -12.119% -11.592%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
Benchmarking busy_systems/04x_entities_03_systems
Benchmarking busy_systems/04x_entities_03_systems: Warming up for 500.00 ms
Benchmarking busy_systems/04x_entities_03_systems: Collecting 100 samples in estimated 4.1049 s (15k iterations)
Benchmarking busy_systems/04x_entities_03_systems: Analyzing
busy_systems/04x_entities_03_systems
time: [264.08 us 265.39 us 267.10 us]
change: [-7.8936% -7.3958% -6.9182%] (p = 0.00 < 0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high severe
Benchmarking busy_systems/04x_entities_06_systems
Benchmarking busy_systems/04x_entities_06_systems: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 3.0s. You may wish to increase target time to 3.5s, enable flat sampling, or reduce sample count to 60.
Benchmarking busy_systems/04x_entities_06_systems: Collecting 100 samples in estimated 3.4622 s (5050 iterations)
Benchmarking busy_systems/04x_entities_06_systems: Analyzing
busy_systems/04x_entities_06_systems
time: [701.66 us 705.20 us 708.76 us]
change: [+24.173% +25.197% +26.162%] (p = 0.00 < 0.05)
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high severe
Benchmarking busy_systems/04x_entities_09_systems
Benchmarking busy_systems/04x_entities_09_systems: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 3.0s. You may wish to increase target time to 4.3s, enable flat sampling, or reduce sample count to 50.
Benchmarking busy_systems/04x_entities_09_systems: Collecting 100 samples in estimated 4.3103 s (5050 iterations)
Benchmarking busy_systems/04x_entities_09_systems: Analyzing
busy_systems/04x_entities_09_systems
time: [836.70 us 840.23 us 844.77 us]
change: [-18.621% -17.918% -17.142%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low mild
3 (3.00%) high mild
5 (5.00%) high severe
Benchmarking busy_systems/04x_entities_12_systems
Benchmarking busy_systems/04x_entities_12_systems: Warming up for 500.00 ms
Benchmarking busy_systems/04x_entities_12_systems: Collecting 100 samples in estimated 3.1137 s (2500 iterations)
Benchmarking busy_systems/04x_entities_12_systems: Analyzing
busy_systems/04x_entities_12_systems
time: [1.2381 ms 1.2447 ms 1.2526 ms]
change: [-10.682% -9.9284% -9.0505%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low severe
3 (3.00%) high mild
3 (3.00%) high severe
Benchmarking busy_systems/04x_entities_15_systems
Benchmarking busy_systems/04x_entities_15_systems: Warming up for 500.00 ms
Benchmarking busy_systems/04x_entities_15_systems: Collecting 100 samples in estimated 3.0958 s (2300 iterations)
Benchmarking busy_systems/04x_entities_15_systems: Analyzing
busy_systems/04x_entities_15_systems
time: [1.3320 ms 1.3395 ms 1.3495 ms]
change: [-6.4833% -5.8871% -5.0671%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
Benchmarking busy_systems/05x_entities_03_systems
Benchmarking busy_systems/05x_entities_03_systems: Warming up for 500.00 ms
Benchmarking busy_systems/05x_entities_03_systems: Collecting 100 samples in estimated 4.0839 s (10k iterations)
Benchmarking busy_systems/05x_entities_03_systems: Analyzing
busy_systems/05x_entities_03_systems
time: [401.74 us 409.11 us 416.19 us]
change: [+3.1613% +4.6876% +6.4640%] (p = 0.00 < 0.05)
Performance has regressed.
Benchmarking busy_systems/05x_entities_06_systems
Benchmarking busy_systems/05x_entities_06_systems: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 3.0s. You may wish to increase target time to 4.1s, enable flat sampling, or reduce sample count to 60.
Benchmarking busy_systems/05x_entities_06_systems: Collecting 100 samples in estimated 4.1014 s (5050 iterations)
Benchmarking busy_systems/05x_entities_06_systems: Analyzing
busy_systems/05x_entities_06_systems
time: [803.79 us 807.45 us 811.17 us]
change: [+20.112% +20.852% +21.648%] (p = 0.00 < 0.05)
Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low severe
3 (3.00%) low mild
1 (1.00%) high mild
5 (5.00%) high severe
Benchmarking busy_systems/05x_entities_09_systems
Benchmarking busy_systems/05x_entities_09_systems: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 3.0s. You may wish to increase target time to 5.8s, enable flat sampling, or reduce sample count to 50.
Benchmarking busy_systems/05x_entities_09_systems: Collecting 100 samples in estimated 5.8422 s (5050 iterations)
Benchmarking busy_systems/05x_entities_09_systems: Analyzing
busy_systems/05x_entities_09_systems
time: [1.1529 ms 1.1562 ms 1.1599 ms]
change: [-7.0057% -6.4936% -5.9409%] (p = 0.00 < 0.05)
Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
4 (4.00%) low mild
6 (6.00%) high mild
1 (1.00%) high severe
Benchmarking busy_systems/05x_entities_12_systems
Benchmarking busy_systems/05x_entities_12_systems: Warming up for 500.00 ms
Benchmarking busy_systems/05x_entities_12_systems: Collecting 100 samples in estimated 3.0991 s (2100 iterations)
Benchmarking busy_systems/05x_entities_12_systems: Analyzing
busy_systems/05x_entities_12_systems
time: [1.4792 ms 1.4832 ms 1.4877 ms]
change: [-6.6721% -6.2530% -5.8034%] (p = 0.00 < 0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
Benchmarking busy_systems/05x_entities_15_systems
Benchmarking busy_systems/05x_entities_15_systems: Warming up for 500.00 ms
Benchmarking busy_systems/05x_entities_15_systems: Collecting 100 samples in estimated 3.1274 s (1900 iterations)
Benchmarking busy_systems/05x_entities_15_systems: Analyzing
busy_systems/05x_entities_15_systems
time: [1.6442 ms 1.6489 ms 1.6546 ms]
change: [-14.777% -14.019% -13.190%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
Benchmarking contrived/01x_entities_03_systems
Benchmarking contrived/01x_entities_03_systems: Warming up for 500.00 ms
Benchmarking contrived/01x_entities_03_systems: Collecting 100 samples in estimated 3.1621 s (76k iterations)
Benchmarking contrived/01x_entities_03_systems: Analyzing
contrived/01x_entities_03_systems
time: [41.461 us 41.708 us 42.011 us]
change: [-5.1583% -4.3486% -3.5811%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
Benchmarking contrived/01x_entities_06_systems
Benchmarking contrived/01x_entities_06_systems: Warming up for 500.00 ms
Benchmarking contrived/01x_entities_06_systems: Collecting 100 samples in estimated 3.2021 s (40k iterations)
Benchmarking contrived/01x_entities_06_systems: Analyzing
contrived/01x_entities_06_systems
time: [79.063 us 79.304 us 79.551 us]
change: [-3.4858% -2.6393% -1.8022%] (p = 0.00 < 0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
Benchmarking contrived/01x_entities_09_systems
Benchmarking contrived/01x_entities_09_systems: Warming up for 500.00 ms
Benchmarking contrived/01x_entities_09_systems: Collecting 100 samples in estimated 3.4181 s (30k iterations)
Benchmarking contrived/01x_entities_09_systems: Analyzing
contrived/01x_entities_09_systems
time: [114.79 us 115.38 us 115.99 us]
change: [-9.3136% -8.6247% -7.9455%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking contrived/01x_entities_12_systems
Benchmarking contrived/01x_entities_12_systems: Warming up for 500.00 ms
Benchmarking contrived/01x_entities_12_systems: Collecting 100 samples in estimated 3.0530 s (20k iterations)
Benchmarking contrived/01x_entities_12_systems: Analyzing
contrived/01x_entities_12_systems
time: [155.00 us 155.93 us 156.76 us]
change: [-2.9018% -2.1434% -1.3431%] (p = 0.00 < 0.05)
Performance has improved.
Benchmarking contrived/01x_entities_15_systems
Benchmarking contrived/01x_entities_15_systems: Warming up for 500.00 ms
Benchmarking contrived/01x_entities_15_systems: Collecting 100 samples in estimated 3.8754 s (20k iterations)
Benchmarking contrived/01x_entities_15_systems: Analyzing
contrived/01x_entities_15_systems
time: [186.65 us 187.76 us 189.03 us]
change: [-5.2682% -4.6178% -3.9702%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking contrived/02x_entities_03_systems
Benchmarking contrived/02x_entities_03_systems: Warming up for 500.00 ms
Benchmarking contrived/02x_entities_03_systems: Collecting 100 samples in estimated 3.1252 s (45k iterations)
Benchmarking contrived/02x_entities_03_systems: Analyzing
contrived/02x_entities_03_systems
time: [68.856 us 69.470 us 70.097 us]
change: [-1.1155% -0.2422% +0.6851%] (p = 0.61 > 0.05)
No change in performance detected.
Benchmarking contrived/02x_entities_06_systems
Benchmarking contrived/02x_entities_06_systems: Warming up for 500.00 ms
Benchmarking contrived/02x_entities_06_systems: Collecting 100 samples in estimated 3.5056 s (25k iterations)
Benchmarking contrived/02x_entities_06_systems: Analyzing
contrived/02x_entities_06_systems
time: [142.76 us 144.07 us 145.37 us]
change: [-9.6781% -8.7615% -7.8968%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking contrived/02x_entities_09_systems
Benchmarking contrived/02x_entities_09_systems: Warming up for 500.00 ms
Benchmarking contrived/02x_entities_09_systems: Collecting 100 samples in estimated 3.2815 s (15k iterations)
Benchmarking contrived/02x_entities_09_systems: Analyzing
contrived/02x_entities_09_systems
time: [218.05 us 219.32 us 220.58 us]
change: [-0.9170% -0.0598% +0.6834%] (p = 0.89 > 0.05)
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) low mild
Benchmarking contrived/02x_entities_12_systems
Benchmarking contrived/02x_entities_12_systems: Warming up for 500.00 ms
Benchmarking contrived/02x_entities_12_systems: Collecting 100 samples in estimated 4.3186 s (15k iterations)
Benchmarking contrived/02x_entities_12_systems: Analyzing
contrived/02x_entities_12_systems
time: [283.14 us 284.39 us 285.61 us]
change: [-3.9887% -3.3212% -2.6587%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high mild
Benchmarking contrived/02x_entities_15_systems
Benchmarking contrived/02x_entities_15_systems: Warming up for 500.00 ms
Benchmarking contrived/02x_entities_15_systems: Collecting 100 samples in estimated 3.5671 s (10k iterations)
Benchmarking contrived/02x_entities_15_systems: Analyzing
contrived/02x_entities_15_systems
time: [349.51 us 351.08 us 352.70 us]
change: [-4.0491% -3.4162% -2.7658%] (p = 0.00 < 0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) low mild
1 (1.00%) high mild
Benchmarking contrived/03x_entities_03_systems
Benchmarking contrived/03x_entities_03_systems: Warming up for 500.00 ms
Benchmarking contrived/03x_entities_03_systems: Collecting 100 samples in estimated 3.4563 s (30k iterations)
Benchmarking contrived/03x_entities_03_systems: Analyzing
contrived/03x_entities_03_systems
time: [114.11 us 115.22 us 116.28 us]
change: [+18.539% +20.412% +22.194%] (p = 0.00 < 0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) low mild
2 (2.00%) high mild
Benchmarking contrived/03x_entities_06_systems
Benchmarking contrived/03x_entities_06_systems: Warming up for 500.00 ms
Benchmarking contrived/03x_entities_06_systems: Collecting 100 samples in estimated 3.2301 s (15k iterations)
Benchmarking contrived/03x_entities_06_systems: Analyzing
contrived/03x_entities_06_systems
time: [203.69 us 206.29 us 209.29 us]
change: [+8.5553% +9.8439% +11.130%] (p = 0.00 < 0.05)
Performance has regressed.
Benchmarking contrived/03x_entities_09_systems
Benchmarking contrived/03x_entities_09_systems: Warming up for 500.00 ms
Benchmarking contrived/03x_entities_09_systems: Collecting 100 samples in estimated 3.0869 s (10k iterations)
Benchmarking contrived/03x_entities_09_systems: Analyzing
contrived/03x_entities_09_systems
time: [301.72 us 303.48 us 305.23 us]
change: [-7.0037% -5.9763% -4.9063%] (p = 0.00 < 0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
Benchmarking contrived/03x_entities_12_systems
Benchmarking contrived/03x_entities_12_systems: Warming up for 500.00 ms
Benchmarking contrived/03x_entities_12_systems: Collecting 100 samples in estimated 3.9264 s (10k iterations)
Benchmarking contrived/03x_entities_12_systems: Analyzing
contrived/03x_entities_12_systems
time: [391.56 us 395.16 us 399.36 us]
change: [-5.3395% -4.4862% -3.5254%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) high mild
1 (1.00%) high severe
Benchmarking contrived/03x_entities_15_systems
Benchmarking contrived/03x_entities_15_systems: Warming up for 500.00 ms
Benchmarking contrived/03x_entities_15_systems: Collecting 100 samples in estimated 4.8334 s (10k iterations)
Benchmarking contrived/03x_entities_15_systems: Analyzing
contrived/03x_entities_15_systems
time: [478.01 us 479.89 us 481.77 us]
change: [-6.0153% -5.4015% -4.7542%] (p = 0.00 < 0.05)
Performance has improved.
Benchmarking contrived/04x_entities_03_systems
Benchmarking contrived/04x_entities_03_systems: Warming up for 500.00 ms
Benchmarking contrived/04x_entities_03_systems: Collecting 100 samples in estimated 3.4562 s (25k iterations)
Benchmarking contrived/04x_entities_03_systems: Analyzing
contrived/04x_entities_03_systems
time: [134.75 us 135.55 us 136.33 us]
change: [-12.643% -11.577% -10.447%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking contrived/04x_entities_06_systems
Benchmarking contrived/04x_entities_06_systems: Warming up for 500.00 ms
Benchmarking contrived/04x_entities_06_systems: Collecting 100 samples in estimated 3.9668 s (15k iterations)
Benchmarking contrived/04x_entities_06_systems: Analyzing
contrived/04x_entities_06_systems
time: [264.00 us 265.77 us 267.65 us]
change: [-1.1567% -0.1135% +0.9133%] (p = 0.84 > 0.05)
No change in performance detected.
Benchmarking contrived/04x_entities_09_systems
Benchmarking contrived/04x_entities_09_systems: Warming up for 500.00 ms
Benchmarking contrived/04x_entities_09_systems: Collecting 100 samples in estimated 3.9394 s (10k iterations)
Benchmarking contrived/04x_entities_09_systems: Analyzing
contrived/04x_entities_09_systems
time: [388.72 us 391.31 us 393.90 us]
change: [-6.1555% -5.2817% -4.3973%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking contrived/04x_entities_12_systems
Benchmarking contrived/04x_entities_12_systems: Warming up for 500.00 ms
Benchmarking contrived/04x_entities_12_systems: Collecting 100 samples in estimated 5.2335 s (10k iterations)
Benchmarking contrived/04x_entities_12_systems: Analyzing
contrived/04x_entities_12_systems
time: [508.85 us 511.80 us 514.69 us]
change: [-4.9155% -3.9297% -2.9575%] (p = 0.00 < 0.05)
Performance has improved.
Benchmarking contrived/04x_entities_15_systems
Benchmarking contrived/04x_entities_15_systems: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 3.0s. You may wish to increase target time to 3.2s, enable flat sampling, or reduce sample count to 60.
Benchmarking contrived/04x_entities_15_systems: Collecting 100 samples in estimated 3.1893 s (5050 iterations)
Benchmarking contrived/04x_entities_15_systems: Analyzing
contrived/04x_entities_15_systems
time: [632.06 us 636.15 us 640.03 us]
change: [-6.8164% -5.8750% -4.9047%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking contrived/05x_entities_03_systems
Benchmarking contrived/05x_entities_03_systems: Warming up for 500.00 ms
Benchmarking contrived/05x_entities_03_systems: Collecting 100 samples in estimated 3.1470 s (20k iterations)
Benchmarking contrived/05x_entities_03_systems: Analyzing
contrived/05x_entities_03_systems
time: [155.26 us 155.70 us 156.15 us]
change: [-14.819% -14.241% -13.616%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
Benchmarking contrived/05x_entities_06_systems
Benchmarking contrived/05x_entities_06_systems: Warming up for 500.00 ms
Benchmarking contrived/05x_entities_06_systems: Collecting 100 samples in estimated 3.1365 s (10k iterations)
Benchmarking contrived/05x_entities_06_systems: Analyzing
contrived/05x_entities_06_systems
time: [307.82 us 309.68 us 311.64 us]
change: [-7.2611% -6.3805% -5.4363%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
5 (5.00%) high mild
Benchmarking contrived/05x_entities_09_systems
Benchmarking contrived/05x_entities_09_systems: Warming up for 500.00 ms
Benchmarking contrived/05x_entities_09_systems: Collecting 100 samples in estimated 4.6260 s (10k iterations)
Benchmarking contrived/05x_entities_09_systems: Analyzing
contrived/05x_entities_09_systems
time: [453.72 us 458.94 us 464.55 us]
change: [-9.9365% -8.9761% -8.0166%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking contrived/05x_entities_12_systems
Benchmarking contrived/05x_entities_12_systems: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 3.0s. You may wish to increase target time to 3.2s, enable flat sampling, or reduce sample count to 60.
Benchmarking contrived/05x_entities_12_systems: Collecting 100 samples in estimated 3.1823 s (5050 iterations)
Benchmarking contrived/05x_entities_12_systems: Analyzing
contrived/05x_entities_12_systems
time: [612.77 us 619.52 us 626.33 us]
change: [-4.6612% -3.3202% -1.9764%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking contrived/05x_entities_15_systems
Benchmarking contrived/05x_entities_15_systems: Warming up for 500.00 ms
Warning: Unable to complete 100 samples in 3.0s. You may wish to increase target time to 4.0s, enable flat sampling, or reduce sample count to 60.
Benchmarking contrived/05x_entities_15_systems: Collecting 100 samples in estimated 4.0445 s (5050 iterations)
Benchmarking contrived/05x_entities_15_systems: Analyzing
contrived/05x_entities_15_systems
time: [780.76 us 790.28 us 800.10 us]
change: [-2.5465% -1.3942% -0.1122%] (p = 0.02 < 0.05)
Change within noise threshold.