-
Notifications
You must be signed in to change notification settings - Fork 145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bump Timeit version to v0.1.14 #4766
Conversation
Datadog ReportBranch report: ✅ |
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4766) - mean (68ms) : 66, 71
. : milestone, 68,
section CallTarget+Inlining+NGEN
This PR (4766) - mean (981ms) : 961, 1000
. : milestone, 981,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4766) - mean (106ms) : 104, 108
. : milestone, 106,
section CallTarget+Inlining+NGEN
This PR (4766) - mean (670ms) : 655, 685
. : milestone, 670,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4766) - mean (90ms) : 89, 92
. : milestone, 90,
section CallTarget+Inlining+NGEN
This PR (4766) - mean (633ms) : 619, 647
. : milestone, 633,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4766) - mean (187ms) : 185, 190
. : milestone, 187,
master - mean (189ms) : 187, 191
. : milestone, 189,
section CallTarget+Inlining+NGEN
This PR (4766) - mean (1,124ms) : 1101, 1148
. : milestone, 1124,
master - mean (1,135ms) : 1117, 1153
. : milestone, 1135,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4766) - mean (271ms) : 266, 276
. : milestone, 271,
master - mean (273ms) : 270, 277
. : milestone, 273,
section CallTarget+Inlining+NGEN
This PR (4766) - mean (1,088ms) : 1070, 1106
. : milestone, 1088,
master - mean (1,094ms) : 1074, 1115
. : milestone, 1094,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4766) - mean (261ms) : 257, 265
. : milestone, 261,
master - mean (264ms) : 259, 268
. : milestone, 264,
section CallTarget+Inlining+NGEN
This PR (4766) - mean (1,055ms) : 1031, 1079
. : milestone, 1055,
master - mean (1,064ms) : 1041, 1087
. : milestone, 1064,
|
Throughput/Crank Report:zap:Throughput results for AspNetCoreSimpleController comparing the following branches/commits: Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red. Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards! gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4766) (11.708M) : 0, 11707990
master (10.849M) : 0, 10848650
benchmarks/2.38.0 (11.867M) : 0, 11866956
benchmarks/2.9.0 (11.404M) : 0, 11403964
section Automatic
This PR (4766) (8.225M) : 0, 8225187
master (7.689M) : 0, 7688765
benchmarks/2.38.0 (8.176M) : 0, 8175636
benchmarks/2.9.0 (8.122M) : 0, 8121934
section Trace stats
This PR (4766) (8.419M) : 0, 8418804
master (7.974M) : 0, 7973986
benchmarks/2.38.0 (8.450M) : 0, 8450370
section Manual
This PR (4766) (10.397M) : 0, 10397073
master (9.776M) : 0, 9775925
benchmarks/2.38.0 (10.334M) : 0, 10334368
section Manual + Automatic
This PR (4766) (7.721M) : 0, 7720994
master (7.300M) : 0, 7299779
benchmarks/2.38.0 (7.750M) : 0, 7750484
section Version Conflict
This PR (4766) (7.035M) : 0, 7034738
master (6.575M) : 0, 6574617
benchmarks/2.38.0 (7.137M) : 0, 7136691
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4766) (9.537M) : 0, 9537150
master (9.534M) : 0, 9534375
benchmarks/2.38.0 (9.548M) : 0, 9548121
benchmarks/2.9.0 (9.656M) : 0, 9655806
section Automatic
This PR (4766) (6.798M) : 0, 6798424
master (6.740M) : 0, 6739842
benchmarks/2.38.0 (6.747M) : 0, 6747227
section Trace stats
This PR (4766) (6.952M) : 0, 6951643
master (6.775M) : 0, 6774905
benchmarks/2.38.0 (6.815M) : 0, 6814846
section Manual
This PR (4766) (8.438M) : 0, 8438164
master (8.302M) : 0, 8302488
benchmarks/2.38.0 (8.263M) : 0, 8263131
section Manual + Automatic
This PR (4766) (6.116M) : 0, 6115847
master (6.349M) : 0, 6348627
benchmarks/2.38.0 (6.275M) : 0, 6275411
section Version Conflict
This PR (4766) (5.480M) : 0, 5480441
master (5.749M) : 0, 5749334
benchmarks/2.38.0 (5.670M) : 0, 5669744
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4766) (7.112M) : 0, 7111617
master (9.450M) : 0, 9449674
benchmarks/2.38.0 (9.007M) : 0, 9007066
benchmarks/2.9.0 (9.731M) : 0, 9730905
section Automatic
This PR (4766) (6.602M) : 0, 6602341
master (6.635M) : 0, 6634884
benchmarks/2.38.0 (6.423M) : 0, 6422756
benchmarks/2.9.0 (6.908M) : 0, 6908188
section Trace stats
This PR (4766) (6.844M) : 0, 6844275
master (6.864M) : 0, 6863822
benchmarks/2.38.0 (6.721M) : 0, 6720838
section Manual
This PR (4766) (8.229M) : 0, 8228654
master (8.250M) : 0, 8250092
benchmarks/2.38.0 (7.977M) : 0, 7977494
section Manual + Automatic
This PR (4766) (6.512M) : 0, 6511883
master (6.462M) : 0, 6462400
benchmarks/2.38.0 (6.349M) : 0, 6349066
section Version Conflict
This PR (4766) (5.830M) : 0, 5830396
master (5.794M) : 0, 5794375
benchmarks/2.38.0 (5.682M) : 0, 5681654
gantt
title Throughput Linux x64 (ASM) (Total requests)
dateFormat X
axisFormat %s
section Baseline
master (7.751M) : 0, 7750914
benchmarks/2.38.0 (7.578M) : 0, 7577544
benchmarks/2.9.0 (7.904M) : 0, 7903721
section No attack
master (2.190M) : 0, 2190399
benchmarks/2.38.0 (2.184M) : 0, 2184279
benchmarks/2.9.0 (3.239M) : 0, 3238804
section Attack
master (1.724M) : 0, 1723833
benchmarks/2.38.0 (1.711M) : 0, 1710542
benchmarks/2.9.0 (2.459M) : 0, 2458912
section Blocking
master (3.465M) : 0, 3464934
benchmarks/2.38.0 (3.486M) : 0, 3486124
section IAST default
master (6.848M) : 0, 6847970
section IAST full
master (6.223M) : 0, 6222736
section Base vuln
master (0.962M) : 0, 961814
section IAST vuln
master (0.904M) : 0, 903897
|
Summary of changes
This PR bumps the version of timeit tool from v0.1.3 to v0.1.14