Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Changes at 4/8/2022 8:59:46 PM #4576

Closed
performanceautofiler bot opened this issue Apr 12, 2022 · 3 comments
Closed

[Perf] Changes at 4/8/2022 8:59:46 PM #4576

performanceautofiler bot opened this issue Apr 12, 2022 · 3 comments
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Apr 12, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 24.17 μs 26.22 μs 1.08 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.222932209930544 > 25.25924379029418.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/21/2022 1:28:14 PM, 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -33.13322634367094 (T) = (0 -26194.832335236242) / Math.Sqrt((16980.837561018365 / (39)) + (65952.09843920812 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.0881224660590953 = (24073.42293933817 - 26194.832335236242) / 24073.42293933817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentBag - Duration of single invocation 2.14 μs 2.49 μs 1.16 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ConcurrentBag(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4930842014102157 > 2.2449272418560535.
IsChangePoint: Marked as a change because one of 3/3/2022 7:35:51 PM, 3/11/2022 8:42:43 PM, 3/23/2022 6:35:24 PM, 3/29/2022 11:07:04 AM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -70.44286894837911 (T) = (0 -2495.333820793321) / Math.Sqrt((968.7619655387663 / (39)) + (3.1703155392944633 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.16440848707366165 = (2143.005524688746 - 2495.333820793321) / 2143.005524688746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateGetPositionSingleSegment - Duration of single invocation 32.93 ns 35.27 ns 1.07 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionSingleSegment

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.27249963223523 > 34.58420222384465.
IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -15.834575901314098 (T) = (0 -34.85541144276243) / Math.Sqrt((0.2389035291315352 / (39)) + (0.10070296015102535 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.05172600629783976 = (33.141152005412785 - 34.85541144276243) / 33.141152005412785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Char_ToLowerInvariant - Duration of single invocation 22.45 ns 29.28 ns 1.30 0.08 False
Char_ToUpperInvariant - Duration of single invocation 33.09 ns 37.04 ns 1.12 0.10 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.27720794595902 > 23.783293586902115.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/24/2022 10:17:16 AM, 3/4/2022 3:52:45 PM, 3/11/2022 8:42:43 PM, 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -122.19227935359008 (T) = (0 -29.33356670733879) / Math.Sqrt((0.07683104693362362 / (39)) + (0.01596188379132147 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.2864184013977807 = (22.802508635966248 - 29.33356670733879) / 22.802508635966248 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.Char_ToUpperInvariant(input: "Hello World!")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.03828763176484 > 34.81715721542249.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/23/2022 5:12:02 AM, 3/16/2022 12:48:00 PM, 3/21/2022 1:28:14 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -133.42685412240755 (T) = (0 -37.0531314665092) / Math.Sqrt((0.016902597752947876 / (39)) + (0.007777958102332498 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.11849136122943592 = (33.12777617323814 - 37.0531314665092) / 33.12777617323814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.CtorFromCollectionNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 29.17 μs 31.40 μs 1.08 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollectionNonGeneric<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.402713462639568 > 30.442382035450507.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -11.96348823909402 (T) = (0 -31390.121849613748) / Math.Sqrt((1219808.6472099978 / (39)) + (116715.47999533376 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.0799845400953987 = (29065.343700976453 - 31390.121849613748) / 29065.343700976453 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Linq.Tests.Perf_OrderBy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OrderByValueType - Duration of single invocation 54.44 μs 57.48 μs 1.06 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_OrderBy*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_OrderBy.OrderByValueType(NumberOfPeople: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.482879792542015 > 57.25308219410163.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 3/29/2022 11:07:04 AM, 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -60.71499059062252 (T) = (0 -57433.543226083646) / Math.Sqrt((57904.73452049728 / (39)) + (10126.865132610692 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.050235659808586006 = (54686.33890849924 - 57433.543226083646) / 54686.33890849924 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.IterateForNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayList - Duration of single invocation 1.45 μs 1.61 μs 1.11 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForNonGeneric&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForNonGeneric<Int32>.ArrayList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6085317357895048 > 1.524575460310654.
IsChangePoint: Marked as a change because one of 2/4/2022 6:00:25 PM, 2/12/2022 12:09:13 AM, 2/19/2022 11:51:48 PM, 3/4/2022 3:52:45 PM, 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/23/2022 3:13:36 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -12.047675136062567 (T) = (0 -1608.1628730034784) / Math.Sqrt((5235.722749420881 / (39)) + (0.24785821064339852 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.0950579892306685 = (1468.5641206392102 - 1608.1628730034784) / 1468.5641206392102 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 2.84 μs 3.31 μs 1.17 0.26 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.Stack(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3129564521741752 > 2.983069160348327.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -19.149407454553266 (T) = (0 -3283.4104539573236) / Math.Sqrt((6786.216701386223 / (39)) + (3636.233110815095 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.12751361846411774 = (2912.0805285083266 - 3283.4104539573236) / 2912.0805285083266 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 83.66 ns 90.31 ns 1.08 0.08 False
EncodeUtf16 - Duration of single invocation 108.58 ns 115.86 ns 1.07 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,©2020,512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.31272499811051 > 87.74520586030843.
IsChangePoint: Marked as a change because one of 2/21/2022 2:16:42 PM, 3/3/2022 7:35:51 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -12.321904327319015 (T) = (0 -92.1991133668373) / Math.Sqrt((2.994037618511004 / (39)) + (4.64236409287772 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.08379386223687786 = (85.0707100117217 - 92.1991133668373) / 85.0707100117217 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 115.86048335006498 > 114.39500547051426.
IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/29/2022 11:07:04 AM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -9.689921666491882 (T) = (0 -116.15381376850921) / Math.Sqrt((1.4565102635441782 / (39)) + (8.543542226116443 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.06348139040908804 = (109.2203538454288 - 116.15381376850921) / 109.2203538454288 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLineAsync - Duration of single invocation 21.53 μs 27.05 μs 1.26 0.01 False
ReadLineAsync - Duration of single invocation 21.46 μs 25.90 μs 1.21 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.047377409297052 > 22.556814918478338.
IsChangePoint: Marked as a change because one of 2/24/2022 10:17:16 AM, 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -51.2063026531744 (T) = (0 -26141.089020456988) / Math.Sqrt((45832.41946916701 / (39)) + (144546.44028967695 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.23143960805798175 = (21228.072289864293 - 26141.089020456988) / 21228.072289864293 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.89828390993959 > 22.658561493612716.
IsChangePoint: Marked as a change because one of 2/24/2022 10:17:16 AM, 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -69.48965885335853 (T) = (0 -25904.05210476315) / Math.Sqrt((44912.81545588085 / (39)) + (49328.05893952455 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.2009939397796785 = (21568.844976449447 - 25904.05210476315) / 21568.844976449447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 13.03 ns 15.44 ns 1.18 0.49 False
SequenceEqual - Duration of single invocation 13.22 ns 14.74 ns 1.12 0.10 False
BinarySearch - Duration of single invocation 13.13 ns 16.35 ns 1.25 0.09 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.LastIndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.439962444559955 > 14.900140201401067.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 2/24/2022 10:17:16 AM, 3/11/2022 8:42:43 PM, 3/21/2022 1:28:14 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -10.24844448484873 (T) = (0 -15.30584233759866) / Math.Sqrt((1.4595934210259047 / (39)) + (0.32538490567838196 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.1872894185671829 = (12.891416446774791 - 15.30584233759866) / 12.891416446774791 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.SequenceEqual(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.73969601972988 > 13.902668539397569.
IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 4/4/2022 8:14:13 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -17.571482898430887 (T) = (0 -14.57997749063767) / Math.Sqrt((0.03204914487118026 / (17)) + (0.14470823956598625 / (40))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (40) - 2, .025) and -0.09817995995997533 = (13.27649203430106 - 14.57997749063767) / 13.27649203430106 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.BinarySearch(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.352312156389807 > 13.360557518355241.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 2/19/2022 11:51:48 PM, 2/25/2022 11:02:46 PM, 3/17/2022 6:54:53 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -78.98468015637509 (T) = (0 -16.154763871463985) / Math.Sqrt((0.027376226211249174 / (39)) + (0.019334620340563147 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.2595276796908698 = (12.826049107097752 - 16.154763871463985) / 12.826049107097752 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HeapSort - Duration of single invocation 255.00 ns 277.85 ns 1.09 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 10)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 277.848078601306 > 268.22308977882767.
IsChangePoint: Marked as a change because one of 2/10/2022 7:02:09 PM, 2/25/2022 7:19:51 AM, 3/17/2022 6:54:53 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -49.29452651722794 (T) = (0 -278.60089842069203) / Math.Sqrt((6.220631268190085 / (39)) + (0.7268071445043258 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.08589982822028323 = (256.56224559617084 - 278.60089842069203) / 256.56224559617084 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Apr 12, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadOnlySpan - Duration of single invocation 249.86 ns 327.89 ns 1.31 0.09 False
Span - Duration of single invocation 255.69 ns 330.70 ns 1.29 0.10 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<String>.ReadOnlySpan(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 327.8935167574049 > 262.3388136795414.
IsChangePoint: Marked as a change because one of 2/12/2022 4:06:00 AM, 3/11/2022 8:42:43 PM, 3/15/2022 12:53:39 AM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -1185.3362829351306 (T) = (0 -327.689064267649) / Math.Sqrt((0.054426647888173654 / (39)) + (0.05245201225636542 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.31141170995154654 = (249.8750482255159 - 327.689064267649) / 249.8750482255159 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 330.7041763768503 > 268.4778297299551.
IsChangePoint: Marked as a change because one of 2/12/2022 4:06:00 AM, 3/11/2022 8:42:43 PM, 3/15/2022 12:53:39 AM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -491.36276633659617 (T) = (0 -330.9828697138345) / Math.Sqrt((0.5386300641317153 / (39)) + (0.16878850719737265 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.29209522230132956 = (256.1598123738329 - 330.9828697138345) / 256.1598123738329 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 3.72 μs 4.60 μs 1.24 0.16 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.599030696633586 > 3.8968255224144253.
IsChangePoint: Marked as a change because one of 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -30.63234424384296 (T) = (0 -4584.725024232314) / Math.Sqrt((23224.176759821294 / (39)) + (2030.849098881888 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.2162787123007159 = (3769.469101008795 - 4584.725024232314) / 3769.469101008795 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
StaticDelegate - Duration of single invocation 280.64 μs 311.83 μs 1.11 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.StaticDelegate

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 311.82996732026146 > 294.67731445312506.
IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 2/8/2022 5:20:12 PM, 2/19/2022 11:51:48 PM, 3/4/2022 3:52:45 PM, 3/12/2022 3:27:04 PM, 3/23/2022 6:35:24 PM, 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -908.9931328355216 (T) = (0 -311828.32880071545) / Math.Sqrt((45770.95463357383 / (39)) + (10.697792789532569 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.11097408746392766 = (280680.10975174094 - 311828.32880071545) / 280680.10975174094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 3.23 μs 4.02 μs 1.24 0.21 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<String>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.0193518518518525 > 3.448403953066163.
IsChangePoint: Marked as a change because one of 4/2/2022 3:40:16 AM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -21.747891689441108 (T) = (0 -4047.1641407843) / Math.Sqrt((19782.13638260721 / (39)) + (9458.898384843993 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.20873225270052279 = (3348.2718209448085 - 4047.1641407843) / 3348.2718209448085 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ForEach_Array - Duration of single invocation 6.55 ns 7.56 ns 1.15 0.14 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.561889195225527 > 6.875451282431633.
IsChangePoint: Marked as a change because one of 2/23/2022 5:12:02 AM, 3/23/2022 6:35:24 PM, 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -709.7673406407654 (T) = (0 -7.561885719190449) / Math.Sqrt((1.5362099429496015E-06 / (39)) + (3.601282496813482E-05 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.15481964156412847 = (6.54810971949556 - 7.561885719190449) / 6.54810971949556 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 14.40 ns 15.44 ns 1.07 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.Parse(value: "65535")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.44329697182096 > 15.111301105077596.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -5.278518391351173 (T) = (0 -15.80415339828473) / Math.Sqrt((0.15717345968196803 / (39)) + (0.9457649395473001 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.0862962871868082 = (14.548658211115585 - 15.80415339828473) / 14.548658211115585 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf_Word_NotFound - Duration of single invocation 655.73 ns 737.96 ns 1.13 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 737.955548771897 > 684.77563577895.
IsChangePoint: Marked as a change because one of 2/4/2022 6:00:25 PM, 2/19/2022 11:51:48 PM, 2/26/2022 3:32:05 AM, 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/18/2022 9:29:53 AM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -110.52164990843524 (T) = (0 -733.1169744700421) / Math.Sqrt((1.4483322172711608 / (39)) + (8.818545332855074 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.12289733923323122 = (652.8797859389799 - 733.1169744700421) / 652.8797859389799 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline dee2bb50f12e5d6ef16eee7ed944b0fb7ef27053
Compare 285b8259af30e255980b1ebfb2ddd60ca291a83a
Diff Diff

Regressions in Span.Sorting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BubbleSortArray - Duration of single invocation 210.46 μs 370.57 μs 1.76 0.11 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.Sorting*'

Payloads

Baseline
Compare

Histogram

Span.Sorting.BubbleSortArray(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 370.56865999999997 > 220.973343.
IsChangePoint: Marked as a change because one of 2/4/2022 6:00:25 PM, 2/19/2022 11:51:48 PM, 3/21/2022 1:28:14 PM, 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -3995.6758095288747 (T) = (0 -370743.08153439156) / Math.Sqrt((2265.3533806912064 / (39)) + (27923.429165602334 / (18))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (18) - 2, .025) and -0.7616861385422146 = (210447.86209260827 - 370743.08153439156) / 210447.86209260827 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@DrewScoggins
Copy link
Member

Over same time period as dotnet/runtime#66357

Could also be dotnet/runtime#62843

@performanceautofiler
Copy link
Author

Closing because it's stale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@DrewScoggins