Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 21 Improvements on 8/24/2022 2:39:26 AM #8259

Closed
performanceautofiler bot opened this issue Sep 1, 2022 · 1 comment
Closed

[Perf] Linux/x64: 21 Improvements on 8/24/2022 2:39:26 AM #8259

performanceautofiler bot opened this issue Sep 1, 2022 · 1 comment
Assignees
@AndyAyersMS
Labels
arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Sep 1, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArraySet - Duration of single invocation 674.16 ns 571.04 ns 0.85 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArraySet(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 571.0404743246287 < 638.5472519845177.
IsChangePoint: Marked as a change because one of 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/10/2022 1:04:01 AM, 7/13/2022 7:22:00 AM, 7/28/2022 6:38:38 AM, 8/15/2022 9:13:40 PM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 19.96267650911531 (T) = (0 -570.9952228898386) / Math.Sqrt((2612.3925064787777 / (299)) + (0.17716301576729734 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.09367965977344422 = (630.0147944898877 - 570.9952228898386) / 630.0147944898877 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in Benchstone.BenchI.Fib

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 111.84 μs 102.31 μs 0.91 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.Fib*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Fib.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 102.31414710595776 < 106.20537465720663.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 31.08051598694069 (T) = (0 -102633.41915005932) / Math.Sqrt((12031001.80956207 / (299)) + (585855.9247998412 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.06500184993016145 = (109768.57990830595 - 102633.41915005932) / 109768.57990830595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetByteCount - Duration of single invocation 11.28 μs 8.95 μs 0.79 0.03 False
GetByteCount - Duration of single invocation 8.84 μs 7.09 μs 0.80 0.06 False
GetByteCount - Duration of single invocation 10.19 μs 8.17 μs 0.80 0.03 False
GetByteCount - Duration of single invocation 22.64 μs 18.52 μs 0.82 0.07 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetByteCount(Input: Greek)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.95036211309524 < 10.600367400381515.
IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 22.870625084350998 (T) = (0 -8994.113821170331) / Math.Sqrt((1148075.7239917805 / (299)) + (6444.181909887111 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1382409270547711 = (10436.923849761397 - 8994.113821170331) / 10436.923849761397 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Cyrillic)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.091751111111112 < 8.3992353016699.
IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 21.536268128031224 (T) = (0 -7120.486077055639) / Math.Sqrt((746911.5670425688 / (299)) + (3879.7414283301814 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.13318248748405537 = (8214.515713219005 - 7120.486077055639) / 8214.515713219005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: Chinese)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.17259414464504 < 9.708832363627453.
IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 23.59902113946944 (T) = (0 -8196.966402204716) / Math.Sqrt((929072.3695706881 / (299)) + (2170.7852075853048 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1391906537995743 = (9522.394753711449 - 8196.966402204716) / 9522.394753711449 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishMostlyAscii)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.51660334504062 < 21.5396255386727.
IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 22.85324305381667 (T) = (0 -18630.208011116873) / Math.Sqrt((3779690.287731805 / (299)) + (91488.94117451973 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.1288828865948544 = (21386.571018324372 - 18630.208011116873) / 21386.571018324372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseInt64 - Duration of single invocation 22.71 ns 16.38 ns 0.72 0.04 False
TryParseInt64 - Duration of single invocation 22.21 ns 16.05 ns 0.72 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.38095459484731 < 21.65804788338516.
IsChangePoint: Marked as a change because one of 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/13/2022 7:22:00 AM, 8/15/2022 9:13:40 PM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 13.515335398516552 (T) = (0 -16.65463942161734) / Math.Sqrt((7.7869803377131115 / (299)) + (0.020023389708470323 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1166471581574682 = (18.85389238900103 - 16.65463942161734) / 18.85389238900103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.053214632901728 < 21.22925331379762.
IsChangePoint: Marked as a change because one of 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/13/2022 7:22:00 AM, 8/17/2022 6:09:21 PM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 11.934825087014305 (T) = (0 -16.126509570257397) / Math.Sqrt((8.292554477184611 / (299)) + (0.027999548784549955 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.1107679939383931 = (18.135322908226648 - 16.126509570257397) / 18.135322908226648 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyOperatorBenchmark - Duration of single invocation 45.75 ns 43.34 ns 0.95 0.04 False
UnaryNegateOperatorBenchmark - Duration of single invocation 4.70 ns 0.00 ns 0.00 0.35 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.MultiplyOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.34291419350301 < 43.56142687950498.
IsChangePoint: Marked as a change because one of 8/4/2022 6:25:19 PM, 8/5/2022 10:47:59 PM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 14.167783036401584 (T) = (0 -43.4842524694979) / Math.Sqrt((0.5827925190356668 / (299)) + (1.3106758174828874 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.053279592704489065 = (45.93146206039758 - 43.4842524694979) / 45.93146206039758 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 0.23254348511180994.
IsChangePoint: Marked as a change because one of 8/20/2022 7:42:33 PM, 8/22/2022 12:23:00 AM, 8/23/2022 2:33:47 PM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 5.205281763920735 (T) = (0 -0.008183450540239957) / Math.Sqrt((0.3336343557244232 / (299)) + (0.0016664925825381558 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.9557042066139901 = (0.18474554612728908 - 0.008183450540239957) / 0.18474554612728908 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf8 - Duration of single invocation 66.19 ns 54.63 ns 0.83 0.04 False
EncodeUtf8 - Duration of single invocation 98.16 ns 86.78 ns 0.88 0.05 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,16)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.63376337660926 < 62.60433771178265.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 22.05223845929727 (T) = (0 -54.717800684919695) / Math.Sqrt((92.66937100261548 / (299)) + (0.08590079259662654 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.18368185381686614 = (67.02999429911513 - 54.717800684919695) / 67.02999429911513 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 86.77557243261097 < 93.17061833458844.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 18.190821253562667 (T) = (0 -86.64986846180011) / Math.Sqrt((127.63254800929757 / (299)) + (0.44901374141374084 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.12177116486508911 = (98.66434008453983 - 86.64986846180011) / 98.66434008453983 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Char_ToLower - Duration of single invocation 16.03 ns 14.46 ns 0.90 0.08 False
Char_ToUpper - Duration of single invocation 15.45 ns 14.03 ns 0.91 0.06 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Char.Char_ToLower(c: 'İ', cultureName: en-US)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.462120171305429 < 15.219446729109086.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 7.269423329180448 (T) = (0 -14.612963409008477) / Math.Sqrt((1.1752481008958684 / (299)) + (0.5066077303842007 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.05690072337241479 = (15.494618404610337 - 14.612963409008477) / 15.494618404610337 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.Char_ToUpper(c: 'İ', cultureName: en-US)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.032153913999577 < 15.037095727931908.
IsChangePoint: Marked as a change because one of 7/7/2022 4:14:07 PM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 25.25853919246057 (T) = (0 -14.003597990319959) / Math.Sqrt((0.44996715641369467 / (299)) + (0.04534599878195415 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.08195604690575635 = (15.253733705364748 - 14.003597990319959) / 15.253733705364748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 22.76 ns 21.05 ns 0.93 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.Parse(value: "4294967295")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.053260330918633 < 21.44450438348218.
IsChangePoint: Marked as a change because one of 6/27/2022 8:54:23 AM, 7/2/2022 7:35:06 PM, 7/13/2022 7:22:00 AM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 14.142264974733786 (T) = (0 -21.101774048549263) / Math.Sqrt((0.9695911008494321 / (299)) + (0.14447483271214742 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.050327646168449744 = (22.220057226486585 - 21.101774048549263) / 22.220057226486585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ctor_seeded - Duration of single invocation 294.37 ns 269.53 ns 0.92 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.ctor_seeded

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 269.5316158209155 < 278.95858451416416.
IsChangePoint: Marked as a change because one of 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/13/2022 7:22:00 AM, 8/9/2022 4:43:10 PM, 8/10/2022 9:29:58 PM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 19.025386391079177 (T) = (0 -270.2755778687907) / Math.Sqrt((208.8863896161188 / (299)) + (5.574054970311093 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.05982765061747102 = (287.4745019317978 - 270.2755778687907) / 287.4745019317978 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterUInt64 - Duration of single invocation 23.81 ns 21.66 ns 0.91 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.664174943426783 < 22.610779031889503.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/13/2022 7:22:00 AM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 14.35971182443487 (T) = (0 -21.759099820672542) / Math.Sqrt((7.0748822172842925 / (299)) + (0.06409871735900609 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.09448170875304086 = (24.02944261977172 - 21.759099820672542) / 24.02944261977172 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 2.65 μs 2.24 μs 0.84 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.ICollection(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2393050384568247 < 2.3613385493857004.
IsChangePoint: Marked as a change because one of 7/13/2022 9:19:11 PM, 8/15/2022 5:23:15 PM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 29.689794006472702 (T) = (0 -2233.6938774746914) / Math.Sqrt((8892.915041089496 / (299)) + (805.6171359152569 / (48))) is greater than 1.9668639088811137 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (48) - 2, .975) and 0.08312663326865818 = (2436.207614403526 - 2233.6938774746914) / 2436.207614403526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Sep 1, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SealedClassInterfaceMethod - Duration of single invocation 130.46 μs 108.80 μs 0.83 0.04 False
ClassVirtualMethod - Duration of single invocation 130.46 μs 108.76 μs 0.83 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.SealedClassInterfaceMethod

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.79713839699075 < 123.95906929991321.
IsChangePoint: Marked as a change because one of 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/13/2022 7:22:00 AM, 7/28/2022 3:42:13 AM, 8/15/2022 9:13:40 PM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 23.21136995990415 (T) = (0 -108953.58799398043) / Math.Sqrt((105772654.5002555 / (299)) + (371893.8127809018 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.11356877095635182 = (122912.62359013247 - 108953.58799398043) / 122912.62359013247 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.ClassVirtualMethod

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.7595101376488 < 123.95262712075034.
IsChangePoint: Marked as a change because one of 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/13/2022 7:22:00 AM, 8/4/2022 6:25:19 PM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 11.463394985451858 (T) = (0 -108823.28690750124) / Math.Sqrt((97944132.05933084 / (299)) + (177708.0156473064 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.05717028719981241 = (115421.9955418015 - 108823.28690750124) / 115421.9955418015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 7ad5248be06e55c3130e8d90491a35d740023dea
Compare 04690205c07a07a242155e88fff0acb730f3dc73
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf_Word_NotFound - Duration of single invocation 590.03 ns 539.99 ns 0.92 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 539.9920134118524 < 560.547958943174.
IsChangePoint: Marked as a change because one of 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/13/2022 7:22:00 AM, 8/23/2022 11:46:46 PM, 9/1/2022 1:45:20 PM falls between 8/23/2022 11:46:46 PM and 9/1/2022 1:45:20 PM.
IsImprovementStdDev: Marked as improvement because 33.12737056350064 (T) = (0 -540.7583481100186) / Math.Sqrt((424.25202957989296 / (299)) + (7.437837327754903 / (47))) is greater than 1.9668840363001725 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (47) - 2, .975) and 0.07143838742990183 = (582.3613003054185 - 540.7583481100186) / 582.3613003054185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@AndyAyersMS AndyAyersMS mentioned this issue Sep 2, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AndyAyersMS AndyAyersMS

Labels
arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@AndyAyersMS