[Perf] Windows/x64: 9 Improvements on 7/12/2023 9:48:15 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare	d9f6b98de38081b71fa9b5a463f302288546435d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Devirtualization.EqualityComparer

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ValueTupleCompare - Duration of single invocation 📝 - Benchmark Source	3.45 ns	1.26 ns	0.36	0.23	False	43.69306792166419	17.763598416308596	0.4065541574731339)
ValueTupleCompareWrapped - Duration of single invocation 📝 - Benchmark Source	3.43 ns	0.93 ns	0.27	0.35	False	43.65031925696862	18.769477865624324	0.4299963479105102)
ValueTupleCompareCached - Duration of single invocation 📝 - Benchmark Source	4.42 ns	1.79 ns	0.41	0.30	False
ValueTupleCompareNoOpt - Duration of single invocation 📝 - Benchmark Source	7.09 ns	4.41 ns	0.62	0.44	False	68.14724427649156	45.93401803923917	0.6740407264727036)

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Devirtualization.EqualityComparer*'

Payloads

Baseline
Compare

Devirtualization.EqualityComparer.ValueTupleCompare

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2565320347846707 < 3.2822995026425765.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/16/2023 8:27:26 PM, 7/1/2023 6:36:20 PM, 7/5/2023 3:16:33 PM, 7/12/2023 4:12:15 PM, 7/17/2023 6:26:12 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 6:26:12 PM.
IsImprovementStdDev: Marked as improvement because 24.730515224375335 (T) = (0 -0.8969217241015922) / Math.Sqrt((0.20408782872104278 / (299)) + (0.03104216550990351 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.6784598184992017 = (2.7894545556178505 - 0.8969217241015922) / 2.7894545556178505 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Devirtualization.EqualityComparer.ValueTupleCompareWrapped

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9326349007457804 < 3.253045294606385.
IsChangePoint: Marked as a change because one of 7/1/2023 6:36:20 PM, 7/5/2023 3:16:33 PM, 7/12/2023 4:12:15 PM, 7/17/2023 6:26:12 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 6:26:12 PM.
IsImprovementStdDev: Marked as improvement because 32.27263922197236 (T) = (0 -1.1998147831583863) / Math.Sqrt((0.15428070581587716 / (299)) + (0.017135526238660086 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.6096674625835075 = (3.0738272322867113 - 1.1998147831583863) / 3.0738272322867113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Devirtualization.EqualityComparer.ValueTupleCompareCached

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.7941944875037978 < 4.198470651973903.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/5/2023 3:16:33 PM, 7/12/2023 7:15:00 AM, 7/17/2023 6:26:12 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 6:26:12 PM.
IsImprovementStdDev: Marked as improvement because 62.61592073355686 (T) = (0 -1.7453224458052092) / Math.Sqrt((0.24413241619131412 / (299)) + (0.0005732691231184778 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.520031165187214 = (3.6363245261246604 - 1.7453224458052092) / 3.6363245261246604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Devirtualization.EqualityComparer.ValueTupleCompareNoOpt

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.414920208090882 < 6.514658519753463.
IsChangePoint: Marked as a change because one of 7/5/2023 3:16:33 PM, 7/12/2023 4:12:15 PM, 7/17/2023 6:26:12 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 6:26:12 PM.
IsImprovementStdDev: Marked as improvement because 3.947826219658748 (T) = (0 -4.530460493633428) / Math.Sqrt((0.519251974213766 / (299)) + (0.20961780808920732 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.14300938818784925 = (5.286476224113513 - 4.530460493633428) / 5.286476224113513 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare	d9f6b98de38081b71fa9b5a463f302288546435d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in BenchmarksGame.BinaryTrees_2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
RunBench - Duration of single invocation 📝 - Benchmark Source	157.52 ms	135.51 ms	0.86	0.03	False	1189600000	1001775000	0.8421107935440484)

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.BinaryTrees_2*'

Payloads

Baseline
Compare

BenchmarksGame.BinaryTrees_2.RunBench

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 135.5099575 < 150.1101329375.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/12/2023 4:12:15 PM, 7/17/2023 6:26:12 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 6:26:12 PM.
IsImprovementStdDev: Marked as improvement because 20.250473266710692 (T) = (0 -131924217.25274725) / Math.Sqrt((71139015725084.39 / (299)) + (4293436764448.663 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.13035092335292267 = (151698220.34582 - 131924217.25274725) / 151698220.34582 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare	d9f6b98de38081b71fa9b5a463f302288546435d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableList - Duration of single invocation 📝 - Benchmark Source	84.23 μs	76.45 μs	0.91	0.05	False	640857.1428571428	595529.5138888889	0.9292703070045079)

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'

Payloads

Baseline
Compare

System.Collections.CreateAddAndClear<Int32>.ImmutableList(Size: 512)

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 76.44939746732027 < 80.31523164414412.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/12/2023 4:12:15 PM, 7/17/2023 6:26:12 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 6:26:12 PM.
IsImprovementStdDev: Marked as improvement because 44.39911058696315 (T) = (0 -75766.02287985367) / Math.Sqrt((6230069.326437654 / (299)) + (184561.61087794177 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11747308360739048 = (85851.23181234243 - 75766.02287985367) / 85851.23181234243 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare	d9f6b98de38081b71fa9b5a463f302288546435d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableList - Duration of single invocation 📝 - Benchmark Source	635.33 μs	462.08 μs	0.73	0.05	False	6193965.517241379	5055808.080808081	0.8162473728235733)

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<Int32>*'

Payloads

Baseline
Compare

System.Collections.ContainsTrue<Int32>.ImmutableList(Size: 512)

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 462.0780542986426 < 603.5381508333332.
IsChangePoint: Marked as a change because one of 5/22/2023 3:21:21 PM, 6/30/2023 8:34:57 AM, 7/12/2023 4:12:15 PM, 7/17/2023 6:26:12 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 6:26:12 PM.
IsImprovementStdDev: Marked as improvement because 32.06492684580829 (T) = (0 -474236.7897135544) / Math.Sqrt((3701553056.9318767 / (299)) + (35686512.63219341 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.22448092535375225 = (611508.8657617829 - 474236.7897135544) / 611508.8657617829 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare	d9f6b98de38081b71fa9b5a463f302288546435d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Fib

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Test - Duration of single invocation 📝 - Benchmark Source	159.72 μs	99.94 μs	0.63	0.04	False	1272411.0032362458	942972.6368159204	0.7410912310704378)

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Fib*'

Payloads

Baseline
Compare

Benchstone.BenchI.Fib.Test

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 99.93920259676628 < 151.74362288232604.
IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 4:12:15 PM, 7/17/2023 6:26:12 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 6:26:12 PM.
IsImprovementStdDev: Marked as improvement because 22.301859234286507 (T) = (0 -104440.55985362288) / Math.Sqrt((56991740.20592351 / (299)) + (26577033.76880598 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.31459184812542557 = (152377.1778435674 - 104440.55985362288) / 152377.1778435674 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare	d9f6b98de38081b71fa9b5a463f302288546435d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Ackermann

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Test - Duration of single invocation 📝 - Benchmark Source	3.53 μs	2.94 μs	0.83	0.06	False	23682.795698924732	20559.304314769055	0.8681113740174902)

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Ackermann*'

Payloads

Baseline
Compare

Benchstone.BenchI.Ackermann.Test

ETL Files

Baseline
Compare

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.9422995364279902 < 3.3500482845055726.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/9/2023 1:22:51 PM, 7/12/2023 4:12:15 PM, 7/17/2023 6:26:12 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 6:26:12 PM.
IsImprovementStdDev: Marked as improvement because 45.65593780180842 (T) = (0 -2938.4271900194885) / Math.Sqrt((21426.638209423287 / (299)) + (383.9760338270061 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.15323750041750525 = (3470.1905096981873 - 2938.4271900194885) / 3470.1905096981873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository