Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/arm64: 556 Improvements on 3/6/2023 9:06:14 PM #14023

Closed
performanceautofiler bot opened this issue Mar 14, 2023 · 23 comments
Closed

[Perf] Linux/arm64: 556 Improvements on 3/6/2023 9:06:14 PM #14023

performanceautofiler bot opened this issue Mar 14, 2023 · 23 comments
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-improvement runtime-mono

Comments

@performanceautofiler
Copy link

No description provided.

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanAnyBenchmark - Duration of single invocation 2.96 ns 0.64 ns 0.22 0.45 False
AndNotBenchmark - Duration of single invocation 24.20 ns 15.50 ns 0.64 0.02 False
XorBenchmark - Duration of single invocation 16.12 ns 8.05 ns 0.50 0.27 False
DivideBenchmark - Duration of single invocation 50.57 ns 45.47 ns 0.90 0.00 False
EqualsStaticBenchmark - Duration of single invocation 27.21 ns 25.64 ns 0.94 0.00 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.83 ns 8.39 ns 0.61 0.04 False
BitwiseOrBenchmark - Duration of single invocation 16.10 ns 8.03 ns 0.50 0.27 False
OnesComplementBenchmark - Duration of single invocation 8.39 ns 6.33 ns 0.75 0.06 False
AddBenchmark - Duration of single invocation 26.11 ns 23.76 ns 0.91 0.01 False
SubtractBenchmark - Duration of single invocation 25.99 ns 23.91 ns 0.92 0.01 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.46 ns 8.34 ns 0.62 0.04 False
BitwiseAndBenchmark - Duration of single invocation 14.77 ns 8.37 ns 0.57 0.27 False
EqualsAllBenchmark - Duration of single invocation 5.35 ns 1.09 ns 0.20 0.26 False
ConditionalSelectBenchmark - Duration of single invocation 57.85 ns 32.49 ns 0.56 0.35 False
MultiplyBenchmark - Duration of single invocation 26.92 ns 24.67 ns 0.92 0.00 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.43 ns 6.44 ns 0.76 0.06 False
EqualityOperatorBenchmark - Duration of single invocation 2.99 ns 0.33 ns 0.11 0.46 False
OneBenchmark - Duration of single invocation 14.80 ns 0.82 ns 0.06 0.12 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.84 ns 8.36 ns 0.60 0.04 False
GreaterThanAllBenchmark - Duration of single invocation 2.93 ns 0.63 ns 0.21 0.49 False
InequalityOperatorBenchmark - Duration of single invocation 3.29 ns 1.14 ns 0.35 0.50 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.01 ns 1.22 ns 0.41 0.52 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.6390277163253985 < 2.8255137056303297.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 34.87513391293991 (T) = (0 -0.7822981192048216) / Math.Sqrt((0.42703751769306203 / (299)) + (0.08449953303921927 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7582322031266864 = (3.2357416054659502 - 0.7822981192048216) / 3.2357416054659502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.501027775640003 < 22.951103236309038.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 113.6237992894317 (T) = (0 -15.479159044759959) / Math.Sqrt((0.2758939047684509 / (299)) + (0.11157995842258556 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.35010331995058697 = (23.817876779402912 - 15.479159044759959) / 23.817876779402912 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.052164738102254 < 14.025368508435491.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 143.36691243787772 (T) = (0 -8.15213300640247) / Math.Sqrt((0.39838836498158503 / (299)) + (0.03064350089965945 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4732193132401421 = (15.475383231197998 - 8.15213300640247) / 15.475383231197998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.46977137009365 < 47.962305811169756.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.800616076332833 (T) = (0 -46.45857722338665) / Math.Sqrt((0.027443565097154152 / (299)) + (1.0586512164524329 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.08071249205620425 = (50.53759223521079 - 46.45857722338665) / 50.53759223521079 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.6376446334454 < 25.82691383592501.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.24210901496469 (T) = (0 -25.367081302184996) / Math.Sqrt((0.195685339663436 / (299)) + (0.1223728843613042 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06498085382286863 = (27.130012691076402 - 25.367081302184996) / 27.130012691076402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.391950361852286 < 13.135812337710169.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 82.93907618755158 (T) = (0 -8.1703455961116) / Math.Sqrt((0.1534284180054967 / (299)) + (0.10305593229981146 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.41308373924759456 = (13.920802919376458 - 8.1703455961116) / 13.920802919376458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.03425289092432 < 15.492590117805479.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 139.730335275272 (T) = (0 -8.177111824679463) / Math.Sqrt((0.4544383123575239 / (299)) + (0.030806239305178243 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.47500181793018037 = (15.575505028305006 - 8.177111824679463) / 15.575505028305006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.326640324545655 < 7.988030000791036.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 39.63680778639468 (T) = (0 -6.457919910365082) / Math.Sqrt((0.42588734547549256 / (299)) + (0.05430982247666351 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2715086052651036 = (8.864785441583932 - 6.457919910365082) / 8.864785441583932 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.76452520975187 < 24.73205275418947.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 69.2244043056749 (T) = (0 -23.684883092826198) / Math.Sqrt((0.037708844607217434 / (299)) + (0.02821848990136803 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.09382078202681415 = (26.137084831631 - 23.684883092826198) / 26.137084831631 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.91120081917488 < 24.743918109007964.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 62.873720040920624 (T) = (0 -23.6709320612877) / Math.Sqrt((0.07641298714688813 / (299)) + (0.027157111322389592 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09002033877372739 = (26.01259464347716 - 23.6709320612877) / 26.01259464347716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.338453251979267 < 12.836551222207873.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 127.03410074719653 (T) = (0 -8.15107985314234) / Math.Sqrt((0.1955493107047355 / (299)) + (0.034260446122656496 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.41556149103908957 = (13.946856218688213 - 8.15107985314234) / 13.946856218688213 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.372199810474923 < 14.674881152578749.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 128.9758864503565 (T) = (0 -8.213447963229036) / Math.Sqrt((0.4167594568343267 / (299)) + (0.0452123116392941 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4734112473834852 = (15.597461818958433 - 8.213447963229036) / 15.597461818958433 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0880679275715672 < 4.778702447126945.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 51.99863596292334 (T) = (0 -0.9720882573319604) / Math.Sqrt((0.08670981986782435 / (299)) + (0.12751653076125655 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.8001634765494451 = (4.864417377499473 - 0.9720882573319604) / 4.864417377499473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.48737748821542 < 54.99845618150482.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 100.30384667777673 (T) = (0 -32.47685416218258) / Math.Sqrt((16.943789985489687 / (299)) + (0.012232351978157086 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.42479550422753326 = (56.46140529303066 - 32.47685416218258) / 56.46140529303066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.66978615778552 < 25.624437811463036.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 69.09643646500328 (T) = (0 -24.57637422169276) / Math.Sqrt((0.09501320006400336 / (299)) + (0.019603476148080404 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0865087544616477 = (26.903787356176625 - 24.57637422169276) / 26.903787356176625 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.440687060636671 < 8.011529642611848.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 41.393734174034805 (T) = (0 -6.467913033545556) / Math.Sqrt((0.41783753199837437 / (299)) + (0.046851597080141846 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2702841943666615 = (8.863605507258944 - 6.467913033545556) / 8.863605507258944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.3296145318345992 < 2.783965857477417.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 40.82494352014309 (T) = (0 -0.7735550107813224) / Math.Sqrt((0.5717426821140812 / (299)) + (0.04491165558321555 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7644978388592459 = (3.2847045098621703 - 0.7735550107813224) / 3.2847045098621703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8206701152100013 < 14.079315108863815.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 118.30924859147507 (T) = (0 -0.917523425141539) / Math.Sqrt((4.501634209800545 / (299)) + (0.05999020430281979 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.9447043239339041 = (16.593041091401204 - 0.917523425141539) / 16.593041091401204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.364971779075889 < 13.002268925806082.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 124.96720683561769 (T) = (0 -8.220873093048857) / Math.Sqrt((0.11631823005264554 / (299)) + (0.0394547298234269 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.40666883889885047 = (13.855454815135495 - 8.220873093048857) / 13.855454815135495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.6269701447644817 < 2.7294025181674018.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 32.31322869785022 (T) = (0 -0.7505852125353177) / Math.Sqrt((0.5681990310022845 / (299)) + (0.09414667786404762 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.7692094480315889 = (3.252235440894704 - 0.7505852125353177) / 3.252235440894704 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1438483727886197 < 2.968909214829777.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 33.368556963823735 (T) = (0 -0.7597381573150269) / Math.Sqrt((0.440902555626936 / (299)) + (0.09632802733345486 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.76491603490111 = (3.231773621801201 - 0.7597381573150269) / 3.231773621801201 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2226036731309013 < 3.0434529819430716.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.304049596143237 (T) = (0 -1.126279163826123) / Math.Sqrt((0.6354176720418676 / (299)) + (0.1278879515795232 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.6598284574651632 = (3.310915297127716 - 1.126279163826123) / 3.310915297127716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.83 ns 8.01 ns 0.58 0.04 False
AndNotBenchmark - Duration of single invocation 24.03 ns 15.34 ns 0.64 0.01 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.50 ns 8.36 ns 0.62 0.04 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.61 ns 1.71 ns 0.47 0.43 False
OnesComplementBenchmark - Duration of single invocation 8.45 ns 6.36 ns 0.75 0.06 False
ConditionalSelectBenchmark - Duration of single invocation 54.19 ns 32.55 ns 0.60 0.34 False
EqualsAllBenchmark - Duration of single invocation 6.04 ns 1.54 ns 0.26 0.21 False
LessThanBenchmark - Duration of single invocation 47.72 ns 43.85 ns 0.92 0.00 False
EqualityOperatorBenchmark - Duration of single invocation 2.31 ns 0.00 ns 0.00 0.42 False
XorBenchmark - Duration of single invocation 14.75 ns 8.37 ns 0.57 0.31 False
BitwiseAndBenchmark - Duration of single invocation 14.77 ns 8.05 ns 0.55 0.28 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 2.82 ns 1.58 ns 0.56 0.47 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.85 ns 8.33 ns 0.60 0.05 False
AddBenchmark - Duration of single invocation 42.70 ns 38.99 ns 0.91 0.00 False
SubtractBenchmark - Duration of single invocation 42.37 ns 38.81 ns 0.92 0.00 False
EqualsBenchmark - Duration of single invocation 8.85 ns 6.53 ns 0.74 0.10 False
OneBenchmark - Duration of single invocation 22.94 ns 0.79 ns 0.03 0.12 False
LessThanAnyBenchmark - Duration of single invocation 2.25 ns 0.77 ns 0.34 0.52 False
MultiplyBenchmark - Duration of single invocation 43.92 ns 40.58 ns 0.92 0.00 False
GreaterThanAllBenchmark - Duration of single invocation 2.26 ns 0.80 ns 0.36 0.41 False
DivideBenchmark - Duration of single invocation 91.43 ns 84.44 ns 0.92 0.00 False
BitwiseOrBenchmark - Duration of single invocation 14.75 ns 8.48 ns 0.58 0.29 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.44 ns 6.35 ns 0.75 0.05 False
InequalityOperatorBenchmark - Duration of single invocation 2.85 ns 0.90 ns 0.32 0.45 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<SByte>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.009037430481328 < 13.115884485987298.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 114.0993597346588 (T) = (0 -8.211220225074088) / Math.Sqrt((0.146600017711946 / (299)) + (0.04773095540546808 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.40893649772064916 = (13.89228093666536 - 8.211220225074088) / 13.89228093666536 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.337849400122577 < 22.83077100429329.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 96.15516640479558 (T) = (0 -15.567043988419975) / Math.Sqrt((0.15089550181442382 / (299)) + (0.1711691135848178 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3505581115035838 = (23.969879775480294 - 15.567043988419975) / 23.969879775480294 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.361390150156387 < 12.821391806092429.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 133.87050373130734 (T) = (0 -8.147761497892951) / Math.Sqrt((0.08495882681346016 / (299)) + (0.03535890991081849 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4078594626564788 = (13.759844131674695 - 8.147761497892951) / 13.759844131674695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7123519272469925 < 3.1842197954911873.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 33.10612709428613 (T) = (0 -1.1656439037027713) / Math.Sqrt((0.36715068519875516 / (299)) + (0.11528108628094517 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.6880565240281941 = (3.736715121454006 - 1.1656439037027713) / 3.736715121454006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.357960588759699 < 8.009227963864227.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.212415275965306 (T) = (0 -6.429905254313776) / Math.Sqrt((0.475450170620735 / (299)) + (0.11038871649265988 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.276361716784203 = (8.885523891494152 - 6.429905254313776) / 8.885523891494152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.55308546163538 < 58.31700190934131.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 107.11002485425374 (T) = (0 -32.43586151590618) / Math.Sqrt((14.287691577649749 / (299)) + (0.06623746964493336 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.42581448577963654 = (56.49021215721197 - 32.43586151590618) / 56.49021215721197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5406139914232593 < 5.5675385863134235.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 63.666813415035946 (T) = (0 -1.0068068555225347) / Math.Sqrt((0.19398067435917604 / (299)) + (0.09251485034618151 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.8092948853557793 = (5.279390945548747 - 1.0068068555225347) / 5.279390945548747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.85122762786971 < 44.91236867645366.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.762615342683112 (T) = (0 -44.36344235921997) / Math.Sqrt((0.248035778709569 / (299)) + (0.07862941192007615 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06010474588538942 = (47.20041107241329 - 44.36344235921997) / 47.20041107241329 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 2.219823233076387.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 20.764673918886135 (T) = (0 -0.7345950437679029) / Math.Sqrt((0.9354481090310744 / (299)) + (0.17994420207554107 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7444972083308471 = (2.8750959587131235 - 0.7345950437679029) / 2.8750959587131235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.368576579649103 < 14.03063346030062.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 129.5506380425796 (T) = (0 -8.284578783956503) / Math.Sqrt((0.412444096289158 / (299)) + (0.04338285656585798 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4688760983777741 = (15.598203655781058 - 8.284578783956503) / 15.598203655781058 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.04965802635379 < 14.048380785322601.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 111.11901401207328 (T) = (0 -8.11910124614216) / Math.Sqrt((0.39122508788527804 / (299)) + (0.07697424884215916 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.47908357275575364 = (15.586187767381139 - 8.11910124614216) / 15.586187767381139 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5807702495132667 < 3.290075755213106.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.20763056994314 (T) = (0 -1.1845312255228904) / Math.Sqrt((0.3210560267170296 / (299)) + (0.10518072605509643 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.6870628062576176 = (3.7852043451825215 - 1.1845312255228904) / 3.7852043451825215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.32814209902413 < 13.14080415696833.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 112.66717561954178 (T) = (0 -8.200454488680252) / Math.Sqrt((0.15971272250483465 / (299)) + (0.0489513062665229 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4107323880925404 = (13.916350267640498 - 8.200454488680252) / 13.916350267640498 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.99477186422514 < 40.4013262287074.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.885416259717204 (T) = (0 -39.329089551721516) / Math.Sqrt((0.5003209021834806 / (299)) + (0.18428655894663076 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05106549704151823 = (41.44552593367159 - 39.329089551721516) / 41.44552593367159 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.81246311277145 < 40.29692065022124.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.684867702057307 (T) = (0 -39.025683148839555) / Math.Sqrt((0.5173095496212246 / (299)) + (0.2135328985242293 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.058881258608579394 = (41.46733183863819 - 39.025683148839555) / 41.46733183863819 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.531677206084435 < 8.330411700413338.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 18.10331935652548 (T) = (0 -6.424863735046507) / Math.Sqrt((1.1164912595878782 / (299)) + (0.09949470923906359 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.201893034264764 = (8.050128630474692 - 6.424863735046507) / 8.050128630474692 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7918402250709345 < 21.8261194275901.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 142.14246130938818 (T) = (0 -0.947372430230816) / Math.Sqrt((7.528285328811811 / (299)) + (0.04701108326323285 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.9611142091636138 = (24.36294620358711 - 0.947372430230816) / 24.36294620358711 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7691619768133654 < 2.1796604672901707.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.977114328968398 (T) = (0 -0.7724202390023783) / Math.Sqrt((0.9135221911827041 / (299)) + (0.1296990605806318 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7323318628096516 = (2.8857384637196564 - 0.7724202390023783) / 2.8857384637196564 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.58321641166918 < 41.73557471423837.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.946729182222334 (T) = (0 -40.723111866065594) / Math.Sqrt((0.5511093200436843 / (299)) + (0.09814101424812223 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06365999848726114 = (43.491799773878995 - 40.723111866065594) / 43.491799773878995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8034004119969389 < 2.1477351046307853.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.01520954098611 (T) = (0 -0.622405505636148) / Math.Sqrt((0.9284868301253533 / (299)) + (0.18321783272625417 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7856629689421079 = (2.903863614066938 - 0.622405505636148) / 2.903863614066938 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 84.44162724269573 < 87.01497341605602.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/10/2023 12:53:36 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.39343310789509 (T) = (0 -84.0997411245466) / Math.Sqrt((0.9893054834020831 / (299)) + (0.23914491708739463 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07388644353078934 = (90.80931872455811 - 84.0997411245466) / 90.80931872455811 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.483256794847978 < 15.32613378928716.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 138.70978440541316 (T) = (0 -8.196982640575428) / Math.Sqrt((0.4168533696461243 / (299)) + (0.03521305951901671 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.47511813144558585 = (15.61681424269974 - 8.196982640575428) / 15.61681424269974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.35367059362502 < 8.00348432326465.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.570799836018406 (T) = (0 -6.530087588851676) / Math.Sqrt((0.4704594763702457 / (299)) + (0.2541658612004558 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.26703270822875386 = (8.909111855552853 - 6.530087588851676) / 8.909111855552853 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8971356945436478 < 2.4299628586619.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.267016980663456 (T) = (0 -0.7608659854520176) / Math.Sqrt((0.966602267127429 / (299)) + (0.09060505271537714 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7354487805735462 = (2.876063044054655 - 0.7608659854520176) / 2.876063044054655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OneBenchmark - Duration of single invocation 10.58 ns 0.74 ns 0.07 0.25 False
BitwiseOrBenchmark - Duration of single invocation 14.83 ns 8.44 ns 0.57 0.28 False
OnesComplementBenchmark - Duration of single invocation 8.41 ns 6.88 ns 0.82 0.07 False
EqualityOperatorBenchmark - Duration of single invocation 2.42 ns 0.73 ns 0.30 0.44 False
XorBenchmark - Duration of single invocation 14.73 ns 8.05 ns 0.55 0.31 False
DotBenchmark - Duration of single invocation 7.74 ns 6.42 ns 0.83 0.24 False
AddBenchmark - Duration of single invocation 13.42 ns 11.55 ns 0.86 0.01 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.56 ns 8.47 ns 0.62 0.03 False
SubtractBenchmark - Duration of single invocation 13.55 ns 11.73 ns 0.87 0.02 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 3.29 ns 0.90 ns 0.27 0.45 False
GreaterThanAllBenchmark - Duration of single invocation 2.37 ns 0.74 ns 0.31 0.40 False
EqualsBenchmark - Duration of single invocation 8.65 ns 5.34 ns 0.62 0.10 False
GreaterThanAnyBenchmark - Duration of single invocation 7.79 ns 6.69 ns 0.86 0.08 False
LessThanAnyBenchmark - Duration of single invocation 2.00 ns 0.62 ns 0.31 0.42 False
BitwiseAndBenchmark - Duration of single invocation 14.77 ns 8.04 ns 0.54 0.28 False
EqualsAllBenchmark - Duration of single invocation 5.36 ns 0.46 ns 0.09 0.21 False
MultiplyBenchmark - Duration of single invocation 13.65 ns 12.32 ns 0.90 0.02 False
AndNotBenchmark - Duration of single invocation 23.89 ns 15.37 ns 0.64 0.04 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.82 ns 8.43 ns 0.61 0.03 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.45 ns 6.36 ns 0.75 0.04 False
ConditionalSelectBenchmark - Duration of single invocation 57.28 ns 31.73 ns 0.55 0.23 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.14 ns 0.73 ns 0.23 0.41 False
DivideBenchmark - Duration of single invocation 25.58 ns 22.34 ns 0.87 0.01 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.83 ns 8.49 ns 0.61 0.06 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int32>.OneBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.739048723653666 < 10.039335828499445.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 124.07572953216388 (T) = (0 -0.7774539366743428) / Math.Sqrt((1.3720532353279917 / (299)) + (0.0674596138171715 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.9321035359513655 = (11.4505806387421 - 0.7774539366743428) / 11.4505806387421 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.440818844319681 < 15.29482886302618.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 113.14574547676372 (T) = (0 -8.255536999890456) / Math.Sqrt((0.3763163220397801 / (299)) + (0.06604175970957044 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4646468167339311 = (15.420730198194187 - 8.255536999890456) / 15.420730198194187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.883088002217615 < 8.001162918559029.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 6:03:21 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.471642955550323 (T) = (0 -6.343148406610477) / Math.Sqrt((0.4094529175861283 / (299)) + (0.15144933048335138 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.2859292423115986 = (8.883081036877346 - 6.343148406610477) / 8.883081036877346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7327397732049765 < 2.1215574494228258.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.69431034651855 (T) = (0 -0.6823488732574409) / Math.Sqrt((0.8121461359392154 / (299)) + (0.08388020952760089 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7323836191644426 = (2.5497276030973777 - 0.6823488732574409) / 2.5497276030973777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.046725289175619 < 15.51349934840823.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 108.8037986951173 (T) = (0 -8.112999052441234) / Math.Sqrt((0.47676444139694873 / (299)) + (0.07819394548237954 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.48299642534969917 = (15.692346146598375 - 8.112999052441234) / 15.692346146598375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.4227444180376905 < 6.934554607734677.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.79462642505992 (T) = (0 -6.014766629115791) / Math.Sqrt((0.7495673489997075 / (299)) + (0.07526148234166183 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.22159779794919185 = (7.727067849074755 - 6.014766629115791) / 7.727067849074755 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.549908141640243 < 12.750126067949203.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 65.3148186599582 (T) = (0 -11.745803655141081) / Math.Sqrt((0.11215303912334383 / (299)) + (0.0125817557858959 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14292500703346753 = (13.70452265149654 - 11.745803655141081) / 13.70452265149654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.466751429254085 < 12.867871288639813.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 128.77910505009297 (T) = (0 -8.242418960826745) / Math.Sqrt((0.09421016891609484 / (299)) + (0.03614833107332468 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.404284384104313 = (13.836164003244859 - 8.242418960826745) / 13.836164003244859 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.728454836013949 < 12.805391029733808.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 48.317101566691015 (T) = (0 -11.732410220195305) / Math.Sqrt((0.3495195060056005 / (299)) + (0.01854928463537919 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.15360171719014318 = (13.861571388408628 - 11.732410220195305) / 13.861571388408628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8999680288603552 < 3.083389245330293.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 34.46751420307546 (T) = (0 -0.8884833530551859) / Math.Sqrt((0.4127010693671611 / (299)) + (0.09504736709956646 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7392445793560921 = (3.4073437509416697 - 0.8884833530551859) / 3.4073437509416697 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7354042477863773 < 2.1465708736318536.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.03785637573207 (T) = (0 -0.6635411104753915) / Math.Sqrt((0.8036775725354784 / (299)) + (0.11432979093388992 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7414013159714684 = (2.5659106231266895 - 0.6635411104753915) / 2.5659106231266895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.339404632859174 < 8.367531758770467.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 30.756923584310584 (T) = (0 -5.564881284047117) / Math.Sqrt((1.0771834587031694 / (299)) + (0.08701592439599709 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3196811812809832 = (8.17981383276347 - 5.564881284047117) / 8.17981383276347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.6924836100848335 < 7.535095696387851.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 20.778451142918225 (T) = (0 -6.84275267514337) / Math.Sqrt((0.02806680348435593 / (299)) + (0.06221137953486231 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.13597590886508407 = (7.919631808130782 - 6.84275267514337) / 7.919631808130782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.6197093054235837 < 1.970299035065044.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.204813934764985 (T) = (0 -0.7278491780323672) / Math.Sqrt((0.734471418980024 / (299)) + (0.16749930526342524 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7193395107938326 = (2.593343936978975 - 0.7278491780323672) / 2.593343936978975 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.036439112393422 < 15.464175655482435.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 123.85059801865074 (T) = (0 -8.167412195398834) / Math.Sqrt((0.4283881935213691 / (299)) + (0.05208613078069914 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4764994581110825 = (15.601535322062555 - 8.167412195398834) / 15.601535322062555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.45644826413192435 < 5.107256085531006.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 63.70328144831659 (T) = (0 -0.6898798921771729) / Math.Sqrt((0.1948813248705364 / (299)) + (0.1046099457529962 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.8652571086674234 = (5.119972455351206 - 0.6898798921771729) / 5.119972455351206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.323671012278524 < 12.967693613640224.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 55.43740171950847 (T) = (0 -12.326394540988282) / Math.Sqrt((0.11050088796831121 / (299)) + (0.010453664917075564 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11317252039976719 = (13.899427819427537 - 12.326394540988282) / 13.899427819427537 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.365369380704944 < 22.818337093921784.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 78.7832394244698 (T) = (0 -15.54381579189843) / Math.Sqrt((0.26423872802041565 / (299)) + (0.24425656001154913 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.34770940339466316 = (23.829587415167186 - 15.54381579189843) / 23.829587415167186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.430476934317927 < 12.973433897681735.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 123.88510480229448 (T) = (0 -8.22479465318364) / Math.Sqrt((0.10744595092063365 / (299)) + (0.04057500630898261 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.40546083722529863 = (13.833898871856821 - 8.22479465318364) / 13.833898871856821 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.363958437342855 < 7.993514707854868.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.75839095292251 (T) = (0 -6.442529162688637) / Math.Sqrt((0.40157022465627823 / (299)) + (0.0446253971530628 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2730351623027414 = (8.8622294072655 - 6.442529162688637) / 8.8622294072655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.733585288569763 < 54.83075678894976.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 90.81750441206934 (T) = (0 -32.47898689386449) / Math.Sqrt((20.0653130659876 / (299)) + (0.03495721519874961 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.42269238055265945 = (56.25941144680679 - 32.47898689386449) / 56.25941144680679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7251945756618923 < 2.9425154671783393.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 6:03:21 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 37.57496186711744 (T) = (0 -0.6998914604759625) / Math.Sqrt((0.4538757766146451 / (299)) + (0.08624146448598376 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.7909667345683075 = (3.3482300486028214 - 0.6998914604759625) / 3.3482300486028214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.33747805320977 < 24.302252114748132.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.42164798431963 (T) = (0 -22.856370038567615) / Math.Sqrt((0.025082086459690494 / (299)) + (0.20684858479481238 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10720949533555849 = (25.601045171463 - 22.856370038567615) / 25.601045171463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.494064431429182 < 13.129100427068622.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 82.17237086020326 (T) = (0 -8.096453360293316) / Math.Sqrt((0.2722313557701292 / (299)) + (0.09642711000890385 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.4202920408314293 = (13.966434706029254 - 8.096453360293316) / 13.966434706029254 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseAndOperatorBenchmark - Duration of single invocation 14.34 ns 12.90 ns 0.90 0.02 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.37 ns 12.86 ns 0.89 0.03 False
BitwiseOrBenchmark - Duration of single invocation 15.47 ns 12.95 ns 0.84 0.02 False
GreaterThanAllBenchmark - Duration of single invocation 22.85 ns 21.31 ns 0.93 0.02 False
EqualsStaticBenchmark - Duration of single invocation 38.72 ns 36.59 ns 0.94 0.00 False
DivideBenchmark - Duration of single invocation 63.03 ns 56.18 ns 0.89 0.00 False
GreaterThanBenchmark - Duration of single invocation 40.91 ns 37.54 ns 0.92 0.00 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 25.08 ns 23.40 ns 0.93 0.02 False
EqualsAllBenchmark - Duration of single invocation 8.12 ns 5.62 ns 0.69 0.10 False
LessThanAnyBenchmark - Duration of single invocation 23.20 ns 21.22 ns 0.91 0.01 False
AndNotBenchmark - Duration of single invocation 14.58 ns 13.16 ns 0.90 0.05 False
DivisionOperatorBenchmark - Duration of single invocation 62.38 ns 56.32 ns 0.90 0.00 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.18 ns 13.06 ns 0.92 0.04 False
XorBenchmark - Duration of single invocation 15.53 ns 12.96 ns 0.84 0.01 False
GreaterThanOrEqualBenchmark - Duration of single invocation 43.71 ns 40.73 ns 0.93 0.01 False
SubtractBenchmark - Duration of single invocation 38.23 ns 34.53 ns 0.90 0.00 False
ConditionalSelectBenchmark - Duration of single invocation 16.62 ns 15.31 ns 0.92 0.03 False
MultiplyBenchmark - Duration of single invocation 38.50 ns 35.30 ns 0.92 0.01 False
DotBenchmark - Duration of single invocation 25.47 ns 22.79 ns 0.89 0.02 False
LessThanBenchmark - Duration of single invocation 38.75 ns 36.20 ns 0.93 0.00 False
BitwiseAndBenchmark - Duration of single invocation 15.46 ns 12.76 ns 0.83 0.03 False
AddBenchmark - Duration of single invocation 38.34 ns 34.56 ns 0.90 0.00 False
EqualsBenchmark - Duration of single invocation 10.66 ns 8.37 ns 0.79 0.14 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.BitwiseAndOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.901440367551933 < 13.502928884491158.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 27.73622824760495 (T) = (0 -12.913574043837412) / Math.Sqrt((0.12268559337525753 / (299)) + (0.0687942024554175 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10948699182716269 = (14.50127502385799 - 12.913574043837412) / 14.50127502385799 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.858517284908135 < 13.55153165739449.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.831061307959295 (T) = (0 -12.939521408580775) / Math.Sqrt((0.2145167085555043 / (299)) + (0.05825708110214152 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11448409092491603 = (14.612409868610975 - 12.939521408580775) / 14.612409868610975 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.952901829169477 < 14.717519448214299.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 56.78662070134997 (T) = (0 -13.014872522993931) / Math.Sqrt((0.23055151584747852 / (299)) + (0.04106235999796504 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.17855975501453167 = (15.843967473524712 - 13.014872522993931) / 15.843967473524712 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.311809032916546 < 21.966465493302604.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 27.840916194126585 (T) = (0 -21.229187117250998) / Math.Sqrt((0.4037590919153576 / (299)) + (0.032949519315979066 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06405391273008763 = (22.68206193283527 - 21.229187117250998) / 22.68206193283527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.59100912179816 < 36.80795297311161.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.047891474823345 (T) = (0 -36.42171653569617) / Math.Sqrt((0.2089110420657286 / (299)) + (0.06886533379385718 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.055245984559983204 = (38.55153398711182 - 36.42171653569617) / 38.55153398711182 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 56.18265498544145 < 59.90553922165144.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 34.43141876577611 (T) = (0 -57.00707468305768) / Math.Sqrt((0.28364858455983905 / (299)) + (0.6378342205520898 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09107868888574173 = (62.71948295851043 - 57.00707468305768) / 62.71948295851043 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.54386508568108 < 39.17463651202139.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.353933315757445 (T) = (0 -38.04999511798487) / Math.Sqrt((0.4231796326854973 / (299)) + (0.22763591771105118 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06504271676300172 = (40.697041244759994 - 38.04999511798487) / 40.697041244759994 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.398217829178474 < 23.680570588405626.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 16.48153958368026 (T) = (0 -23.35990082661257) / Math.Sqrt((0.34559563944059213 / (299)) + (0.16325856446885745 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.059214358507010144 = (24.83020551795553 - 23.35990082661257) / 24.83020551795553 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.617219285798096 < 7.697548937881311.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 55.65521272977831 (T) = (0 -5.73578832758364) / Math.Sqrt((0.14501211134107958 / (299)) + (0.025987347703549765 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.28053723499114913 = (7.972321302149775 - 5.73578832758364) / 7.972321302149775 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.218711299907437 < 21.903350481171504.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.66349306578138 (T) = (0 -21.158655356215842) / Math.Sqrt((0.4150667800386983 / (299)) + (0.04357548525869557 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.061894602865605955 = (22.554667546790192 - 21.158655356215842) / 22.554667546790192 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.160176261677218 < 13.815241918739453.
IsChangePoint: Marked as a change because one of 11/21/2022 12:46:21 PM, 12/5/2022 9:44:53 PM, 12/15/2022 8:20:21 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 16.94188963314564 (T) = (0 -13.111727841931538) / Math.Sqrt((0.2003372067028891 / (299)) + (0.10931506482715576 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.08404368612803521 = (14.314796069809434 - 13.111727841931538) / 14.314796069809434 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 56.31709040785844 < 59.08650586418017.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.611085251533837 (T) = (0 -57.06873175712494) / Math.Sqrt((1.6983155304420938 / (299)) + (0.7265458624861031 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06987628930109407 = (61.3560659734637 - 57.06873175712494) / 61.3560659734637 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.056003027488307 < 13.609067139045294.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.547210282899712 (T) = (0 -12.860428306078928) / Math.Sqrt((0.06773050780421314 / (299)) + (0.12331971722492395 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10930571560357646 = (14.438655924230794 - 12.860428306078928) / 14.438655924230794 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.96479394032856 < 14.74039482431066.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 50.95817098240326 (T) = (0 -12.907751963926906) / Math.Sqrt((0.5563903939712522 / (299)) + (0.04536828185512884 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.19471410410813633 = (16.02878186464624 - 12.907751963926906) / 16.02878186464624 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.73188469650742 < 41.55166124279086.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.542603386617884 (T) = (0 -41.19999882299026) / Math.Sqrt((0.30112579743835116 / (299)) + (0.18688816633165373 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05290688765161556 = (43.50152934892742 - 41.19999882299026) / 43.50152934892742 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.52503626348985 < 36.27689111688575.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 44.00766811029115 (T) = (0 -34.414862227573806) / Math.Sqrt((0.156030817493846 / (299)) + (0.15384988288485407 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09622338973163404 = (38.078947647643496 - 34.414862227573806) / 38.078947647643496 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.30738810987042 < 15.798724579421226.
IsChangePoint: Marked as a change because one of 11/21/2022 12:46:21 PM, 12/5/2022 5:12:51 PM, 12/14/2022 12:01:27 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.529510441333482 (T) = (0 -15.378752068693302) / Math.Sqrt((0.10023432223581315 / (299)) + (0.035443190461053924 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08026763497002186 = (16.72089909350101 - 15.378752068693302) / 16.72089909350101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.30227599754317 < 36.637173422234824.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 52.26744521916483 (T) = (0 -35.117824411783964) / Math.Sqrt((0.29004303497924855 / (299)) + (0.07938631720293629 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08870967962965347 = (38.53637378427563 - 35.117824411783964) / 38.53637378427563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.79410499045335 < 24.248433498009113.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.206185236001126 (T) = (0 -23.232523135743563) / Math.Sqrt((0.3056685913746381 / (299)) + (0.16526488319549404 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08799363796622256 = (25.47408011928223 - 23.232523135743563) / 25.47408011928223 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.197363406783616 < 36.79207376459269.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 57.47636576088027 (T) = (0 -36.535522938367954) / Math.Sqrt((0.07911811450612566 / (299)) + (0.02674756831365456 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05519654826191419 = (38.66997190913753 - 36.535522938367954) / 38.66997190913753 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.756919176246724 < 14.70671020645647.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 49.05300410757837 (T) = (0 -12.971817927894426) / Math.Sqrt((0.5815024707877164 / (299)) + (0.04578473290899442 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.19009524399806924 = (16.01647333438242 - 12.971817927894426) / 16.01647333438242 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.556520921820855 < 36.31445380437903.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.32692663956062 (T) = (0 -34.46084135198203) / Math.Sqrt((0.1491039380997001 / (299)) + (0.1537797891638007 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0945999775804079 = (38.06145405197676 - 34.46084135198203) / 38.06145405197676 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.373847440368577 < 10.0684112673788.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.404979798337077 (T) = (0 -8.529443407397219) / Math.Sqrt((1.0289937806146576 / (299)) + (0.0694023189315278 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.16646727458329622 = (10.232883661686033 - 8.529443407397219) / 10.232883661686033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Narrow_double - Duration of single invocation 26.67 μs 23.05 μs 0.86 0.00 False
Convert_double_long - Duration of single invocation 21.55 μs 18.36 μs 0.85 0.01 False
Widen_float - Duration of single invocation 69.98 μs 64.10 μs 0.92 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorConvert* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorConvert* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorConvert.Narrow_double

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.04802234567901 < 25.341142076128484.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 134.9346768093088 (T) = (0 -22961.30860041504) / Math.Sqrt((27966.61777928998 / (299)) + (17274.346351865584 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.14133488227140142 = (26740.70266316851 - 22961.30860041504) / 26740.70266316851 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_double_long

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.3577215611567 < 20.43428992488117.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 26.582643229534415 (T) = (0 -18321.111120299585) / Math.Sqrt((1957899.9704216435 / (299)) + (5458.031329132836 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10668956592185981 = (20509.23220124057 - 18321.111120299585) / 20509.23220124057 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_float

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 64.09912134074942 < 66.51618248906013.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 127.63527823575727 (T) = (0 -64070.61881880677) / Math.Sqrt((725938.5753579339 / (299)) + (2553.5686381779506 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0911452769850118 = (70495.99589058873 - 64070.61881880677) / 70495.99589058873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsStaticBenchmark - Duration of single invocation 40.64 ns 30.81 ns 0.76 0.09 False
LessThanOrEqualBenchmark - Duration of single invocation 44.13 ns 32.23 ns 0.73 0.08 False
GreaterThanOrEqualBenchmark - Duration of single invocation 43.02 ns 32.09 ns 0.75 0.06 False
GreaterThanBenchmark - Duration of single invocation 41.82 ns 30.89 ns 0.74 0.07 False
LessThanBenchmark - Duration of single invocation 40.64 ns 30.82 ns 0.76 0.11 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Float* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Float* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualsStaticBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.80793014885534 < 39.07145535711229.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 6.542996000887727 (T) = (0 -30.94454417205493) / Math.Sqrt((97.54384763981223 / (299)) + (0.1043792545764362 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10839408717368165 = (34.70652642260215 - 30.94454417205493) / 34.70652642260215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.22775985882357 < 41.89359841249314.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 7.332862073938487 (T) = (0 -32.22760532052363) / Math.Sqrt((104.82529336934329 / (299)) + (0.03206517955680475 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11893572264081155 = (36.578041067695246 - 32.22760532052363) / 36.578041067695246 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.08999404373263 < 41.153021063036874.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 7.0779134854206065 (T) = (0 -32.255317893105484) / Math.Sqrt((107.21340999915905 / (299)) + (0.04811873348203836 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.11641375927482758 = (36.505002462049504 - 32.255317893105484) / 36.505002462049504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.886709639563943 < 39.10220855908741.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 6.964962805202715 (T) = (0 -30.989655365327234) / Math.Sqrt((92.44501416524919 / (299)) + (0.09948009394239206 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1117471924820605 = (34.888328078491725 - 30.989655365327234) / 34.888328078491725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.817193271370662 < 39.11197194192444.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 6.726225277173144 (T) = (0 -31.075205694911332) / Math.Sqrt((95.60042427649167 / (299)) + (0.05490791301970201 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10939222178650429 = (34.892133726078924 - 31.075205694911332) / 34.892133726078924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanAnyBenchmark - Duration of single invocation 3.38 ns 1.01 ns 0.30 0.41 False
UnaryNegateOperatorBenchmark - Duration of single invocation 9.29 ns 7.00 ns 0.75 0.01 False
DivideBenchmark - Duration of single invocation 12.86 ns 8.08 ns 0.63 0.03 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 5.22 ns 2.20 ns 0.42 0.39 False
EqualsAllBenchmark - Duration of single invocation 4.11 ns 1.16 ns 0.28 0.29 False
AddOperatorBenchmark - Duration of single invocation 9.41 ns 7.15 ns 0.76 0.07 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 3.27 ns 2.08 ns 0.64 0.43 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.28 ns 1.65 ns 0.50 0.44 False
BitwiseOrBenchmark - Duration of single invocation 16.08 ns 8.30 ns 0.52 0.28 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.83 ns 8.07 ns 0.58 0.05 False
GreaterThanOrEqualBenchmark - Duration of single invocation 10.45 ns 8.05 ns 0.77 0.04 False
MultiplyBenchmark - Duration of single invocation 11.93 ns 7.38 ns 0.62 0.04 False
AddBenchmark - Duration of single invocation 11.67 ns 7.31 ns 0.63 0.04 False
EqualsStaticBenchmark - Duration of single invocation 10.12 ns 7.79 ns 0.77 0.03 False
EqualsAnyBenchmark - Duration of single invocation 3.47 ns 2.16 ns 0.62 0.33 False
GreaterThanAnyBenchmark - Duration of single invocation 4.47 ns 2.12 ns 0.47 0.32 False
BitwiseAndBenchmark - Duration of single invocation 16.09 ns 8.05 ns 0.50 0.28 False
GreaterThanBenchmark - Duration of single invocation 10.22 ns 7.76 ns 0.76 0.03 False
AndNotBenchmark - Duration of single invocation 24.33 ns 15.31 ns 0.63 0.01 False
LessThanBenchmark - Duration of single invocation 12.14 ns 9.61 ns 0.79 0.02 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.82 ns 8.01 ns 0.58 0.03 False
LessThanOrEqualAllBenchmark - Duration of single invocation 4.61 ns 3.02 ns 0.65 0.34 False
XorBenchmark - Duration of single invocation 16.35 ns 8.03 ns 0.49 0.26 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.79 ns 8.00 ns 0.58 0.07 False
EqualityOperatorBenchmark - Duration of single invocation 3.31 ns 1.18 ns 0.36 0.37 False
InequalityOperatorBenchmark - Duration of single invocation 3.52 ns 1.20 ns 0.34 0.37 False
SubtractBenchmark - Duration of single invocation 11.63 ns 7.21 ns 0.62 0.04 False
NegateBenchmark - Duration of single invocation 10.82 ns 6.90 ns 0.64 0.04 False
LessThanAllBenchmark - Duration of single invocation 3.90 ns 2.39 ns 0.61 0.31 False
MaxBenchmark - Duration of single invocation 11.81 ns 9.59 ns 0.81 0.05 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.43 ns 6.38 ns 0.76 0.07 False
ConditionalSelectBenchmark - Duration of single invocation 60.04 ns 32.65 ns 0.54 0.32 False
OnesComplementBenchmark - Duration of single invocation 8.44 ns 6.39 ns 0.76 0.04 False
SubtractionOperatorBenchmark - Duration of single invocation 9.44 ns 7.21 ns 0.76 0.02 False
GreaterThanAllBenchmark - Duration of single invocation 3.34 ns 1.03 ns 0.31 0.39 False
MinBenchmark - Duration of single invocation 11.77 ns 9.48 ns 0.81 0.04 False
DivisionOperatorBenchmark - Duration of single invocation 10.61 ns 8.16 ns 0.77 0.03 False
OneBenchmark - Duration of single invocation 9.14 ns 0.80 ns 0.09 0.33 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Double>.LessThanAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0080226375961852 < 3.183630646322453.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.05019468483151 (T) = (0 -1.2270747147200034) / Math.Sqrt((0.3867537375649668 / (299)) + (0.1879916785137313 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.6610463976074942 = (3.6201849045376404 - 1.2270747147200034) / 3.6201849045376404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.998841789381857 < 8.831912299170593.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 91.95884435347392 (T) = (0 -7.029728527154607) / Math.Sqrt((0.0547693492379467 / (299)) + (0.012383340426610372 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.25699783531229037 = (9.461249053169668 - 7.029728527154607) / 9.461249053169668 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.077305375005599 < 12.235248007335143.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 134.95579064510085 (T) = (0 -8.089290745246778) / Math.Sqrt((0.11327383036103923 / (299)) + (0.02093604516541315 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3711167530483183 = (12.862945203989977 - 8.089290745246778) / 12.862945203989977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2009574335749678 < 5.0611649586795435.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 27.742642361543215 (T) = (0 -2.716427355802858) / Math.Sqrt((0.21485649595941017 / (299)) + (0.16471025403625544 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4706915693712783 = (5.132031153511458 - 2.716427355802858) / 5.132031153511458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1643555964637975 < 5.076890046973705.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 39.24597451408454 (T) = (0 -1.2586371520392305) / Math.Sqrt((0.10960688046159697 / (299)) + (0.23373636824837885 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7581358290853532 = (5.203900798036761 - 1.2586371520392305) / 5.203900798036761 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.152242179689078 < 9.021775541782088.
IsChangePoint: Marked as a change because one of 12/4/2022 6:53:17 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 50.17048537654148 (T) = (0 -7.244733757758045) / Math.Sqrt((0.06744027664323299 / (299)) + (0.047498026578492995 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.24537300257712039 = (9.60041687151331 - 7.244733757758045) / 9.60041687151331 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0848142589632808 < 3.103588368751236.
IsChangePoint: Marked as a change because one of 1/6/2023 2:21:07 AM, 3/7/2023 1:52:33 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 18.840906760365787 (T) = (0 -1.7907894637544262) / Math.Sqrt((0.3296272238287221 / (299)) + (0.22060550499374315 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.5260554419994377 = (3.7784787978350445 - 1.7907894637544262) / 3.7784787978350445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6502206945416862 < 3.232907911246835.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.49334614150636 (T) = (0 -1.8626659407705712) / Math.Sqrt((0.41680769692545533 / (299)) + (0.16701279618903325 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.5132868036057588 = (3.827029870095814 - 1.8626659407705712) / 3.827029870095814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.29855883149661 < 15.399481136375714.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 126.80596955067445 (T) = (0 -8.221862261734055) / Math.Sqrt((0.4114985082656827 / (299)) + (0.0442877967396984 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4667787119547211 = (15.41923108860554 - 8.221862261734055) / 15.41923108860554 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.072504432716928 < 12.837023608654915.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 113.6675338568164 (T) = (0 -8.19380757229662) / Math.Sqrt((0.21875395051505142 / (299)) + (0.041682258415281476 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.41165420275923553 = (13.926856638942775 - 8.19380757229662) / 13.926856638942775 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.049583044442086 < 9.924444397123583.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 47.35502176460006 (T) = (0 -8.279660505113275) / Math.Sqrt((0.4209277513877359 / (299)) + (0.042878679163271344 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.24220644791945617 = (10.926010761613412 - 8.279660505113275) / 10.926010761613412 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.377009579825214 < 11.275128089966957.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 183.09013448694677 (T) = (0 -7.456484996936582) / Math.Sqrt((0.0799904957002595 / (299)) + (0.007824647733352435 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.37707160962552705 = (11.970051633790717 - 7.456484996936582) / 11.970051633790717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.313617430234956 < 11.115149082251179.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 175.0076812541407 (T) = (0 -7.265163207894929) / Math.Sqrt((0.049295217431527864 / (299)) + (0.010782278146818735 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.37380816083725704 = (11.602136523543487 - 7.265163207894929) / 11.602136523543487 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.792566948393395 < 9.682795760676033.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 44.96756440529091 (T) = (0 -7.886097721357083) / Math.Sqrt((0.3607714682142047 / (299)) + (0.05898365787665052 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2565979900738712 = (10.608119988995883 - 7.886097721357083) / 10.608119988995883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1643636997448987 < 3.5364053192030993.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.962443476243283 (T) = (0 -2.1368173897465237) / Math.Sqrt((0.513994370429997 / (299)) + (0.16645569331193666 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4999290027223677 = (4.2730280327778996 - 2.1368173897465237) / 4.2730280327778996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.119012603377594 < 3.997787774339125.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.914938075694142 (T) = (0 -2.2023191021478556) / Math.Sqrt((0.5166221023875182 / (299)) + (0.1795492597927845 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.4843441246048368 = (4.270908579215062 - 2.2023191021478556) / 4.270908579215062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.052380176006679 < 14.049697491769393.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 125.28802509251305 (T) = (0 -8.305633739074453) / Math.Sqrt((0.4077517522644879 / (299)) + (0.044501220555065765 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.46112204779393945 = (15.412829018282933 - 8.305633739074453) / 15.412829018282933 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.75792816142111 < 9.666708077652869.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 64.04655215484748 (T) = (0 -7.9005049835023735) / Math.Sqrt((0.21447633540856526 / (299)) + (0.020905203102133327 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.24419018961854805 = (10.453033124186419 - 7.9005049835023735) / 10.453033124186419 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.306443036876727 < 22.932844325451303.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 150.00808740346753 (T) = (0 -15.528836354460948) / Math.Sqrt((0.29933421349633393 / (299)) + (0.04861288916939243 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.34702134696767895 = (23.781537546974455 - 15.528836354460948) / 23.781537546974455 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.606353010559593 < 11.516186387966945.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 32.66169408804617 (T) = (0 -9.616862200109315) / Math.Sqrt((4.264191886716171 / (299)) + (0.0031289755103880022 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.28949037268038347 = (13.535161003220656 - 9.616862200109315) / 13.535161003220656 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.009934228546252 < 12.836316167967235.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 130.95113258696333 (T) = (0 -8.185295828300562) / Math.Sqrt((0.10045123748299034 / (299)) + (0.03707720084276145 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4096207615203869 = (13.864470995592464 - 8.185295828300562) / 13.864470995592464 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.01828909618734 < 4.768012051129068.
IsChangePoint: Marked as a change because one of 1/7/2023 7:20:27 AM, 3/5/2023 11:54:18 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 18.428790284974735 (T) = (0 -2.6353084954314263) / Math.Sqrt((0.20899806111004215 / (299)) + (0.46420618783733225 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.4878408961794064 = (5.145487946563105 - 2.6353084954314263) / 5.145487946563105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.034275731889053 < 14.71100509570926.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 77.95236926692895 (T) = (0 -8.09456736295777) / Math.Sqrt((0.3998913336125652 / (299)) + (0.17950378839232356 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4748597385285707 = (15.41410544351904 - 8.09456736295777) / 15.41410544351904 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.99693296179482 < 13.110544737758282.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 124.73502667577732 (T) = (0 -8.260247540033879) / Math.Sqrt((0.07003967182379742 / (299)) + (0.04196588922310659 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4020623836108465 = (13.814564117768922 - 8.260247540033879) / 13.814564117768922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.179321610149263 < 3.1593600124166903.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 27.708550838802697 (T) = (0 -1.1406450899813458) / Math.Sqrt((0.38287112650776933 / (299)) + (0.15942505959911693 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.6837735139633332 = (3.607051086318831 - 1.1406450899813458) / 3.607051086318831 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2019452339770456 < 3.2034008854403524.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.57922931088703 (T) = (0 -1.1860643942221745) / Math.Sqrt((0.325985217809169 / (299)) + (0.22333796015876714 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.6738496871193985 = (3.636557585202667 - 1.1860643942221745) / 3.636557585202667 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.213449871267392 < 11.072789429009516.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 169.3771824990484 (T) = (0 -7.263268672788315) / Math.Sqrt((0.05168752235233822 / (299)) + (0.011593647161898021 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.37392262114439867 = (11.601231601858462 - 7.263268672788315) / 11.601231601858462 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.899259099954264 < 10.169805464808384.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 156.1345870663214 (T) = (0 -7.052279223379649) / Math.Sqrt((0.03823277971689862 / (299)) + (0.009919193022664058 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3399488011633724 = (10.684442715670595 - 7.052279223379649) / 10.684442715670595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3919403073954797 < 3.6205233966050536.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 17.5271508660988 (T) = (0 -2.228091180724873) / Math.Sqrt((0.48244270477850904 / (299)) + (0.2978973836005711 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.4778049489572533 = (4.266779580303763 - 2.228091180724873) / 4.266779580303763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.590510730238215 < 11.212662501035185.
IsChangePoint: Marked as a change because one of 11/24/2022 8:21:24 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 53.79021252002746 (T) = (0 -9.376967240125436) / Math.Sqrt((0.08015767750493648 / (299)) + (0.03961316450887486 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.20081062068650063 = (11.733097915015106 - 9.376967240125436) / 11.733097915015106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.37505917669809 < 8.014643369591006.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 32.24858224798514 (T) = (0 -6.504409836515895) / Math.Sqrt((0.17123299223848096 / (299)) + (0.09580591237623606 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.25091867920500843 = (8.683182527649786 - 6.504409836515895) / 8.683182527649786 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.65099635776824 < 56.161833561182824.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 102.31098851658412 (T) = (0 -32.46774767735751) / Math.Sqrt((16.130289644975228 / (299)) + (0.026863375575200354 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.42510906079878336 = (56.47636005964868 - 32.46774767735751) / 56.47636005964868 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.393658483348002 < 7.9687402832069925.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 52.22334989477065 (T) = (0 -6.417120608758683) / Math.Sqrt((0.15050834314733588 / (299)) + (0.033182353000036434 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2611314343077244 = (8.685063767391735 - 6.417120608758683) / 8.685063767391735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.205430060246068 < 8.974142557667264.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 3/6/2023 6:03:21 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 89.25293733372597 (T) = (0 -7.260784410940939) / Math.Sqrt((0.08864025350684536 / (299)) + (0.010345824619064628 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.2467664353399131 = (9.6394860128379 - 7.260784410940939) / 9.6394860128379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0253004353702708 < 3.2021849168364422.
IsChangePoint: Marked as a change because one of 1/5/2023 11:12:07 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.745722157198283 (T) = (0 -1.1823858299694228) / Math.Sqrt((0.3504823419498958 / (299)) + (0.14819614837647857 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.67573162794535 = (3.6463187034785856 - 1.1823858299694228) / 3.6463187034785856 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.477745715113977 < 11.076543482790054.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 74.58430250937766 (T) = (0 -9.25322494778712) / Math.Sqrt((0.18416067386716556 / (299)) + (0.014166382708717338 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.22051624530407707 = (11.870965741161351 - 9.25322494778712) / 11.870965741161351 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.156434930382314 < 10.026799137704923.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 99.55821909978386 (T) = (0 -8.011888085194633) / Math.Sqrt((0.11289763546199794 / (299)) + (0.008544787965550082 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2518168691324319 = (10.708458604117839 - 8.011888085194633) / 10.708458604117839 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7999545554100257 < 8.090606433012798.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 119.24228094780125 (T) = (0 -0.9221411241705555) / Math.Sqrt((1.079232149762783 / (299)) + (0.03654554622028462 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.9025693141825976 = (9.464586197194242 - 0.9221411241705555) / 9.464586197194242 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InequalityOperatorBenchmark - Duration of single invocation 7.55 ns 0.70 ns 0.09 0.41 False
EqualityOperatorBenchmark - Duration of single invocation 5.43 ns 0.03 ns 0.01 0.47 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector4* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector4* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector4.InequalityOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.6973230801312015 < 7.218363934194393.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.593156502638003 (T) = (0 -0.8566225880288454) / Math.Sqrt((9.35064392950267 / (299)) + (0.12811270528065088 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.84050796451186 = (5.370942727058898 - 0.8566225880288454) / 5.370942727058898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.027982488531652677 < 5.151495941148367.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.69998646616387 (T) = (0 -0.8030350042919262) / Math.Sqrt((4.276422423724885 / (299)) + (0.1338308533216315 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7920975241741532 = (3.8625562350907403 - 0.8030350042919262) / 3.8625562350907403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsStaticBenchmark - Duration of single invocation 38.65 ns 36.40 ns 0.94 0.00 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.20 ns 12.86 ns 0.91 0.02 False
EqualsAnyBenchmark - Duration of single invocation 23.29 ns 20.88 ns 0.90 0.01 False
AndNotBenchmark - Duration of single invocation 14.47 ns 13.14 ns 0.91 0.04 False
LessThanAnyBenchmark - Duration of single invocation 23.31 ns 21.11 ns 0.91 0.02 False
GreaterThanBenchmark - Duration of single invocation 40.13 ns 37.63 ns 0.94 0.00 False
EqualsAllBenchmark - Duration of single invocation 7.08 ns 5.57 ns 0.79 0.11 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.27 ns 12.99 ns 0.91 0.03 False
ConditionalSelectBenchmark - Duration of single invocation 16.50 ns 15.49 ns 0.94 0.02 False
EqualsBenchmark - Duration of single invocation 10.62 ns 8.79 ns 0.83 0.14 False
DotBenchmark - Duration of single invocation 25.67 ns 22.68 ns 0.88 0.02 False
DivisionOperatorBenchmark - Duration of single invocation 62.44 ns 56.38 ns 0.90 0.01 False
BitwiseAndBenchmark - Duration of single invocation 15.50 ns 12.99 ns 0.84 0.02 False
GreaterThanAllBenchmark - Duration of single invocation 23.75 ns 21.08 ns 0.89 0.01 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.29 ns 12.89 ns 0.90 0.01 False
MultiplyBenchmark - Duration of single invocation 38.59 ns 35.24 ns 0.91 0.00 False
DivideBenchmark - Duration of single invocation 62.84 ns 56.43 ns 0.90 0.00 False
XorBenchmark - Duration of single invocation 15.50 ns 12.83 ns 0.83 0.03 False
BitwiseOrBenchmark - Duration of single invocation 15.57 ns 12.83 ns 0.82 0.02 False
SubtractBenchmark - Duration of single invocation 38.27 ns 34.83 ns 0.91 0.01 False
LessThanBenchmark - Duration of single invocation 38.68 ns 36.67 ns 0.95 0.00 False
AddBenchmark - Duration of single invocation 38.26 ns 34.66 ns 0.91 0.00 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualsStaticBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.4036193224489 < 36.75336734163659.
IsChangePoint: Marked as a change because one of 12/12/2022 3:40:04 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 48.4432346135428 (T) = (0 -36.43092360835646) / Math.Sqrt((0.08690435122362018 / (299)) + (0.04089564049640669 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.05703104669806632 = (38.63427685586958 - 36.43092360835646) / 38.63427685586958 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.858400762525457 < 13.632423408959907.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.041209729576945 (T) = (0 -12.906373050373626) / Math.Sqrt((0.18499510645943687 / (299)) + (0.05683288618598888 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11617202958934919 = (14.602811273756101 - 12.906373050373626) / 14.602811273756101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.882637048391455 < 22.007303019572976.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.809035146502588 (T) = (0 -21.19912583014192) / Math.Sqrt((0.4131140488956938 / (299)) + (0.044800501635999995 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06016008780230524 = (22.556102964993784 - 21.19912583014192) / 22.556102964993784 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.136479522422556 < 13.774412112288713.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 12/14/2022 8:02:51 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.250255186967188 (T) = (0 -13.14160899057062) / Math.Sqrt((0.08334368267191307 / (299)) + (0.03901706026801293 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.09254849402340595 = (14.481885703002606 - 13.14160899057062) / 14.481885703002606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.112159592490464 < 22.163908937119185.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.788304555026667 (T) = (0 -21.20416558108131) / Math.Sqrt((0.4202539123427807 / (299)) + (0.07509891720026926 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05912758818727819 = (22.536706693555324 - 21.20416558108131) / 22.536706693555324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.63051312050592 < 38.21663359371569.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.003765310360524 (T) = (0 -37.632467152429925) / Math.Sqrt((0.48529618764113236 / (299)) + (0.10053339531877616 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05549539131058075 = (39.843603521054476 - 37.632467152429925) / 39.843603521054476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.567130714343409 < 7.715526992337954.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 32.44208550969523 (T) = (0 -5.623156591142354) / Math.Sqrt((0.15379998100266234 / (299)) + (0.11031064572229252 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.29598900635106734 = (7.9873136099724595 - 5.623156591142354) / 7.9873136099724595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.990120295388882 < 13.587021367159107.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 9:06:14 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 32.981894055479685 (T) = (0 -12.89311265906149) / Math.Sqrt((0.17422685259178436 / (299)) + (0.04651041125712254 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11548315540225237 = (14.57644672094956 - 12.89311265906149) / 14.57644672094956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.4920668809961 < 15.798536535353213.
IsChangePoint: Marked as a change because one of 11/21/2022 12:46:21 PM, 12/5/2022 3:49:45 AM, 12/14/2022 12:01:27 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 33.2086039145573 (T) = (0 -15.369719702509684) / Math.Sqrt((0.07955394229160248 / (299)) + (0.034693313987494166 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08205502453219228 = (16.74361766039069 - 15.369719702509684) / 16.74361766039069 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.794285203623732 < 10.18545892546895.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.721583617973746 (T) = (0 -8.520353545273757) / Math.Sqrt((0.973174941173133 / (299)) + (0.04197190575884804 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.16453408431552402 = (10.198325730970422 - 8.520353545273757) / 10.198325730970422 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.683169927040748 < 24.297201224891474.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.9771194634498 (T) = (0 -23.02293842657201) / Math.Sqrt((0.27440826780605526 / (299)) + (0.20909702864303065 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09621900658788177 = (25.474023678736184 - 23.02293842657201) / 25.474023678736184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 56.376362749416764 < 58.98631734883969.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.38515335058657 (T) = (0 -57.043001115825085) / Math.Sqrt((2.7632882079022267 / (299)) + (0.7155145937815118 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06897755227788474 = (61.269200603475525 - 57.043001115825085) / 61.269200603475525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.992473668491025 < 14.729960362878254.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 82.41749714417912 (T) = (0 -12.91690121493653) / Math.Sqrt((0.275032898659448 / (299)) + (0.008768937288453082 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18616056285804394 = (15.871559702609332 - 12.91690121493653) / 15.871559702609332 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.08246408403119 < 21.917808874813847.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 20.705701051116367 (T) = (0 -21.268407333683164) / Math.Sqrt((0.3692823902104769 / (299)) + (0.07762590849972707 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06110807943227452 = (22.652668393208312 - 21.268407333683164) / 22.652668393208312 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.887794354190298 < 13.58543765532076.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 46.670652812559034 (T) = (0 -12.94200532488644) / Math.Sqrt((0.20854465165859032 / (299)) + (0.014028047428580732 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11435190304435676 = (14.613033516781357 - 12.94200532488644) / 14.613033516781357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.24423780874229 < 36.5760038999224.
IsChangePoint: Marked as a change because one of 12/16/2022 11:20:37 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 50.97547381063652 (T) = (0 -35.074238889365866) / Math.Sqrt((0.26007029023331507 / (299)) + (0.08939493458012764 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08967903118970118 = (38.52952979343593 - 35.074238889365866) / 38.52952979343593 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 56.434769275092016 < 59.97241138242562.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.464216538669096 (T) = (0 -57.20461194862515) / Math.Sqrt((0.23606747078806092 / (299)) + (0.7447107292359271 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08934756667061843 = (62.817173550487105 - 57.20461194862515) / 62.817173550487105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.830256877312953 < 14.664457622419661.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 40.47710167686828 (T) = (0 -12.868034160367158) / Math.Sqrt((0.6267963101903139 / (299)) + (0.09477924915980425 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1965125915869183 = (16.01522814872982 - 12.868034160367158) / 16.01522814872982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.831053456321785 < 14.756883976476873.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 40.07766419613775 (T) = (0 -12.936355940433344) / Math.Sqrt((0.5614799914295832 / (299)) + (0.09786475841986923 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.19295262658337414 = (16.029239876796115 - 12.936355940433344) / 16.029239876796115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.82950570740731 < 36.36008183444766.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.28688878813183 (T) = (0 -34.36145856661095) / Math.Sqrt((0.13233101667020153 / (299)) + (0.2567951259383132 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09779891156138541 = (38.086252618114514 - 34.36145856661095) / 38.086252618114514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.66916605343657 < 36.71471750950804.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 52.62786496252616 (T) = (0 -36.433001381508376) / Math.Sqrt((0.08428937140813088 / (299)) + (0.035196619679059474 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0569606269253717 = (38.63359518354407 - 36.433001381508376) / 38.63359518354407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.65928874155108 < 36.27331569537763.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 34.8895574159757 (T) = (0 -34.34512410907001) / Math.Sqrt((0.10642404628089742 / (299)) + (0.265778114146855 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09796628949262795 = (38.07521127979985 - 34.34512410907001) / 38.07521127979985 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseAndBenchmark - Duration of single invocation 15.53 ns 12.95 ns 0.83 0.03 False
DotBenchmark - Duration of single invocation 10.42 ns 8.77 ns 0.84 0.06 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.36 ns 12.94 ns 0.90 0.02 False
LessThanBenchmark - Duration of single invocation 19.69 ns 17.98 ns 0.91 0.02 False
BitwiseOrBenchmark - Duration of single invocation 15.48 ns 12.93 ns 0.84 0.03 False
XorBenchmark - Duration of single invocation 15.45 ns 12.83 ns 0.83 0.04 False
LessThanAnyBenchmark - Duration of single invocation 11.16 ns 9.60 ns 0.86 0.09 False
SubtractBenchmark - Duration of single invocation 20.20 ns 16.84 ns 0.83 0.03 False
DivideBenchmark - Duration of single invocation 23.96 ns 19.76 ns 0.82 0.02 False
AddOperatorBenchmark - Duration of single invocation 18.20 ns 16.64 ns 0.91 0.05 False
MaxBenchmark - Duration of single invocation 20.72 ns 19.25 ns 0.93 0.02 False
EqualsAllBenchmark - Duration of single invocation 7.90 ns 4.83 ns 0.61 0.31 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.18 ns 12.82 ns 0.90 0.03 False
ConditionalSelectBenchmark - Duration of single invocation 16.50 ns 15.19 ns 0.92 0.04 False
EqualsStaticBenchmark - Duration of single invocation 19.72 ns 17.99 ns 0.91 0.00 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 12.34 ns 10.25 ns 0.83 0.07 False
MultiplyOperatorBenchmark - Duration of single invocation 18.72 ns 16.77 ns 0.90 0.01 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 11.69 ns 10.49 ns 0.90 0.04 False
MinBenchmark - Duration of single invocation 20.82 ns 19.10 ns 0.92 0.02 False
AndNotBenchmark - Duration of single invocation 14.55 ns 13.29 ns 0.91 0.04 False
DivisionOperatorBenchmark - Duration of single invocation 21.69 ns 19.76 ns 0.91 0.02 False
GreaterThanAllBenchmark - Duration of single invocation 11.57 ns 9.28 ns 0.80 0.08 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.19 ns 12.82 ns 0.90 0.04 False
SubtractionOperatorBenchmark - Duration of single invocation 18.18 ns 17.05 ns 0.94 0.03 False
EqualsBenchmark - Duration of single invocation 10.81 ns 7.72 ns 0.71 0.09 False
AddBenchmark - Duration of single invocation 20.18 ns 16.84 ns 0.83 0.02 False
EqualsAnyBenchmark - Duration of single invocation 10.93 ns 9.06 ns 0.83 0.09 False
MultiplyBenchmark - Duration of single invocation 20.55 ns 17.02 ns 0.83 0.04 False
GreaterThanBenchmark - Duration of single invocation 20.14 ns 17.70 ns 0.88 0.01 False
GreaterThanOrEqualBenchmark - Duration of single invocation 20.89 ns 19.07 ns 0.91 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.BitwiseAndBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.947936409404614 < 14.708760112045212.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 40.74425100167383 (T) = (0 -12.884022539997048) / Math.Sqrt((0.18810609626770866 / (299)) + (0.10865846368412596 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18506267326901 = (15.809832385122851 - 12.884022539997048) / 15.809832385122851 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.768194758726604 < 9.904751915730728.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 41.31262652205627 (T) = (0 -8.371324165056707) / Math.Sqrt((0.5659798976937674 / (299)) + (0.043983828172450175 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.23149091474996084 = (10.892941053953898 - 8.371324165056707) / 10.892941053953898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.941290257009245 < 13.656495834100726.
IsChangePoint: Marked as a change because one of 12/13/2022 1:36:02 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 45.231555241443154 (T) = (0 -12.967208777634664) / Math.Sqrt((0.25094603158683393 / (299)) + (0.01192425784356508 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1138908900295173 = (14.63387367506764 - 12.967208777634664) / 14.63387367506764 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.97871605697167 < 18.69957495597108.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.738227683983546 (T) = (0 -17.938481836638775) / Math.Sqrt((0.4886640096068098 / (299)) + (0.020115992527253167 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1081296634985107 = (20.113329373645808 - 17.938481836638775) / 20.113329373645808 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.933182978136724 < 14.714233578284482.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 58.55116260115222 (T) = (0 -12.971387150883075) / Math.Sqrt((0.5038940037142817 / (299)) + (0.023465528909347243 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1889270563892478 = (15.992873702748327 - 12.971387150883075) / 15.992873702748327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.834248459125119 < 14.74967234108858.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.570092277158295 (T) = (0 -12.907534225397491) / Math.Sqrt((0.497921349268063 / (299)) + (0.08674723597245033 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.19331740567716493 = (16.000759550579673 - 12.907534225397491) / 16.000759550579673 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.603324422951015 < 10.335286530416766.
IsChangePoint: Marked as a change because one of 12/15/2022 5:36:17 AM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.152871466392725 (T) = (0 -9.304718777839643) / Math.Sqrt((0.49727158218196676 / (299)) + (0.13717429629572692 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.16978769169437938 = (11.207637714778805 - 9.304718777839643) / 11.207637714778805 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.841068094181445 < 19.158831575959695.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 56.188617645420955 (T) = (0 -16.72146493274396) / Math.Sqrt((0.23824160560792507 / (299)) + (0.08731304050022975 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1828567869579217 = (20.463322298783993 - 16.72146493274396) / 20.463322298783993 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.760546962781632 < 22.667820850051807.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 84.00890569658014 (T) = (0 -19.754416543050425) / Math.Sqrt((0.061570813138743956 / (299)) + (0.053373865299437113 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.17329958885066096 = (23.895496211966798 - 19.754416543050425) / 23.895496211966798 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.641937625217995 < 17.373186269730382.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.487360892603963 (T) = (0 -16.705330635578786) / Math.Sqrt((0.7207106597692149 / (299)) + (0.13853528985027003 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11283228474360732 = (18.829957795241594 - 16.705330635578786) / 18.829957795241594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.253074579968885 < 19.696840171346974.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 37.17040310046069 (T) = (0 -19.211119715556457) / Math.Sqrt((0.5822755298833481 / (299)) + (0.023252206877155433 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09460127259330302 = (21.218408126751203 - 19.211119715556457) / 21.218408126751203 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.827517128593846 < 7.791049812852094.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.27978896061665 (T) = (0 -4.677952571138243) / Math.Sqrt((0.2966854813913855 / (299)) + (0.2317772946992258 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.4038433756128535 = (7.846851615457957 - 4.677952571138243) / 7.846851615457957 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.823869336876198 < 13.653022682543755.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 41.92514487237743 (T) = (0 -12.97088292683666) / Math.Sqrt((0.2844758551793466 / (299)) + (0.01576236365511061 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1156438797017053 = (14.667035857072557 - 12.97088292683666) / 14.667035857072557 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.19298919451598 < 15.81569712878293.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.537063221307985 (T) = (0 -15.39407749716144) / Math.Sqrt((0.15171713876827872 / (299)) + (0.04616743541587289 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08642793639140169 = (16.85042495318325 - 15.39407749716144) / 16.85042495318325 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.985402049282744 < 18.67204400901622.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 52.86227286316589 (T) = (0 -17.929773078853867) / Math.Sqrt((0.21778807359674912 / (299)) + (0.019422699079555465 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10363086583711288 = (20.00266675357843 - 17.929773078853867) / 20.00266675357843 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.250651121368815 < 11.451036161735965.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.896767884971574 (T) = (0 -10.256870251515613) / Math.Sqrt((0.07476453838598496 / (299)) + (0.05630559187467872 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.15133480921265186 = (12.085885414953585 - 10.256870251515613) / 12.085885414953585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.769246443294684 < 17.73154186520033.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 56.884406658921534 (T) = (0 -16.950367609373142) / Math.Sqrt((0.19007844211729202 / (299)) + (0.01430334794422412 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10536798390925357 = (18.946748276951663 - 16.950367609373142) / 18.946748276951663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.48830627020225 < 11.537513605656425.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 48.83370937399653 (T) = (0 -10.47768604430543) / Math.Sqrt((0.06741219093601404 / (299)) + (0.026663514759822312 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14557957940918112 = (12.262916231637194 - 10.47768604430543) / 12.262916231637194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.099086484903037 < 19.77518846822476.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.210186045888282 (T) = (0 -19.233691956262884) / Math.Sqrt((0.5013179057377266 / (299)) + (0.09308579308459 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09854571790427996 = (21.3362921872732 - 19.233691956262884) / 21.3362921872732 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.292210157126018 < 13.808429967808319.
IsChangePoint: Marked as a change because one of 11/21/2022 12:46:21 PM, 12/5/2022 3:49:45 AM, 12/12/2022 3:40:04 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.772634320274058 (T) = (0 -13.16850681783414) / Math.Sqrt((0.11245757549347 / (299)) + (0.06623371498591625 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09877215914245602 = (14.6117399184028 - 13.16850681783414) / 14.6117399184028 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.764213331891185 < 20.55768301417454.
IsChangePoint: Marked as a change because one of 12/5/2022 9:44:53 PM, 1/5/2023 11:12:07 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 45.646378305617255 (T) = (0 -19.788443636281166) / Math.Sqrt((0.2822794882737133 / (299)) + (0.03391586820088623 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10071494904817044 = (22.00463981396833 - 19.788443636281166) / 22.00463981396833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.28122952128273 < 10.529913445605244.
IsChangePoint: Marked as a change because one of 12/14/2022 8:02:51 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.190892522882994 (T) = (0 -9.358005681373394) / Math.Sqrt((0.3754386167813039 / (299)) + (0.13911453311862274 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.17836388421096322 = (11.389477046522812 - 9.358005681373394) / 11.389477046522812 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.815707223784164 < 13.5815731470635.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 32.90039526911039 (T) = (0 -12.958575175558778) / Math.Sqrt((0.3044749310598751 / (299)) + (0.03773515430473112 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11443648399105531 = (14.633140301398651 - 12.958575175558778) / 14.633140301398651 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.054656004934017 < 17.34410721122149.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.984147809002582 (T) = (0 -16.682050050204207) / Math.Sqrt((0.6119265936559892 / (299)) + (0.10626771352930837 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11137262684387646 = (18.772829370487248 - 16.682050050204207) / 18.772829370487248 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.718646024101901 < 10.256339045603319.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.042558385480323 (T) = (0 -7.829531814297005) / Math.Sqrt((1.162695604414055 / (299)) + (0.16867648445914354 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.21924323389727635 = (10.028131876947294 - 7.829531814297005) / 10.028131876947294 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.8409755922223 < 19.178071565001403.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 97.40428327314739 (T) = (0 -16.735211886752445) / Math.Sqrt((0.2202779030008498 / (299)) + (0.017060661856953225 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18129814035671155 = (20.44115533589241 - 16.735211886752445) / 20.44115533589241 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.060358850109374 < 10.388716618118229.
IsChangePoint: Marked as a change because one of 12/15/2022 5:36:17 AM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.42211560962613 (T) = (0 -9.21844871505927) / Math.Sqrt((0.4952305251714721 / (299)) + (0.22493892416736053 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1786220081601428 = (11.223150372473794 - 9.21844871505927) / 11.223150372473794 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.01617189322745 < 19.517671755435334.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 45.63331930655426 (T) = (0 -16.811389134723683) / Math.Sqrt((0.07553101862960669 / (299)) + (0.16128245055247656 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18477974144298445 = (20.621898141344968 - 16.811389134723683) / 20.621898141344968 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.69997505099674 < 19.121118800863716.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 36.00941166613425 (T) = (0 -18.105564131996363) / Math.Sqrt((0.5223188502629601 / (299)) + (0.07378321066779653 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12133945598134213 = (20.60586907565967 - 18.105564131996363) / 20.60586907565967 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.067978993809568 < 19.870473541904474.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 27.961416922969264 (T) = (0 -19.06378846947976) / Math.Sqrt((0.18945124155598253 / (299)) + (0.13876702327399074 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.10341565558516773 = (21.262682745059525 - 19.06378846947976) / 21.262682745059525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsBenchmark - Duration of single invocation 10.93 ns 7.99 ns 0.73 0.13 False
ConditionalSelectBenchmark - Duration of single invocation 16.68 ns 15.26 ns 0.91 0.05 False
LessThanAnyBenchmark - Duration of single invocation 10.94 ns 9.83 ns 0.90 0.09 False
EqualsAllBenchmark - Duration of single invocation 7.85 ns 4.14 ns 0.53 0.34 False
XorBenchmark - Duration of single invocation 15.53 ns 12.84 ns 0.83 0.01 False
DotBenchmark - Duration of single invocation 10.71 ns 7.06 ns 0.66 0.16 False
DivisionOperatorBenchmark - Duration of single invocation 31.16 ns 29.03 ns 0.93 0.00 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.25 ns 12.82 ns 0.90 0.02 False
MultiplyOperatorBenchmark - Duration of single invocation 18.59 ns 16.77 ns 0.90 0.01 False
DivideBenchmark - Duration of single invocation 33.81 ns 28.77 ns 0.85 0.00 False
BitwiseOrBenchmark - Duration of single invocation 15.55 ns 12.94 ns 0.83 0.02 False
GreaterThanBenchmark - Duration of single invocation 20.16 ns 17.91 ns 0.89 0.02 False
LessThanBenchmark - Duration of single invocation 19.66 ns 17.81 ns 0.91 0.01 False
MinBenchmark - Duration of single invocation 20.80 ns 19.10 ns 0.92 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 11.39 ns 8.81 ns 0.77 0.09 False
SubtractBenchmark - Duration of single invocation 20.16 ns 16.95 ns 0.84 0.01 False
EqualsStaticBenchmark - Duration of single invocation 19.65 ns 17.97 ns 0.91 0.02 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.23 ns 12.86 ns 0.90 0.02 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 12.49 ns 10.48 ns 0.84 0.06 False
AndNotBenchmark - Duration of single invocation 14.61 ns 13.09 ns 0.90 0.08 False
LessThanOrEqualBenchmark - Duration of single invocation 20.39 ns 18.59 ns 0.91 0.01 False
SubtractionOperatorBenchmark - Duration of single invocation 18.28 ns 16.55 ns 0.91 0.01 False
GreaterThanOrEqualBenchmark - Duration of single invocation 20.92 ns 18.68 ns 0.89 0.01 False
MultiplyBenchmark - Duration of single invocation 20.51 ns 16.81 ns 0.82 0.01 False
MaxBenchmark - Duration of single invocation 20.74 ns 19.29 ns 0.93 0.01 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 12.04 ns 10.62 ns 0.88 0.05 False
BitwiseAndBenchmark - Duration of single invocation 15.52 ns 12.71 ns 0.82 0.02 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.33 ns 13.03 ns 0.91 0.04 False
EqualsAnyBenchmark - Duration of single invocation 11.07 ns 9.77 ns 0.88 0.10 False
AddBenchmark - Duration of single invocation 20.04 ns 16.56 ns 0.83 0.04 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.EqualsBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.991591695569514 < 10.496732705812539.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 16.829571247725973 (T) = (0 -7.731174027357919) / Math.Sqrt((1.4065108428452702 / (299)) + (0.3646306883261108 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.23492339565590345 = (10.105097951578179 - 7.731174027357919) / 10.105097951578179 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.256405129867302 < 15.845696602078817.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.130538335650236 (T) = (0 -15.380519366980224) / Math.Sqrt((0.13582002102096688 / (299)) + (0.05218943649392659 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08851305823804605 = (16.87409732634111 - 15.380519366980224) / 16.87409732634111 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.825088980876085 < 10.3478789435423.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 20.424479370224788 (T) = (0 -9.090492546789209) / Math.Sqrt((0.4978120429807613 / (299)) + (0.22741994602468174 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.19168977611642432 = (11.246291681321788 - 9.090492546789209) / 11.246291681321788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.142163572995619 < 7.504232465743862.
IsChangePoint: Marked as a change because one of 1/5/2023 9:29:47 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.878854247333695 (T) = (0 -4.561201236134678) / Math.Sqrt((0.30502159304494636 / (299)) + (0.2814269523418991 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.42056852009105206 = (7.871856111185789 - 4.561201236134678) / 7.871856111185789 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.83896852217368 < 14.745781650287997.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 61.9846340910226 (T) = (0 -12.990945664102354) / Math.Sqrt((0.45137003702681594 / (299)) + (0.01980325250523432 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18735415436607214 = (15.98598668029662 - 12.990945664102354) / 15.98598668029662 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.0585359096881435 < 10.153446740238225.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.43467293573545 (T) = (0 -8.361129720610014) / Math.Sqrt((0.6344640292577112 / (299)) + (0.22438154723431078 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2308823948377805 = (10.871067915350235 - 8.361129720610014) / 10.871067915350235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.03068283489667 < 30.094793612391125.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.23278188224315 (T) = (0 -29.22437822584062) / Math.Sqrt((0.10964056338003618 / (299)) + (0.1847550330977848 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06930471349079757 = (31.400586904714768 - 29.22437822584062) / 31.400586904714768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.816344596777496 < 13.63658794967723.
IsChangePoint: Marked as a change because one of 1/5/2023 11:12:07 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 47.811727155368196 (T) = (0 -12.96208531573859) / Math.Sqrt((0.11024244057944708 / (299)) + (0.01558102989119255 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10658754121663158 = (14.508511928958438 - 12.96208531573859) / 14.508511928958438 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.773290512713398 < 17.75641998519774.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 38.72971932616346 (T) = (0 -16.87173764048654) / Math.Sqrt((0.48729668078209776 / (299)) + (0.04156408476893494 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11746033806158407 = (19.117257125225787 - 16.87173764048654) / 19.117257125225787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.768309943593334 < 31.945064602004518.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 56.34696261305927 (T) = (0 -29.245691370316568) / Math.Sqrt((0.06164569002203434 / (299)) + (0.14494189874347405 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.13214125836900947 = (33.69867694753451 - 29.245691370316568) / 33.69867694753451 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.939915980714918 < 14.718884329244268.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 59.84003528817054 (T) = (0 -12.966337037325943) / Math.Sqrt((0.21700586088922103 / (299)) + (0.0360862810843371 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.17891726620651335 = (15.791754598735906 - 12.966337037325943) / 15.791754598735906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.908371533610204 < 19.15844196493568.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.772295660881035 (T) = (0 -18.109363542887234) / Math.Sqrt((0.5533475862362315 / (299)) + (0.07053600227779457 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12026710090644613 = (20.58507026569825 - 18.109363542887234) / 20.58507026569825 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.811085273338314 < 18.66450712578726.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 44.39477267640353 (T) = (0 -17.934047868356483) / Math.Sqrt((0.4844540239243368 / (299)) + (0.019875792615474695 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1091222486555925 = (20.130761870854375 - 17.934047868356483) / 20.130761870854375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.099093469450246 < 19.79842193327603.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 32.24454197203384 (T) = (0 -19.250920354197607) / Math.Sqrt((0.5265854174856563 / (299)) + (0.05835094628725434 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09784111953118559 = (21.33872510814692 - 19.250920354197607) / 21.33872510814692 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.811955829957665 < 10.469440201451006.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.82625136972761 (T) = (0 -9.385659626786682) / Math.Sqrt((0.43236013952045804 / (299)) + (0.07801712009701164 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.173884911865186 = (11.36120107426852 - 9.385659626786682) / 11.36120107426852 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.948834404377042 < 19.13105699366634.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 99.72444100982484 (T) = (0 -16.7127195667906) / Math.Sqrt((0.17930635444954707 / (299)) + (0.019203655582824587 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1825063349131655 = (20.44385208173493 - 16.7127195667906) / 20.44385208173493 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.974178275734527 < 18.662259905578857.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.16406502592151 (T) = (0 -17.925137056399155) / Math.Sqrt((0.570014208743388 / (299)) + (0.01980640324994516 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11178693172198342 = (20.181122859575478 - 17.925137056399155) / 20.181122859575478 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.85993897967359 < 13.650665690251762.
IsChangePoint: Marked as a change because one of 1/4/2023 2:39:28 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.91281781072941 (T) = (0 -12.922989828004898) / Math.Sqrt((0.12157522979092558 / (299)) + (0.048889850477594116 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11053152323134549 = (14.52889019176122 - 12.922989828004898) / 14.52889019176122 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.477652182009331 < 11.591191734523663.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.73827072373747 (T) = (0 -10.519026898705617) / Math.Sqrt((0.053830782166554764 / (299)) + (0.12398549698206385 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14163315306348415 = (12.254698484974917 - 10.519026898705617) / 12.254698484974917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.093150344078689 < 13.762465072662968.
IsChangePoint: Marked as a change because one of 11/21/2022 12:46:21 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 32.07807094636097 (T) = (0 -13.055918154847541) / Math.Sqrt((0.07649003099053035 / (299)) + (0.04275211050617477 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0998251552461821 = (14.503758054267898 - 13.055918154847541) / 14.503758054267898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.594061195692845 < 19.3429381119637.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.664296375114308 (T) = (0 -18.845695761840016) / Math.Sqrt((0.2808389927570428 / (299)) + (0.07708495004627049 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.09060455450936765 = (20.72332323115219 - 18.845695761840016) / 20.72332323115219 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.553576815652992 < 17.374387569078003.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 36.17997007361313 (T) = (0 -16.698569919449866) / Math.Sqrt((0.36819272504466805 / (299)) + (0.042472621352504916 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10610069235724869 = (18.680593861835145 - 16.698569919449866) / 18.680593861835145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.675905400726993 < 19.87833351532392.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.663857257512237 (T) = (0 -18.963085048635968) / Math.Sqrt((0.4974444919481526 / (299)) + (0.19276661827989072 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.11585321578508032 = (21.4478923490903 - 18.963085048635968) / 21.4478923490903 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.805023270260087 < 19.468803518743876.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 137.52648622337853 (T) = (0 -16.935295313900966) / Math.Sqrt((0.06512765283080023 / (299)) + (0.011792400881799638 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.177804691592699 = (20.597655010592106 - 16.935295313900966) / 20.597655010592106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.290060517179374 < 19.684816265394247.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 36.9191258949829 (T) = (0 -19.230644753407834) / Math.Sqrt((0.5392788050180128 / (299)) + (0.02739411888527763 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09435410992749865 = (21.234176585142265 - 19.230644753407834) / 21.234176585142265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.618056970888055 < 11.42359079685536.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.382044107512954 (T) = (0 -10.28105034699615) / Math.Sqrt((0.05717686529102786 / (299)) + (0.038101562767901574 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.15107849672261625 = (12.110719668785245 - 10.28105034699615) / 12.110719668785245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.708645954509302 < 14.724322600527117.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 52.208790326091545 (T) = (0 -12.902170164379221) / Math.Sqrt((0.5127425111424811 / (299)) + (0.042496705500391176 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.19283257375660637 = (15.984503022410985 - 12.902170164379221) / 15.984503022410985 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.027431066800068 < 13.590806790099464.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 41.615282453593665 (T) = (0 -12.939970172830165) / Math.Sqrt((0.2717112779335731 / (299)) + (0.018555649519815958 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11652332390501872 = (14.646646055247992 - 12.939970172830165) / 14.646646055247992 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.768516198043281 < 10.381807050499308.
IsChangePoint: Marked as a change because one of 12/15/2022 1:21:30 AM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.91464924396231 (T) = (0 -9.206126405760912) / Math.Sqrt((0.5314152560939454 / (299)) + (0.2115132613047451 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.17957258679523444 = (11.221134566676419 - 9.206126405760912) / 11.221134566676419 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.56480781455252 < 19.171203199227833.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 65.27709864529317 (T) = (0 -16.72315349933019) / Math.Sqrt((0.19897654982826668 / (299)) + (0.06234715956123583 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18232629879365478 = (20.452111245179935 - 16.72315349933019) / 20.452111245179935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAllBenchmark - Duration of single invocation 13.13 ns 10.24 ns 0.78 0.05 False
MinBenchmark - Duration of single invocation 45.36 ns 36.56 ns 0.81 0.06 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.44 ns 12.85 ns 0.89 0.01 False
SubtractionOperatorBenchmark - Duration of single invocation 39.30 ns 36.12 ns 0.92 0.03 False
EqualityOperatorBenchmark - Duration of single invocation 11.57 ns 9.57 ns 0.83 0.07 False
SubtractBenchmark - Duration of single invocation 45.25 ns 36.12 ns 0.80 0.01 False
BitwiseAndBenchmark - Duration of single invocation 14.59 ns 12.85 ns 0.88 0.01 False
EqualsStaticBenchmark - Duration of single invocation 41.97 ns 30.79 ns 0.73 0.01 False
MultiplyOperatorBenchmark - Duration of single invocation 40.99 ns 36.98 ns 0.90 0.02 False
ConditionalSelectBenchmark - Duration of single invocation 16.84 ns 15.51 ns 0.92 0.03 False
AddOperatorBenchmark - Duration of single invocation 39.38 ns 36.12 ns 0.92 0.02 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 19.89 ns 18.38 ns 0.92 0.02 False
DivisionOperatorBenchmark - Duration of single invocation 44.11 ns 39.25 ns 0.89 0.06 False
AndNotBenchmark - Duration of single invocation 14.60 ns 13.08 ns 0.90 0.05 False
XorBenchmark - Duration of single invocation 15.69 ns 12.96 ns 0.83 0.03 False
MultiplyBenchmark - Duration of single invocation 46.19 ns 36.96 ns 0.80 0.01 False
BitwiseOrBenchmark - Duration of single invocation 15.54 ns 12.92 ns 0.83 0.02 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.34 ns 12.87 ns 0.90 0.02 False
DivideBenchmark - Duration of single invocation 48.96 ns 38.81 ns 0.79 0.01 False
GreaterThanOrEqualBenchmark - Duration of single invocation 42.79 ns 32.22 ns 0.75 0.01 False
GreaterThanBenchmark - Duration of single invocation 40.80 ns 31.06 ns 0.76 0.01 False
MaxBenchmark - Duration of single invocation 45.89 ns 37.34 ns 0.81 0.07 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.32 ns 12.97 ns 0.91 0.03 False
AddBenchmark - Duration of single invocation 45.29 ns 36.06 ns 0.80 0.01 False
LessThanOrEqualBenchmark - Duration of single invocation 43.09 ns 32.32 ns 0.75 0.02 False
LessThanBenchmark - Duration of single invocation 42.00 ns 30.79 ns 0.73 0.02 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.EqualsAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.23626028912932 < 12.513328868991342.
IsChangePoint: Marked as a change because one of 1/5/2023 11:12:07 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 55.78522942262663 (T) = (0 -10.424144237456796) / Math.Sqrt((0.1675226966670457 / (299)) + (0.03613219652736679 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.19564513014846144 = (12.959633400840604 - 10.424144237456796) / 12.959633400840604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.56483142435748 < 43.45247719267945.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.847105013599442 (T) = (0 -36.9425194462766) / Math.Sqrt((17.94438388957015 / (299)) + (0.22076053126207856 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14517169250550865 = (43.21630334699072 - 36.9425194462766) / 43.21630334699072 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.853200709738061 < 13.712031887366694.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 45.213675526762735 (T) = (0 -12.976032365232488) / Math.Sqrt((0.2553762578800049 / (299)) + (0.013904385431979057 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.11571091074846948 = (14.673970902677887 - 12.976032365232488) / 14.673970902677887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.12036596422035 < 39.133268129467425.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.535163124718256 (T) = (0 -36.143708983082135) / Math.Sqrt((4.4741764470451955 / (299)) + (0.0660497042131534 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0734772355175883 = (39.010060376954996 - 36.143708983082135) / 39.010060376954996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.574969873348063 < 10.79057397679593.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 11.848576662337798 (T) = (0 -10.485816587531984) / Math.Sqrt((0.060735215993539725 / (299)) + (0.14566642745004224 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08214141703033066 = (11.4242180463202 - 10.485816587531984) / 11.4242180463202 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.11858019744989 < 42.96582496372756.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.18399381050762 (T) = (0 -36.00039692771498) / Math.Sqrt((14.968546307408896 / (299)) + (0.11483868230806978 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.15493866466430314 = (42.60092779350032 - 36.00039692771498) / 42.60092779350032 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.854791946860198 < 14.83431974088317.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 87.68398548048674 (T) = (0 -12.975110218269485) / Math.Sqrt((0.17782961957447524 / (299)) + (0.010686677779095233 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.17893959863643108 = (15.802869309884125 - 12.975110218269485) / 15.802869309884125 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.793666396238475 < 39.46414813837177.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 53.94821359179785 (T) = (0 -31.098118882968425) / Math.Sqrt((7.340165907398661 / (299)) + (0.037998972134772976 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.21901536806193736 = (39.81911757443483 - 31.098118882968425) / 39.81911757443483 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.98337291778979 < 39.70850172728708.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.543925004972216 (T) = (0 -37.01442622989482) / Math.Sqrt((3.5265590612075166 / (299)) + (0.14773496695509947 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07283785097744344 = (39.92227925710362 - 37.01442622989482) / 39.92227925710362 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.51435698273485 < 15.8206005258272.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 12/21/2022 10:31:08 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.499248276583124 (T) = (0 -15.424962849291676) / Math.Sqrt((0.08132144035122316 / (299)) + (0.02719174881140233 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07941323881669649 = (16.755577529123645 - 15.424962849291676) / 16.755577529123645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.12051083556053 < 37.74368823794786.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.43180054584577 (T) = (0 -36.14491772506084) / Math.Sqrt((4.020820219962396 / (299)) + (0.031230611892314752 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07008708965942859 = (38.86914282308773 - 36.14491772506084) / 38.86914282308773 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.38190494399278 < 18.895669691038528.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.61008828578133 (T) = (0 -18.559758321062876) / Math.Sqrt((0.16674819400275784 / (299)) + (0.09265772145517225 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.07968654143587464 = (20.166779208053573 - 18.559758321062876) / 20.166779208053573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.25106575646863 < 41.23611664586661.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 18.208348703989557 (T) = (0 -39.19990748669226) / Math.Sqrt((4.959048389777168 / (299)) + (0.15464362630373757 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06584762665950103 = (41.9630764802477 - 39.19990748669226) / 41.9630764802477 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.077373239467638 < 13.816385086479299.
IsChangePoint: Marked as a change because one of 11/23/2022 11:56:05 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 30.91543302999155 (T) = (0 -13.069687166861899) / Math.Sqrt((0.08822044055139293 / (299)) + (0.05550271902079228 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10777331532687072 = (14.648393050080125 - 13.069687166861899) / 14.648393050080125 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.964064505828174 < 14.727309947664653.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 54.20008633889348 (T) = (0 -13.018215990450072) / Math.Sqrt((0.19492607172600107 / (299)) + (0.047089833427820936 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.17550502781030264 = (15.789321256715777 - 13.018215990450072) / 15.789321256715777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.95510909432897 < 43.72181104567785.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.916225841928572 (T) = (0 -36.96222975135599) / Math.Sqrt((12.314843278857717 / (299)) + (0.11567754959382881 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.15626745633433892 = (43.80799345580619 - 36.96222975135599) / 43.80799345580619 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.923094925794624 < 14.738085595915162.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 54.4485375976603 (T) = (0 -13.033673967993979) / Math.Sqrt((0.28400018111465897 / (299)) + (0.041375211550123095 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.17764170962189263 = (15.849142789089292 - 13.033673967993979) / 15.849142789089292 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.869776486163925 < 13.532726554080273.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.94196709458574 (T) = (0 -12.97337171457874) / Math.Sqrt((0.22644086586476528 / (299)) + (0.057991079972418015 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11164501756534172 = (14.603814883801789 - 12.97337171457874) / 14.603814883801789 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.81293304455435 < 46.23510437017429.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.08051878137938 (T) = (0 -39.15435461960331) / Math.Sqrt((8.490857535476142 / (299)) + (0.09905605763052719 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.16235594451185925 = (46.74342802658098 - 39.15435461960331) / 46.74342802658098 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.22485246898976 < 41.26916102956506.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 56.50256161180563 (T) = (0 -32.28508347522379) / Math.Sqrt((7.580858495715419 / (299)) + (0.04915135734658959 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.22462948280418105 = (41.63826552495835 - 32.28508347522379) / 41.63826552495835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.064251552360922 < 39.4987085130925.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 53.13050372934453 (T) = (0 -31.101814890142204) / Math.Sqrt((7.50928944762558 / (299)) + (0.041261105096920055 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.21864849690949684 = (39.805151416647 - 31.101814890142204) / 39.805151416647 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.33793370915894 < 43.612032456422156.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.885771639812013 (T) = (0 -36.96989858435385) / Math.Sqrt((18.51510424585653 / (299)) + (0.2760710580253693 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14364689845157794 = (43.17132561031941 - 36.96989858435385) / 43.17132561031941 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.971993027726862 < 13.630176789057328.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.505218023899175 (T) = (0 -12.950645519916366) / Math.Sqrt((0.2825649303598841 / (299)) + (0.015491892225404712 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11574426351150037 = (14.64581453703105 - 12.950645519916366) / 14.64581453703105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.063081161737415 < 42.92197268755639.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.190572604441996 (T) = (0 -36.07383850872866) / Math.Sqrt((15.059053734915567 / (299)) + (0.0704398647862901 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.15284072373343494 = (42.58212064643409 - 36.07383850872866) / 42.58212064643409 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.32399959502847 < 40.86766108529057.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 65.56740990033805 (T) = (0 -32.30822189429666) / Math.Sqrt((6.091015242879856 / (299)) + (0.015392157610625715 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.2272035082114922 = (41.80689513680982 - 32.30822189429666) / 41.80689513680982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.79040070510712 < 39.723402686517645.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 51.6629104363251 (T) = (0 -30.99773550483512) / Math.Sqrt((7.221477092689791 / (299)) + (0.11138248687168535 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2204604823569457 = (39.76416179459008 - 30.99773550483512) / 39.76416179459008 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseOrBenchmark - Duration of single invocation 16.29 ns 7.98 ns 0.49 0.27 False
InequalityOperatorBenchmark - Duration of single invocation 2.61 ns 0.67 ns 0.26 0.47 False
GreaterThanAllBenchmark - Duration of single invocation 2.90 ns 0.80 ns 0.28 0.47 False
ConditionalSelectBenchmark - Duration of single invocation 61.63 ns 32.49 ns 0.53 0.34 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.54 ns 8.05 ns 0.59 0.05 False
LessThanAnyBenchmark - Duration of single invocation 2.22 ns 0.74 ns 0.33 0.48 False
DivideBenchmark - Duration of single invocation 50.52 ns 46.21 ns 0.91 0.00 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.86 ns 8.04 ns 0.58 0.03 False
SubtractBenchmark - Duration of single invocation 25.93 ns 23.87 ns 0.92 0.01 False
LessThanBenchmark - Duration of single invocation 28.26 ns 26.81 ns 0.95 0.01 False
MultiplyBenchmark - Duration of single invocation 27.18 ns 24.45 ns 0.90 0.02 False
OneBenchmark - Duration of single invocation 14.85 ns 0.79 ns 0.05 0.10 False
AndNotBenchmark - Duration of single invocation 23.79 ns 15.45 ns 0.65 0.02 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.44 ns 6.89 ns 0.82 0.06 False
BitwiseAndBenchmark - Duration of single invocation 16.13 ns 8.46 ns 0.52 0.31 False
EqualsBenchmark - Duration of single invocation 8.69 ns 6.32 ns 0.73 0.11 False
OnesComplementBenchmark - Duration of single invocation 8.44 ns 6.33 ns 0.75 0.07 False
AddBenchmark - Duration of single invocation 26.00 ns 23.55 ns 0.91 0.02 False
XorBenchmark - Duration of single invocation 16.15 ns 8.31 ns 0.51 0.30 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.49 ns 8.06 ns 0.60 0.06 False
EqualsAllBenchmark - Duration of single invocation 5.05 ns 1.32 ns 0.26 0.32 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int16>.BitwiseOrBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.975614705111861 < 15.381473657346275.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 133.6840697925631 (T) = (0 -8.148524951860294) / Math.Sqrt((0.4230547401569419 / (299)) + (0.040438754498323126 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.47737574124606585 = (15.591555147647371 - 8.148524951860294) / 15.591555147647371 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.6746869594598341 < 2.8699074482748816.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 44.18194057799579 (T) = (0 -0.7169879626100993) / Math.Sqrt((0.44619368828401357 / (299)) + (0.042439110053683525 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7786967387121851 = (3.2398436355514173 - 0.7169879626100993) / 3.2398436355514173 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7998293340189869 < 2.7819223011193315.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 6:03:21 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 33.856991702246184 (T) = (0 -0.6941404040826775) / Math.Sqrt((0.5910165325335842 / (299)) + (0.09554920269139892 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.7878740779848847 = (3.272303533149596 - 0.6941404040826775) / 3.272303533149596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.48728002079224 < 57.74403520669096.
IsChangePoint: Marked as a change because one of 1/6/2023 8:18:17 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 100.2283907162092 (T) = (0 -32.50148643956937) / Math.Sqrt((16.240741136108568 / (299)) + (0.06040195943917754 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4236889261813723 = (56.3957347274538 - 32.50148643956937) / 56.3957347274538 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.047447299551957 < 12.882939230395479.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 119.5946439921481 (T) = (0 -8.179625386868072) / Math.Sqrt((0.23285943697085848 / (299)) + (0.037672106711764075 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4147083113241892 = (13.975297351947727 - 8.179625386868072) / 13.975297351947727 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7386465387656171 < 2.568787310141865.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 41.46473262736544 (T) = (0 -0.663062811952377) / Math.Sqrt((0.4261824952598772 / (299)) + (0.059626537342799664 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.796343211068457 = (3.2557854586190995 - 0.663062811952377) / 3.2557854586190995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.20515404111119 < 47.95997716127778.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.606693657774485 (T) = (0 -46.70182755994617) / Math.Sqrt((0.08636223768392905 / (299)) + (0.828337718792891 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0726323660675592 = (50.3595616788028 - 46.70182755994617) / 50.3595616788028 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.044382495102846 < 12.952632311571938.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 124.16546943173232 (T) = (0 -8.146519723940424) / Math.Sqrt((0.18302746329219857 / (299)) + (0.038260596913172504 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.417221589703172 = (13.978760331548893 - 8.146519723940424) / 13.978760331548893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.867434457672612 < 24.72669758850469.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 62.26290238624635 (T) = (0 -23.661204329955748) / Math.Sqrt((0.07268021742477829 / (299)) + (0.027980756883069356 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0898940801276852 = (25.998297355625756 - 23.661204329955748) / 25.998297355625756 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.806668727988928 < 27.29124816468708.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 15.55040549990349 (T) = (0 -26.90341520655065) / Math.Sqrt((0.10301358095425728 / (299)) + (0.2645653091013325 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05805044333342041 = (28.56141819500172 - 26.90341520655065) / 28.56141819500172 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.45468822216279 < 25.739877684749956.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 6:03:21 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 71.05759928764243 (T) = (0 -24.62825090537998) / Math.Sqrt((0.1208117433236114 / (299)) + (0.014883185198733941 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.08358608361502709 = (26.874592872326033 - 24.62825090537998) / 26.874592872326033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.788277010612888 < 14.076551211139167.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 133.81229320250446 (T) = (0 -0.9112993508963304) / Math.Sqrt((3.604301825454271 / (299)) + (0.03394040119172041 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.9445710363094912 = (16.440851320703562 - 0.9112993508963304) / 16.440851320703562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.450322993392467 < 22.746742871126823.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 139.03303721290337 (T) = (0 -15.593836205991318) / Math.Sqrt((0.11776907592882543 / (299)) + (0.07751943610479921 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3492647006393922 = (23.963409117829222 - 15.593836205991318) / 23.963409117829222 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.89117241226326 < 8.029716888593414.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 41.7986476763424 (T) = (0 -6.4442436982745095) / Math.Sqrt((0.4278354776069753 / (299)) + (0.0488348518143423 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.27395859647136145 = (8.875862543037902 - 6.4442436982745095) / 8.875862543037902 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.455442324015452 < 15.387285837775012.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 129.7363556683936 (T) = (0 -8.230558101075767) / Math.Sqrt((0.4054769293655361 / (299)) + (0.043149175677510095 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.46956453448888397 = (15.51660595157411 - 8.230558101075767) / 15.51660595157411 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.322593368295156 < 7.959915203027537.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 20.00598536618413 (T) = (0 -6.443764664576354) / Math.Sqrt((0.7290403991987429 / (299)) + (0.0623320358235669 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18053757939170575 = (7.863404718172543 - 6.443764664576354) / 7.863404718172543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.33290847310497 < 8.02593721697195.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.5658859487894 (T) = (0 -6.531484262983754) / Math.Sqrt((0.5178611635172555 / (299)) + (0.13042004702687135 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.2671725303738987 = (8.912717568183146 - 6.531484262983754) / 8.912717568183146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.549105133097417 < 24.697783611640226.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 36.37327124344134 (T) = (0 -23.627334545820574) / Math.Sqrt((0.07019346545573135 / (299)) + (0.09604187724504178 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.0894934050112391 = (25.94965777937306 - 23.627334545820574) / 25.94965777937306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.312436559203897 < 14.733399260485832.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 163.18549575290314 (T) = (0 -8.102552599707836) / Math.Sqrt((0.3900295973272973 / (299)) + (0.01959214221276877 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4811899267801504 = (15.61756993155821 - 8.102552599707836) / 15.61756993155821 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.05521213000389 < 12.821663118186153.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 92.96049472599167 (T) = (0 -8.155884870230613) / Math.Sqrt((0.12820392966608593 / (299)) + (0.0790574462630496 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.41018518247143354 = (13.827873813691701 - 8.155884870230613) / 13.827873813691701 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3161438493733486 < 4.8048383775376635.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 74.82705325269079 (T) = (0 -0.8831577344109268) / Math.Sqrt((0.09777690541741021 / (299)) + (0.06006909853241418 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.8184190033776285 = (4.863712342363685 - 0.8831577344109268) / 4.863712342363685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseAndOperatorBenchmark - Duration of single invocation 10.50 ns 8.03 ns 0.77 0.07 False
BitwiseAndBenchmark - Duration of single invocation 11.94 ns 8.34 ns 0.70 0.05 False
DivisionOperatorBenchmark - Duration of single invocation 9.20 ns 7.31 ns 0.79 0.04 False
OneBenchmark - Duration of single invocation 8.63 ns 0.79 ns 0.09 0.21 False
UnaryNegateOperatorBenchmark - Duration of single invocation 7.82 ns 5.47 ns 0.70 0.05 False
SubtractBenchmark - Duration of single invocation 9.74 ns 5.89 ns 0.60 0.05 False
AddOperatorBenchmark - Duration of single invocation 8.27 ns 5.95 ns 0.72 0.05 False
GreaterThanOrEqualBenchmark - Duration of single invocation 9.66 ns 7.22 ns 0.75 0.02 False
AndNotBenchmark - Duration of single invocation 18.39 ns 15.90 ns 0.86 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 1.92 ns 0.34 ns 0.18 0.41 False
LessThanAnyBenchmark - Duration of single invocation 1.22 ns 0.00 ns 0.00 0.44 False
XorBenchmark - Duration of single invocation 11.95 ns 8.01 ns 0.67 0.05 False
EqualsAnyBenchmark - Duration of single invocation 4.52 ns 1.37 ns 0.30 0.32 False
MaxBenchmark - Duration of single invocation 9.77 ns 7.24 ns 0.74 0.04 False
LessThanBenchmark - Duration of single invocation 9.55 ns 7.44 ns 0.78 0.06 False
EqualsBenchmark - Duration of single invocation 9.49 ns 6.43 ns 0.68 0.13 False
NegateBenchmark - Duration of single invocation 8.70 ns 5.46 ns 0.63 0.03 False
LessThanOrEqualAllBenchmark - Duration of single invocation 4.13 ns 2.41 ns 0.58 0.37 False
MultiplyBenchmark - Duration of single invocation 10.48 ns 6.31 ns 0.60 0.04 False
GreaterThanAnyBenchmark - Duration of single invocation 3.20 ns 0.42 ns 0.13 0.38 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 4.04 ns 2.44 ns 0.60 0.37 False
EqualsStaticBenchmark - Duration of single invocation 9.33 ns 6.78 ns 0.73 0.04 False
EqualityOperatorBenchmark - Duration of single invocation 2.37 ns 0.33 ns 0.14 0.41 False
ConditionalSelectBenchmark - Duration of single invocation 35.45 ns 32.67 ns 0.92 0.02 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 10.44 ns 8.37 ns 0.80 0.12 False
SubtractionOperatorBenchmark - Duration of single invocation 8.27 ns 5.86 ns 0.71 0.06 False
AddBenchmark - Duration of single invocation 9.99 ns 5.93 ns 0.59 0.05 False
GreaterThanBenchmark - Duration of single invocation 9.34 ns 7.43 ns 0.80 0.05 False
EqualsAllBenchmark - Duration of single invocation 5.48 ns 1.90 ns 0.35 0.23 False
LessThanAllBenchmark - Duration of single invocation 2.81 ns 1.39 ns 0.49 0.34 False
BitwiseOrOperatorBenchmark - Duration of single invocation 10.64 ns 8.06 ns 0.76 0.05 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.26 ns 0.75 ns 0.23 0.42 False
DivideBenchmark - Duration of single invocation 10.92 ns 7.31 ns 0.67 0.04 False
MultiplyOperatorBenchmark - Duration of single invocation 8.27 ns 6.29 ns 0.76 0.02 False
MinBenchmark - Duration of single invocation 9.53 ns 6.98 ns 0.73 0.04 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 2.38 ns 0.83 ns 0.35 0.46 False
LessThanOrEqualBenchmark - Duration of single invocation 9.93 ns 7.78 ns 0.78 0.06 False
BitwiseOrBenchmark - Duration of single invocation 11.97 ns 8.30 ns 0.69 0.07 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.BitwiseAndOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.033123262484102 < 10.002904763720377.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 44.19549494132297 (T) = (0 -8.082337164203238) / Math.Sqrt((0.27555243752565617 / (299)) + (0.07315403693237703 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2562416920879818 = (10.866886565466542 - 8.082337164203238) / 10.866886565466542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.341476130944695 < 11.34209442738129.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 72.89147872972369 (T) = (0 -8.266346599567663) / Math.Sqrt((0.41508795757883604 / (299)) + (0.040799634313541305 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3288704989364857 = (12.317066358233824 - 8.266346599567663) / 12.317066358233824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.309281364129408 < 8.734983769286243.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.41449014064411 (T) = (0 -7.162331677041711) / Math.Sqrt((0.5076046014953524 / (299)) + (0.04062319911586415 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.26086845336055703 = (9.690198868667123 - 7.162331677041711) / 9.690198868667123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7938173613729951 < 7.99861617337446.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 124.99016318298038 (T) = (0 -0.9401934525368798) / Math.Sqrt((0.7569403225079809 / (299)) + (0.040332171822302205 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.8961355810844098 = (9.052122587822572 - 0.9401934525368798) / 9.052122587822572 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.465629062692135 < 7.465640523929218.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 59.506005381474026 (T) = (0 -5.748245796831608) / Math.Sqrt((0.3863807716469756 / (299)) + (0.011333180259359366 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3030590528781724 = (8.247823320713563 - 5.748245796831608) / 8.247823320713563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.890994325418623 < 9.334113458131668.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 134.87859869009068 (T) = (0 -5.932696077526309) / Math.Sqrt((0.16566251687642305 / (299)) + (0.009354207777263044 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4112272190923796 = (10.076376269264323 - 5.932696077526309) / 10.076376269264323 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.950082820569274 < 7.773768910907869.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 55.87018863013725 (T) = (0 -5.902567081198733) / Math.Sqrt((0.6087087445572441 / (299)) + (0.011059984456892403 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.32109164620778147 = (8.694203051455201 - 5.902567081198733) / 8.694203051455201 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.223819266485935 < 9.197493356329067.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 78.68859910441707 (T) = (0 -7.0830046471099815) / Math.Sqrt((0.117714823955095 / (299)) + (0.021967042623731833 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2867031711998483 = (9.929953928190624 - 7.0830046471099815) / 9.929953928190624 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.904982967844264 < 17.52366814516864.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 49.61898774061229 (T) = (0 -15.66201293975055) / Math.Sqrt((0.11546116602929093 / (299)) + (0.0598620270693522 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1431460634911241 = (18.278509641401772 - 15.66201293975055) / 18.278509641401772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.34261138118670686 < 1.604633042294716.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.533373457694037 (T) = (0 -0.7380737956925442) / Math.Sqrt((0.8294090761150781 / (299)) + (0.1392279396008675 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7102070933331225 = (2.5469008340531065 - 0.7380737956925442) / 2.5469008340531065 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 1.8482987338314307.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.06591548095824 (T) = (0 -0.653109480738018) / Math.Sqrt((0.7678279464887773 / (299)) + (0.08486546756293623 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7499072880162131 = (2.611469464893315 - 0.653109480738018) / 2.611469464893315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.014638866594508 < 11.24249536695114.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 70.46646611728006 (T) = (0 -8.11103894066358) / Math.Sqrt((0.27348335808292235 / (299)) + (0.05951881025448579 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.33606667440859633 = (12.216646804765539 - 8.11103894066358) / 12.216646804765539 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3673400389293546 < 3.0419482620645018.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.832065827866803 (T) = (0 -1.7110355588187416) / Math.Sqrt((0.5138662201134597 / (299)) + (0.11408435626608283 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.548445756108423 = (3.789213769917706 - 1.7110355588187416) / 3.789213769917706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.239556759254881 < 9.23214221042866.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.13210853881257 (T) = (0 -7.171922855366913) / Math.Sqrt((0.33870737448997057 / (299)) + (0.08788754099598434 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.28915793439883686 = (10.08933376684957 - 7.171922855366913) / 10.08933376684957 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.4360846509105 < 9.077217724649119.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 26.72813413341805 (T) = (0 -7.472552494164842) / Math.Sqrt((0.4453641457686282 / (299)) + (0.1794292559672627 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.25299678197120556 = (10.00337389962462 - 7.472552494164842) / 10.00337389962462 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.434742785808421 < 8.650573608548246.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.865927215322362 (T) = (0 -5.653846810613441) / Math.Sqrt((2.3104012891595853 / (299)) + (0.16699732475114293 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2986397465612505 = (8.0612592214811 - 5.653846810613441) / 8.0612592214811 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.463873425658261 < 8.272525694546392.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 77.86349484485666 (T) = (0 -5.764019659403321) / Math.Sqrt((0.4848372930044939 / (299)) + (0.008235212152144037 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.37451509839956065 = (9.215281847179398 - 5.764019659403321) / 9.215281847179398 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4100665198600457 < 3.7855741818021644.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.049462481499713 (T) = (0 -2.2034114614011844) / Math.Sqrt((0.5831808485252395 / (299)) + (0.16187515720634324 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4827047520508043 = (4.2594852168786685 - 2.2034114614011844) / 4.2594852168786685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.3069490855372665 < 9.794191455770484.
IsChangePoint: Marked as a change because one of 12/16/2022 5:41:28 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 79.02031706104579 (T) = (0 -6.181205156231358) / Math.Sqrt((0.12429014765046249 / (299)) + (0.061616040777076546 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.41114897449026194 = (10.497061036584943 - 6.181205156231358) / 10.497061036584943 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.41701287230854667 < 3.0643329199972036.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.601307624212698 (T) = (0 -1.5909608417039713) / Math.Sqrt((0.5324074915274652 / (299)) + (0.14720458664271058 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.5790586034994525 = (3.7795304879260123 - 1.5909608417039713) / 3.7795304879260123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.442155888554418 < 3.84732683598016.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.668008154543852 (T) = (0 -2.1615106723929594) / Math.Sqrt((0.6002245824451106 / (299)) + (0.12056318097435727 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4889967689616065 = (4.229935431133423 - 2.1615106723929594) / 4.229935431133423 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.7777646960102995 < 8.85524433077883.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 37.841745353920466 (T) = (0 -6.726735078542476) / Math.Sqrt((0.38631063881498184 / (299)) + (0.11485465990154115 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.3084412154185524 = (9.726917260712263 - 6.726735078542476) / 9.726917260712263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.32794395888546507 < 1.9395792749468668.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.673258856318395 (T) = (0 -0.7924031797719882) / Math.Sqrt((0.7964901494324819 / (299)) + (0.13729545761529222 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.6945036178357533 = (2.5938218127439607 - 0.7924031797719882) / 2.5938218127439607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.66521645626992 < 33.971706092838076.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 86.15093362127696 (T) = (0 -32.513157073054806) / Math.Sqrt((0.34530419825020814 / (299)) + (0.011904455185029942 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09719681042738067 = (36.01356026272604 - 32.513157073054806) / 36.01356026272604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.372297185333126 < 9.980584093535695.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 50.35243635601903 (T) = (0 -8.228533238332107) / Math.Sqrt((0.22037716744931699 / (299)) + (0.04623106146874241 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.24000581657840203 = (10.82710028290759 - 8.228533238332107) / 10.82710028290759 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.864575644732641 < 7.7740432239006365.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 44.255949916597544 (T) = (0 -5.880679304878018) / Math.Sqrt((0.5881040705480494 / (299)) + (0.05281767247767639 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.3246333398209527 = (8.707387633436012 - 5.880679304878018) / 8.707387633436012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.932706182532704 < 9.321059592024827.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 128.6609101264588 (T) = (0 -5.944718266941284) / Math.Sqrt((0.16374214114411004 / (299)) + (0.011432472382142839 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4091817010340206 = (10.061838432129527 - 5.944718266941284) / 10.061838432129527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.434401792236276 < 8.833114109290204.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.40819343640727 (T) = (0 -6.745731534286307) / Math.Sqrt((0.1695206632228372 / (299)) + (0.2070199332130792 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.294774998778627 = (9.56536073253704 - 6.745731534286307) / 9.56536073253704 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9033480010197608 < 5.157602661155761.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 50.37053258080928 (T) = (0 -0.8448148476063614) / Math.Sqrt((0.21290238857334912 / (299)) + (0.14846647497526574 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8346458401064256 = (5.109123641945887 - 0.8448148476063614) / 5.109123641945887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3902558700039302 < 3.0985893890291494.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.239393623278836 (T) = (0 -1.6478166942287726) / Math.Sqrt((0.5332142717746254 / (299)) + (0.07287853321632677 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.5682539565066733 = (3.816634151169105 - 1.6478166942287726) / 3.816634151169105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.06262537967208 < 9.98084917268464.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 47.23152385855591 (T) = (0 -8.235760938471735) / Math.Sqrt((0.1527316816678601 / (299)) + (0.0612321763365268 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.24090150554067305 = (10.849397013147433 - 8.235760938471735) / 10.849397013147433 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7490782041629043 < 2.9553320611311475.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 41.0949612757981 (T) = (0 -0.9236915155005793) / Math.Sqrt((0.3844257502322514 / (299)) + (0.05364851802775363 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7252721526934827 = (3.3622056320705083 - 0.9236915155005793) / 3.3622056320705083 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.313189841996788 < 10.376535884031467.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 60.15291000202671 (T) = (0 -7.113711261193406) / Math.Sqrt((0.16789040946331804 / (299)) + (0.10346904742736154 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3711734261433017 = (11.312675953822733 - 7.113711261193406) / 11.312675953822733 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.289233870444317 < 7.930118683318568.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 47.305156279401245 (T) = (0 -6.294398370333465) / Math.Sqrt((0.751175459061216 / (299)) + (0.015480181238231966 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.29690917349984236 = (8.952468348457472 - 6.294398370333465) / 8.952468348457472 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.979320516877232 < 9.069738810548007.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 37.215679791591135 (T) = (0 -7.191502607865623) / Math.Sqrt((0.20330818153475544 / (299)) + (0.10763592142521099 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.27108685202307875 = (9.866062407881438 - 7.191502607865623) / 9.866062407881438 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8324421741824044 < 2.9807609294239588.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 49.59956736655704 (T) = (0 -0.8335578894611956) / Math.Sqrt((0.427598827062902 / (299)) + (0.030249664151508628 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7552885226495312 = (3.406288493234004 - 0.8335578894611956) / 3.406288493234004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.775462684949306 < 9.436302212498665.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 48.102796820971804 (T) = (0 -7.672450411178751) / Math.Sqrt((0.28296419807609796 / (299)) + (0.047926719027741364 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.25380795854304844 = (10.282139161117522 - 7.672450411178751) / 10.282139161117522 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.301709907975505 < 11.331788661395604.
IsChangePoint: Marked as a change because one of 1/4/2023 2:39:28 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 79.51642313414632 (T) = (0 -8.236796646852303) / Math.Sqrt((0.19989198719694 / (299)) + (0.041988261325655495 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.32190177701133577 = (12.146907877960915 - 8.236796646852303) / 12.146907877960915 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementBenchmark - Duration of single invocation 8.51 ns 6.31 ns 0.74 0.09 False
ConditionalSelectBenchmark - Duration of single invocation 54.29 ns 32.50 ns 0.60 0.30 False
LessThanAnyBenchmark - Duration of single invocation 8.14 ns 6.75 ns 0.83 0.12 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.69 ns 6.87 ns 0.79 0.06 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.82 ns 8.04 ns 0.58 0.05 False
XorBenchmark - Duration of single invocation 16.12 ns 8.79 ns 0.55 0.28 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.87 ns 8.49 ns 0.61 0.01 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.44 ns 8.01 ns 0.60 0.02 False
InequalityOperatorBenchmark - Duration of single invocation 8.23 ns 6.70 ns 0.82 0.13 False
BitwiseOrBenchmark - Duration of single invocation 16.09 ns 7.97 ns 0.50 0.27 False
OneBenchmark - Duration of single invocation 10.42 ns 0.80 ns 0.08 0.26 False
EqualsAllBenchmark - Duration of single invocation 9.77 ns 6.96 ns 0.71 0.10 False
AndNotBenchmark - Duration of single invocation 24.00 ns 15.37 ns 0.64 0.02 False
BitwiseAndBenchmark - Duration of single invocation 16.04 ns 8.44 ns 0.53 0.27 False
GreaterThanAllBenchmark - Duration of single invocation 7.80 ns 6.53 ns 0.84 0.13 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 9.29 ns 7.30 ns 0.79 0.11 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Single>.OnesComplementBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.314891339923418 < 7.9832059428590325.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 38.938060045628596 (T) = (0 -6.347995414243946) / Math.Sqrt((0.35502716220192165 / (299)) + (0.06861390874126039 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.28066936707622336 = (8.824864566718107 - 6.347995414243946) / 8.824864566718107 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.50484882663788 < 51.65325934934316.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 103.8360841834936 (T) = (0 -32.532253447806106) / Math.Sqrt((16.09154517439527 / (299)) + (0.013310855181302402 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4266904738702455 = (56.74465880136663 - 32.532253447806106) / 56.74465880136663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.754901864653994 < 7.979852365669388.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 18.99810206005276 (T) = (0 -6.615231736370046) / Math.Sqrt((0.09942608332132927 / (299)) + (0.14561037805708046 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.18390324783947146 = (8.105940525871388 - 6.615231736370046) / 8.105940525871388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.868653091929514 < 8.028674721188853.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 36.74050906405292 (T) = (0 -6.571153056727161) / Math.Sqrt((0.36307834559276775 / (299)) + (0.060376102582184246 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.25455098022527217 = (8.81502675892302 - 6.571153056727161) / 8.81502675892302 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.040511152954503 < 13.128468238690905.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 91.3928892184948 (T) = (0 -8.2201905905265) / Math.Sqrt((0.0940843465132993 / (299)) + (0.08337835375776828 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.40629965387452516 = (13.84568940235924 - 8.2201905905265) / 13.84568940235924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.790665986994304 < 14.649587800269455.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 106.03438065183423 (T) = (0 -8.219577716507661) / Math.Sqrt((0.4297427022791482 / (299)) + (0.07690339478334138 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.4714512168363586 = (15.551218692263715 - 8.219577716507661) / 15.551218692263715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.492941785523048 < 13.132705380048503.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 137.91768551975363 (T) = (0 -8.281009365162546) / Math.Sqrt((0.20224265340873168 / (299)) + (0.024851168071310464 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4079625297865582 = (13.9873061787408 - 8.281009365162546) / 13.9873061787408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.01032496564664 < 12.823842384220072.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 128.6496077413783 (T) = (0 -8.184166791249151) / Math.Sqrt((0.08231905090871838 / (299)) + (0.03957526538122935 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4081279870705297 = (13.827595514681663 - 8.184166791249151) / 13.827595514681663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.703823298925156 < 7.82071877319618.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 12.9024982553509 (T) = (0 -6.432847018671695) / Math.Sqrt((0.11384422680167142 / (299)) + (0.39500380569838833 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2065221585887684 = (8.107153953071485 - 6.432847018671695) / 8.107153953071485 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.974936081815275 < 14.673882072721835.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 87.28349060330883 (T) = (0 -8.116807661925982) / Math.Sqrt((0.41573937343059036 / (299)) + (0.14061662783162684 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.47796682198545937 = (15.548451714882953 - 8.116807661925982) / 15.548451714882953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7960797826778532 < 11.514242293737345.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 150.62482792981308 (T) = (0 -0.8118156041413546) / Math.Sqrt((0.6497464573863243 / (299)) + (0.09236787894027795 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.9350561540950233 = (12.500269930566974 - 0.8118156041413546) / 12.500269930566974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.961306679327537 < 8.942146982070945.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 38.76983682512765 (T) = (0 -6.879790459493194) / Math.Sqrt((0.08513396628174504 / (299)) + (0.10500104366055203 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.27781217768959676 = (9.526317457809741 - 6.879790459493194) / 9.526317457809741 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.368619980064004 < 22.89079362860504.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 117.22298771829581 (T) = (0 -15.505441290286592) / Math.Sqrt((0.15597478585812186 / (299)) + (0.11152610694460129 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3521294478734707 = (23.932931106981965 - 15.505441290286592) / 23.932931106981965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.437785748156237 < 14.713960698406641.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 95.78550216672323 (T) = (0 -8.188926823731373) / Math.Sqrt((0.43855448799128227 / (299)) + (0.10741760962889707 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.47415140919207993 = (15.572784575023407 - 8.188926823731373) / 15.572784575023407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.5283701795442015 < 7.825561140025077.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 17.214339038885267 (T) = (0 -6.6514318582574825) / Math.Sqrt((0.10321723482297289 / (299)) + (0.1668280916450077 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18104581622643606 = (8.121860770781 - 6.6514318582574825) / 8.121860770781 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.3000044972356735 < 8.810305421961523.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 32.90863318000148 (T) = (0 -7.323047394628404) / Math.Sqrt((0.2676831106728824 / (299)) + (0.04041526391161931 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18581688490816645 = (8.994349377783916 - 7.323047394628404) / 8.994349377783916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DotBenchmark - Duration of single invocation 6.14 ns 3.64 ns 0.59 0.35 False
AddBenchmark - Duration of single invocation 9.67 ns 5.78 ns 0.60 0.05 False
SubtractBenchmark - Duration of single invocation 9.69 ns 5.76 ns 0.59 0.02 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.80 ns 8.03 ns 0.58 0.01 False
LessThanOrEqualBenchmark - Duration of single invocation 8.53 ns 7.32 ns 0.86 0.04 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 2.23 ns 0.59 ns 0.27 0.43 False
UnaryNegateOperatorBenchmark - Duration of single invocation 7.82 ns 5.81 ns 0.74 0.04 False
MinBenchmark - Duration of single invocation 9.52 ns 7.11 ns 0.75 0.04 False
OnesComplementBenchmark - Duration of single invocation 8.37 ns 6.97 ns 0.83 0.10 False
EqualsStaticBenchmark - Duration of single invocation 9.32 ns 6.82 ns 0.73 0.06 False
OneBenchmark - Duration of single invocation 7.06 ns 0.81 ns 0.12 0.16 False
LessThanAnyBenchmark - Duration of single invocation 1.97 ns 0.60 ns 0.31 0.37 False
GreaterThanOrEqualBenchmark - Duration of single invocation 9.65 ns 7.35 ns 0.76 0.05 False
EqualsAnyBenchmark - Duration of single invocation 3.43 ns 1.42 ns 0.41 0.38 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.70 ns 0.92 ns 0.25 0.38 False
MultiplyOperatorBenchmark - Duration of single invocation 8.33 ns 6.26 ns 0.75 0.03 False
ConditionalSelectBenchmark - Duration of single invocation 54.45 ns 32.47 ns 0.60 0.28 False
LessThanBenchmark - Duration of single invocation 9.54 ns 7.43 ns 0.78 0.05 False
MultiplyBenchmark - Duration of single invocation 10.39 ns 6.30 ns 0.61 0.04 False
EqualsBenchmark - Duration of single invocation 9.39 ns 5.83 ns 0.62 0.10 False
AndNotBenchmark - Duration of single invocation 24.04 ns 15.36 ns 0.64 0.01 False
SubtractionOperatorBenchmark - Duration of single invocation 8.14 ns 5.96 ns 0.73 0.04 False
DivisionOperatorBenchmark - Duration of single invocation 14.39 ns 12.00 ns 0.83 0.04 False
GreaterThanBenchmark - Duration of single invocation 8.88 ns 6.79 ns 0.77 0.05 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 4.10 ns 2.43 ns 0.59 0.36 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.86 ns 8.06 ns 0.58 0.04 False
BitwiseOrBenchmark - Duration of single invocation 16.14 ns 8.32 ns 0.52 0.26 False
XorBenchmark - Duration of single invocation 14.76 ns 8.02 ns 0.54 0.29 False
AddOperatorBenchmark - Duration of single invocation 8.24 ns 5.95 ns 0.72 0.04 False
LessThanAllBenchmark - Duration of single invocation 3.59 ns 1.51 ns 0.42 0.38 False
NegateBenchmark - Duration of single invocation 8.67 ns 5.80 ns 0.67 0.03 False
MaxBenchmark - Duration of single invocation 9.70 ns 7.25 ns 0.75 0.03 False
DivideBenchmark - Duration of single invocation 16.11 ns 12.72 ns 0.79 0.04 False
LessThanOrEqualAllBenchmark - Duration of single invocation 4.06 ns 2.33 ns 0.57 0.38 False
GreaterThanAllBenchmark - Duration of single invocation 1.83 ns 0.29 ns 0.16 0.43 False
EqualityOperatorBenchmark - Duration of single invocation 1.98 ns 0.73 ns 0.37 0.46 False
EqualsAllBenchmark - Duration of single invocation 4.44 ns 0.58 ns 0.13 0.19 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.84 ns 8.44 ns 0.61 0.03 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.46 ns 6.37 ns 0.75 0.08 False
GreaterThanAnyBenchmark - Duration of single invocation 3.32 ns 1.41 ns 0.42 0.39 False
BitwiseAndBenchmark - Duration of single invocation 16.16 ns 8.01 ns 0.50 0.31 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int64>.DotBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.635157955411192 < 5.834656292701599.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 3.8697098971469877 (T) = (0 -4.351087366929202) / Math.Sqrt((0.7518803671235871 / (299)) + (0.6611370743308467 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.13360046071324208 = (5.02203333407948 - 4.351087366929202) / 5.02203333407948 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.781704337229469 < 9.23571687061564.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 92.33399213322424 (T) = (0 -5.859702767781172) / Math.Sqrt((0.23021786917632905 / (299)) + (0.03232036786732171 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.42027395872434353 = (10.107710108876956 - 5.859702767781172) / 10.107710108876956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.755376267011895 < 9.262323434445413.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 110.2372911817632 (T) = (0 -5.873927259147681) / Math.Sqrt((0.36270342855500864 / (299)) + (0.008619465430727797 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4266510365604925 = (10.2449426679175 - 5.873927259147681) / 10.2449426679175 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.02889883307776 < 13.114081394133336.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 126.5435050164749 (T) = (0 -8.218637493239669) / Math.Sqrt((0.08137210996318067 / (299)) + (0.04050650468193461 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.40534461206300254 = (13.820840876851546 - 8.218637493239669) / 13.820840876851546 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.32415035025422 < 9.40991207094615.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 52.285737090645135 (T) = (0 -7.670991080471882) / Math.Sqrt((0.4097392663099842 / (299)) + (0.03001138996252688 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2586777382727588 = (10.347714450931075 - 7.670991080471882) / 10.347714450931075 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.5929263117322662 < 2.5156169536388053.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 34.45153000026508 (T) = (0 -0.8615144504267548) / Math.Sqrt((0.401707419781512 / (299)) + (0.10457890379444322 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.7482933910989998 = (3.4226930082936384 - 0.8615144504267548) / 3.4226930082936384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.8067233391352335 < 7.422517081025561.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 50.5377246791283 (T) = (0 -5.751707038398162) / Math.Sqrt((0.6884232382926238 / (299)) + (0.010969879019324972 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.31580241145167604 = (8.406500015005426 - 5.751707038398162) / 8.406500015005426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.108074669271574 < 9.042115782430363.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 47.34001012923896 (T) = (0 -7.297995021769693) / Math.Sqrt((0.2099502652413178 / (299)) + (0.0535677412408487 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2600112302171044 = (9.862305104861042 - 7.297995021769693) / 9.862305104861042 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.966874378033887 < 7.995927736946581.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 36.66644605988901 (T) = (0 -6.500914513865105) / Math.Sqrt((0.5481865937099304 / (299)) + (0.06134589523093372 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.26970154251512707 = (8.901722915113458 - 6.500914513865105) / 8.901722915113458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.8203711045983395 < 8.851928138741025.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 38.40598642364579 (T) = (0 -6.717515453145698) / Math.Sqrt((0.1833211732888066 / (299)) + (0.11615233769404545 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2968551734776901 = (9.553530367804761 - 6.717515453145698) / 9.553530367804761 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8118742829307547 < 7.961664045089943.
IsChangePoint: Marked as a change because one of 12/15/2022 5:36:17 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 126.2204207866999 (T) = (0 -0.8763003496707896) / Math.Sqrt((0.9521099648760304 / (299)) + (0.024005554356157678 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.9030775769298233 = (9.041255077128053 - 0.8763003496707896) / 9.041255077128053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.6013627972759292 < 1.9560591901566513.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.019480632533437 (T) = (0 -0.6361564925264305) / Math.Sqrt((0.7272089063957143 / (299)) + (0.09147225647874865 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7565499409857237 = (2.6130882658324817 - 0.6361564925264305) / 2.6130882658324817 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.349385143502186 < 9.228133708433715.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 51.43164945471318 (T) = (0 -7.062144230244459) / Math.Sqrt((0.3831003281315527 / (299)) + (0.052631231537437116 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.30033886100683516 = (10.09366368469044 - 7.062144230244459) / 10.09366368469044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4159718907957368 < 3.145213587948496.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 30.157170320923612 (T) = (0 -1.745272537923416) / Math.Sqrt((0.5361392827859833 / (299)) + (0.06538856552191007 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.5373374350835519 = (3.772236334354377 - 1.745272537923416) / 3.772236334354377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.9195155573661916 < 2.7785806248761133.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.17812855595353 (T) = (0 -0.803507749344952) / Math.Sqrt((0.38480896278800736 / (299)) + (0.05893642584489896 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7625466253019408 = (3.3838548319924953 - 0.803507749344952) / 3.3838548319924953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.2592968554705015 < 7.89841395930053.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 59.75873929637383 (T) = (0 -6.27162957151898) / Math.Sqrt((0.3677647800422042 / (299)) + (0.012178602779510302 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2842663308240133 = (8.762518575854356 - 6.27162957151898) / 8.762518575854356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.47070296675458 < 57.01482648764404.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 100.00736209666886 (T) = (0 -32.52245850758798) / Math.Sqrt((16.627403900556736 / (299)) + (0.008507070242363513 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.42111305390264897 = (56.18101898279582 - 32.52245850758798) / 56.18101898279582 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.430526143589992 < 9.063818651944255.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 33.63930649229978 (T) = (0 -7.426002724085834) / Math.Sqrt((0.4646429195068487 / (299)) + (0.1044566261469317 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2582361996359452 = (10.01127679787175 - 7.426002724085834) / 10.01127679787175 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.296771988567952 < 9.709026775086222.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 139.731825622337 (T) = (0 -6.2432450534769695) / Math.Sqrt((0.1467880154574921 / (299)) + (0.01021355282803033 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.40389731463890044 = (10.473438900371695 - 6.2432450534769695) / 10.473438900371695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.827900578981072 < 8.514465931509097.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 27.145904001638222 (T) = (0 -5.659973704781876) / Math.Sqrt((1.4444678624436835 / (299)) + (0.08086501846478002 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.30280397590816793 = (8.118195613858472 - 5.659973704781876) / 8.118195613858472 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.359518004821515 < 22.89075384507881.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 145.63489369099221 (T) = (0 -15.593637123307373) / Math.Sqrt((0.08215528819878352 / (299)) + (0.0735599928740592 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.35053523017849825 = (24.009981523082633 - 15.593637123307373) / 24.009981523082633 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.9607578189284665 < 7.751510090039571.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 74.76626529508002 (T) = (0 -5.935680449478919) / Math.Sqrt((0.2900481255481697 / (299)) + (0.006096025801711308 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3058899270273913 = (8.551497349777195 - 5.935680449478919) / 8.551497349777195 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.998163933762152 < 13.673257492431388.
IsChangePoint: Marked as a change because one of 12/5/2022 11:02:29 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.387283916003526 (T) = (0 -12.309665816938255) / Math.Sqrt((0.047483323714288976 / (299)) + (0.14502226539290378 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.15112415057119374 = (14.501137975855025 - 12.309665816938255) / 14.501137975855025 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.7945744230605705 < 8.807155902277675.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 39.1563379252358 (T) = (0 -6.863675306995277) / Math.Sqrt((0.3730956133966779 / (299)) + (0.09743090862707868 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.29359150955450763 = (9.716297864238213 - 6.863675306995277) / 9.716297864238213 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4273786581887937 < 3.377114194728316.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 20.5812281873942 (T) = (0 -2.127010082324246) / Math.Sqrt((0.5991874523442926 / (299)) + (0.20673809117304615 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.499265646875242 = (4.247781421528115 - 2.127010082324246) / 4.247781421528115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.062932896532013 < 13.154882633197849.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 110.93594960520909 (T) = (0 -8.18744010337922) / Math.Sqrt((0.21472085121953885 / (299)) + (0.04780594187647265 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4136144007859991 = (13.96255316357321 - 8.18744010337922) / 13.96255316357321 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.318033354591899 < 14.659612177766078.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 131.89767534574685 (T) = (0 -8.179722280896062) / Math.Sqrt((0.4059001625504225 / (299)) + (0.03930966846207192 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.471725724645246 = (15.483855002031099 - 8.179722280896062) / 15.483855002031099 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.023137397848332 < 15.336472300034142.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 126.27986560846887 (T) = (0 -8.188936180118203) / Math.Sqrt((0.4210547868111211 / (299)) + (0.046754918500685194 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.47184562736245095 = (15.504815645515706 - 8.188936180118203) / 15.504815645515706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.94953667321211 < 7.770127286157613.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 50.300414209735756 (T) = (0 -5.878484304161169) / Math.Sqrt((0.28355653919928275 / (299)) + (0.04471082333046559 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.31209573968648713 = (8.545497743366273 - 5.878484304161169) / 8.545497743366273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5098012345875236 < 3.397919285204285.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.15605194937944 (T) = (0 -1.7318714329460805) / Math.Sqrt((0.5044753151099182 / (299)) + (0.09347130960471366 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.5484578991480679 = (3.8354594835753506 - 1.7318714329460805) / 3.8354594835753506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.804032576959528 < 8.280308126094384.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 78.11032322292697 (T) = (0 -5.699063992551445) / Math.Sqrt((0.18700138865529672 / (299)) + (0.029134213516031654 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3702016787993937 = (9.049030142994225 - 5.699063992551445) / 9.049030142994225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.245024126545166 < 9.229414419008602.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 67.14097185471368 (T) = (0 -7.29899732848392) / Math.Sqrt((0.3613027553938422 / (299)) + (0.014414953201585168 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.2798673355052905 = (10.13562873669306 - 7.29899732848392) / 10.13562873669306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.719860013233308 < 15.32407332632528.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 51.07454735523578 (T) = (0 -12.40442836672697) / Math.Sqrt((0.0186389667965117 / (299)) + (0.1294506390403984 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.23320949463056118 = (16.177076111226143 - 12.40442836672697) / 16.177076111226143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.330725107191765 < 3.519464942859899.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.428872859115646 (T) = (0 -2.118798747326496) / Math.Sqrt((0.543242283369992 / (299)) + (0.14479223308225267 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.505321678651986 = (4.28318496260904 - 2.118798747326496) / 4.28318496260904 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.2906289487621544 < 1.8720143995447893.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.70616997814998 (T) = (0 -0.7316212338996716) / Math.Sqrt((0.8037866757880825 / (299)) + (0.09311472033466509 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7155223908061085 = (2.571806041160239 - 0.7316212338996716) / 2.571806041160239 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7341158482467719 < 1.856013335293957.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 6:03:21 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.39500385842856 (T) = (0 -0.7339626405158216) / Math.Sqrt((0.8449395924386028 / (299)) + (0.08690948787196001 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.7145680345825542 = (2.571410106231086 - 0.7339626405158216) / 2.571410106231086 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.5831857083674749 < 5.1668778419051975.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 70.08776715170141 (T) = (0 -0.7555243181509171) / Math.Sqrt((0.19544370587286236 / (299)) + (0.07713806971505408 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.852274099371551 = (5.114365963834364 - 0.7555243181509171) / 5.114365963834364 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.43898260958303 < 13.13014571974197.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 127.17549507673972 (T) = (0 -8.235363944529093) / Math.Sqrt((0.06300741091237684 / (299)) + (0.04076584939863911 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.402904435014047 = (13.792371652807091 - 8.235363944529093) / 13.792371652807091 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.369005745896575 < 8.016786047927333.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 33.451342583902836 (T) = (0 -6.373778400510952) / Math.Sqrt((0.4500166381280182 / (299)) + (0.10212900710116936 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.28181675186900107 = (8.874863646705881 - 6.373778400510952) / 8.874863646705881 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4069765333193283 < 2.9948258107149157.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.592955974512716 (T) = (0 -1.7204668088857293) / Math.Sqrt((0.5457541698291151 / (299)) + (0.07276059181385384 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.5451935755306622 = (3.7828551144437927 - 1.7204668088857293) / 3.7828551144437927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.010065523801966 < 14.707330710518983.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 118.69690268124396 (T) = (0 -8.293774371825466) / Math.Sqrt((0.4369262828485494 / (299)) + (0.04930248943784033 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.4621561219856155 = (15.420412336837366 - 8.293774371825466) / 15.420412336837366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFourValues - Duration of single invocation 164.76 ns 140.76 ns 0.85 0.42 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span&lt;Int32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 33)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 140.7612398078309 < 238.08347183610493.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/21/2022 10:31:08 AM, 2/24/2023 12:21:41 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 2.5741612528646347 (T) = (0 -158.36263049542123) / Math.Sqrt((915.4796332885186 / (299)) + (564.3645333420409 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07731921644078392 = (171.63317294258766 - 158.36263049542123) / 171.63317294258766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MaxBenchmark - Duration of single invocation 20.66 ns 18.21 ns 0.88 0.11 False
MinBenchmark - Duration of single invocation 20.82 ns 19.07 ns 0.92 0.08 False
EqualsStaticBenchmark - Duration of single invocation 19.66 ns 17.78 ns 0.90 0.04 False
LessThanOrEqualBenchmark - Duration of single invocation 20.39 ns 18.54 ns 0.91 0.02 False
LessThanBenchmark - Duration of single invocation 19.68 ns 18.03 ns 0.92 0.05 False
DivideBenchmark - Duration of single invocation 33.72 ns 29.01 ns 0.86 0.02 False
EqualsBenchmark - Duration of single invocation 11.06 ns 7.87 ns 0.71 0.11 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Int* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Int* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MaxBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.214734725047332 < 19.636115967263883.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 30.278993303619195 (T) = (0 -19.13018939365978) / Math.Sqrt((0.39221394648981783 / (299)) + (0.061377084823523866 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08962876242087782 = (21.013613572119404 - 19.13018939365978) / 21.013613572119404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.066783183922876 < 19.77950979800128.
IsChangePoint: Marked as a change because one of 12/15/2022 1:21:30 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 34.98591497335875 (T) = (0 -19.257251118213567) / Math.Sqrt((0.32686827679218633 / (299)) + (0.03188544477399262 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08206828004108102 = (20.978958128906804 - 19.257251118213567) / 20.978958128906804 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.78005062518444 < 18.691091073758532.
IsChangePoint: Marked as a change because one of 11/22/2022 11:24:47 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 58.327953443532365 (T) = (0 -17.928406774595853) / Math.Sqrt((0.09576610484877666 / (299)) + (0.014163245449529986 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08939003338722225 = (19.688348944043152 - 17.928406774595853) / 19.688348944043152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.543583023324032 < 19.525413941493834.
IsChangePoint: Marked as a change because one of 12/28/2022 6:20:56 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.393390701194924 (T) = (0 -18.783290326838667) / Math.Sqrt((0.14352672563681856 / (299)) + (0.031065932526340913 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08868508864835027 = (20.611196078180647 - 18.783290326838667) / 20.611196078180647 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.02505362406895 < 18.689887454618283.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 32.68334067734248 (T) = (0 -17.85044776779741) / Math.Sqrt((0.1791937779738052 / (299)) + (0.07563989487853098 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10083027398974975 = (19.85214498602227 - 17.85044776779741) / 19.85214498602227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.008207592736035 < 32.0465185797308.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 12.278751038910642 (T) = (0 -29.190848103114273) / Math.Sqrt((10.085933598987113 / (299)) + (0.09238634066687185 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07539646007698993 = (31.571205216827245 - 29.190848103114273) / 31.571205216827245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.8660466524616846 < 10.415590724836992.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 12.112698798638524 (T) = (0 -7.891211485697769) / Math.Sqrt((3.187051864073185 / (299)) + (0.2936605216979332 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18848017033074788 = (9.723990957699652 - 7.891211485697769) / 9.723990957699652 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFourValues - Duration of single invocation 26.94 ns 25.28 ns 0.94 0.04 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span&lt;Char&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span&lt;Char&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.IndexOfAnyFourValues(Size: 4)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.276716084883468 < 25.573973594224597.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 11.763747354242911 (T) = (0 -25.40105944071166) / Math.Sqrt((0.16822637146302746 / (299)) + (0.3526472196829385 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05410776771208651 = (26.854073406726116 - 25.40105944071166) / 26.854073406726116 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in BilinearTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Interpol_Vector - Duration of single invocation 444.53 μs 297.74 μs 0.67 0.06 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'BilinearTest*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter BilinearTest* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'BilinearTest*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter BilinearTest* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

BilinearTest.Interpol_Vector

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 297.74037790880504 < 415.88629051793976.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 6.252990715730643 (T) = (0 -297439.247893035) / Math.Sqrt((16472385205.461954 / (299)) + (628336.2970610406 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.13500475062274703 = (343862.29069717345 - 297439.247893035) / 343862.29069717345 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAnyBenchmark - Duration of single invocation 39.33 ns 34.14 ns 0.87 0.01 False
EqualsBenchmark - Duration of single invocation 11.02 ns 8.55 ns 0.78 0.16 False
GreaterThanAllBenchmark - Duration of single invocation 39.64 ns 34.45 ns 0.87 0.01 False
AndNotBenchmark - Duration of single invocation 14.47 ns 13.06 ns 0.90 0.08 False
MultiplyBenchmark - Duration of single invocation 56.05 ns 51.02 ns 0.91 0.00 False
SubtractionOperatorBenchmark - Duration of single invocation 54.94 ns 49.84 ns 0.91 0.01 False
EqualsStaticBenchmark - Duration of single invocation 59.32 ns 53.27 ns 0.90 0.01 False
BitwiseOrBenchmark - Duration of single invocation 15.43 ns 13.05 ns 0.85 0.03 False
ConditionalSelectBenchmark - Duration of single invocation 16.56 ns 15.50 ns 0.94 0.02 False
AddBenchmark - Duration of single invocation 55.18 ns 49.15 ns 0.89 0.01 False
LessThanOrEqualBenchmark - Duration of single invocation 64.47 ns 59.63 ns 0.93 0.01 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 43.64 ns 38.70 ns 0.89 0.01 False
SubtractBenchmark - Duration of single invocation 56.05 ns 49.48 ns 0.88 0.00 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 44.19 ns 39.06 ns 0.88 0.01 False
AddOperatorBenchmark - Duration of single invocation 54.82 ns 49.85 ns 0.91 0.01 False
DotBenchmark - Duration of single invocation 43.96 ns 39.10 ns 0.89 0.01 False
EqualsAllBenchmark - Duration of single invocation 8.25 ns 4.62 ns 0.56 0.29 False
LessThanAnyBenchmark - Duration of single invocation 40.00 ns 34.44 ns 0.86 0.01 False
LessThanBenchmark - Duration of single invocation 58.95 ns 53.44 ns 0.91 0.01 False
XorBenchmark - Duration of single invocation 15.56 ns 12.84 ns 0.83 0.03 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.24 ns 12.81 ns 0.90 0.02 False
MultiplyOperatorBenchmark - Duration of single invocation 55.17 ns 50.83 ns 0.92 0.01 False
DivideBenchmark - Duration of single invocation 105.27 ns 95.79 ns 0.91 0.00 False
DivisionOperatorBenchmark - Duration of single invocation 105.16 ns 96.63 ns 0.92 0.01 False
BitwiseAndBenchmark - Duration of single invocation 15.57 ns 12.89 ns 0.83 0.03 False
GreaterThanBenchmark - Duration of single invocation 62.47 ns 57.39 ns 0.92 0.00 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.38 ns 12.81 ns 0.89 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.EqualsAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.135897195103944 < 38.11737839247782.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 27.991010609199012 (T) = (0 -34.46502711365334) / Math.Sqrt((4.1260060086802754 / (299)) + (0.1110508433719442 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09929833949649747 = (38.26464258363529 - 34.46502711365334) / 38.26464258363529 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.547171464756323 < 10.403663782960754.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 20.16541506410003 (T) = (0 -8.645875257619688) / Math.Sqrt((1.239494786826037 / (299)) + (0.05102437451543923 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.15590942810222963 = (10.242828844991303 - 8.645875257619688) / 10.242828844991303 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.45134826737277 < 37.63339347642359.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.30921726451613 (T) = (0 -34.6036262636104) / Math.Sqrt((3.7125300378265256 / (299)) + (0.2662712064672288 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09724270815208295 = (38.33104044252893 - 34.6036262636104) / 38.33104044252893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.059621659057973 < 13.723706719623742.
IsChangePoint: Marked as a change because one of 11/21/2022 12:46:21 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.60343485123947 (T) = (0 -13.119481223257937) / Math.Sqrt((0.07742408776758275 / (299)) + (0.06446185488203685 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09236785091233946 = (14.45462375527956 - 13.119481223257937) / 14.45462375527956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.01767107829736 < 53.66434725609339.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 55.43866542926056 (T) = (0 -51.474743116286895) / Math.Sqrt((0.12116411296029443 / (299)) + (0.1966124263944233 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09079546028063988 = (56.61514089247218 - 51.474743116286895) / 56.61514089247218 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.83856358501411 < 52.20951898605322.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.03239862807616 (T) = (0 -49.58807778977658) / Math.Sqrt((3.3344348355681235 / (299)) + (0.10731153765499712 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08113878330470761 = (53.96688519308863 - 49.58807778977658) / 53.96688519308863 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.26905804785356 < 56.120054921517266.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 39.03399440131462 (T) = (0 -53.24748540814399) / Math.Sqrt((2.226821724231888 / (299)) + (0.2099456280935806 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.08448165298440051 = (58.16102493382004 - 53.24748540814399) / 58.16102493382004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.046237656284822 < 14.71137144240329.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 50.65690209241632 (T) = (0 -12.944070571483865) / Math.Sqrt((0.22752503880875044 / (299)) + (0.06072672669673034 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.18102387628946975 = (15.805186740778524 - 12.944070571483865) / 15.805186740778524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.500739978371708 < 15.789270037589308.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.29735626369784 (T) = (0 -15.415938401113522) / Math.Sqrt((0.1008598114842201 / (299)) + (0.0177369650298371 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08437052617778586 = (16.83643749120597 - 15.415938401113522) / 16.83643749120597 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.15272957743035 < 52.18964835922161.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 75.37407187630005 (T) = (0 -49.53368585377922) / Math.Sqrt((0.2406131817405395 / (299)) + (0.11645882250074452 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10269754946491912 = (55.20288708031635 - 49.53368585377922) / 55.20288708031635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 59.63224747545601 < 60.87961084060251.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.894844634325985 (T) = (0 -59.835808200463525) / Math.Sqrt((2.9097936490131304 / (299)) + (0.24158431496291846 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0531997382434881 = (63.19792105829758 - 59.835808200463525) / 63.19792105829758 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.697348544975505 < 41.356614794392655.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.73026613195581 (T) = (0 -38.56269345103071) / Math.Sqrt((3.795592389511109 / (299)) + (0.040822010680582256 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0820537586210089 = (42.00975145679516 - 38.56269345103071) / 42.00975145679516 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.48073202630652 < 52.550948198287415.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 54.68705349923638 (T) = (0 -49.53175321365683) / Math.Sqrt((0.2594131770941796 / (299)) + (0.2367986181332149 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10264762778746897 = (55.1976623090997 - 49.53175321365683) / 55.1976623090997 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.06205240287813 < 41.32779591421266.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.301225523295226 (T) = (0 -38.72379993067293) / Math.Sqrt((3.9925018678873734 / (299)) + (0.04185324035035581 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08258871605693698 = (42.20985789954185 - 38.72379993067293) / 42.20985789954185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.84632829883823 < 52.00266657522513.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 39.33014197426071 (T) = (0 -49.53884226766507) / Math.Sqrt((2.3727057834041463 / (299)) + (0.1327732229841263 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08436255728275617 = (54.103119811978964 - 49.53884226766507) / 54.103119811978964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.09683010612893 < 41.761660913829026.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 38.28363133412564 (T) = (0 -39.51597954815933) / Math.Sqrt((2.810935783171276 / (299)) + (0.11073646189121607 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10289247804066258 = (44.04820891686877 - 39.51597954815933) / 44.04820891686877 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.620673773311147 < 7.855786183764104.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 44.255280582134006 (T) = (0 -4.857995071607817) / Math.Sqrt((0.2566968773457997 / (299)) + (0.11491516918232646 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.40636870371437284 = (8.18352250294833 - 4.857995071607817) / 8.18352250294833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.442451751161954 < 37.998932986027846.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.303617826395403 (T) = (0 -34.44289337207875) / Math.Sqrt((4.191136862094142 / (299)) + (0.08557101432122029 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1013773649749743 = (38.32853973361082 - 34.44289337207875) / 38.32853973361082 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.43959536494112 < 56.03884146113051.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.506871048203735 (T) = (0 -53.21245333328747) / Math.Sqrt((2.18499183188652 / (299)) + (0.1519478746572836 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0853294691079828 = (58.17663468548933 - 53.21245333328747) / 58.17663468548933 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.84296621480007 < 14.687553060855576.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 66.05678547660376 (T) = (0 -12.983812993093553) / Math.Sqrt((0.41273240441868003 / (299)) + (0.01397116656892716 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18392852425649928 = (15.910141916508417 - 12.983812993093553) / 15.910141916508417 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.811587199984688 < 13.541460981308289.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 45.87071060431556 (T) = (0 -12.983380833588406) / Math.Sqrt((0.0895214367907364 / (299)) + (0.018477044707372145 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10356481189885786 = (14.483345818999164 - 12.983380833588406) / 14.483345818999164 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.83392109581124 < 53.04756025579278.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 30.593155364183435 (T) = (0 -51.2846256655213) / Math.Sqrt((2.7554022119881236 / (299)) + (0.18444875262952998 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07197010308823737 = (55.26182490045087 - 51.2846256655213) / 55.26182490045087 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 95.78742686201213 < 99.9940238217659.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 17.852282778103433 (T) = (0 -97.095723540501) / Math.Sqrt((1.1436487853263093 / (299)) + (4.018211559252613 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07070695450862484 = (104.48342857140445 - 97.095723540501) / 104.48342857140445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 96.63483361120693 < 99.245694867671.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 18.696425188843346 (T) = (0 -96.91230765569905) / Math.Sqrt((7.463453111641186 / (299)) + (2.1104325246564115 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06087545823815854 = (103.19430847146965 - 96.91230765569905) / 103.19430847146965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.890888784768237 < 14.735366742371665.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 34.940163649741145 (T) = (0 -12.895508844894616) / Math.Sqrt((0.48859009547217824 / (299)) + (0.15630983124969078 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.19395002984806772 = (15.998398762379388 - 12.895508844894616) / 15.998398762379388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 57.392352042532885 < 58.83739408493349.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.773520562200904 (T) = (0 -57.32747131130546) / Math.Sqrt((2.394329264581769 / (299)) + (0.21232652650326073 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06117523399702102 = (61.06301557784379 - 57.32747131130546) / 61.06301557784379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.811783527110842 < 13.590030679590678.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 49.773496671265654 (T) = (0 -12.960226299320047) / Math.Sqrt((0.09575441587154387 / (299)) + (0.0159664519727771 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10759227828983159 = (14.522763512717791 - 12.960226299320047) / 14.522763512717791 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AndNotBenchmark - Duration of single invocation 24.02 ns 15.29 ns 0.64 0.01 False
OnesComplementBenchmark - Duration of single invocation 8.47 ns 6.33 ns 0.75 0.09 False
LessThanAnyBenchmark - Duration of single invocation 2.17 ns 0.86 ns 0.39 0.44 False
EqualityOperatorBenchmark - Duration of single invocation 2.31 ns 0.98 ns 0.43 0.48 False
EqualsAllBenchmark - Duration of single invocation 5.56 ns 0.64 ns 0.12 0.28 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.85 ns 8.02 ns 0.58 0.03 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 2.81 ns 0.87 ns 0.31 0.47 False
MultiplyBenchmark - Duration of single invocation 44.69 ns 40.37 ns 0.90 0.01 False
SubtractBenchmark - Duration of single invocation 42.46 ns 38.64 ns 0.91 0.00 False
BitwiseOrBenchmark - Duration of single invocation 14.76 ns 8.06 ns 0.55 0.29 False
OneBenchmark - Duration of single invocation 23.24 ns 0.81 ns 0.04 0.10 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 2.32 ns 0.60 ns 0.26 0.52 False
EqualsBenchmark - Duration of single invocation 7.91 ns 6.35 ns 0.80 0.09 False
XorBenchmark - Duration of single invocation 16.08 ns 8.43 ns 0.52 0.29 False
BitwiseAndBenchmark - Duration of single invocation 16.13 ns 8.53 ns 0.53 0.27 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.88 ns 8.05 ns 0.58 0.03 False
ConditionalSelectBenchmark - Duration of single invocation 57.64 ns 32.64 ns 0.57 0.32 False
GreaterThanAllBenchmark - Duration of single invocation 2.27 ns 0.70 ns 0.31 0.43 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.88 ns 8.05 ns 0.58 0.03 False
DivideBenchmark - Duration of single invocation 91.62 ns 83.15 ns 0.91 0.00 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.39 ns 6.40 ns 0.76 0.07 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.AndNotBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.287194866892223 < 22.833891655775687.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 152.01230012143654 (T) = (0 -15.530960552884125) / Math.Sqrt((0.12771128429019277 / (299)) + (0.06374559039229057 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3521119850634627 = (23.97167441723001 - 15.530960552884125) / 23.97167441723001 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.327822257002364 < 8.03921147419971.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.55438623594902 (T) = (0 -6.472875235585014) / Math.Sqrt((0.24884600554186734 / (299)) + (0.07751994387318895 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2593086496654598 = (8.738964256382202 - 6.472875235585014) / 8.738964256382202 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8553769679501506 < 2.1751923133434543.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/7/2023 1:52:33 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 33.32267783170011 (T) = (0 -0.6391928065030041) / Math.Sqrt((0.933265135043863 / (299)) + (0.0286643440468258 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.77616498434819 = (2.8556426019479932 - 0.6391928065030041) / 2.8556426019479932 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.9836680824270048 < 2.184395630902618.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 27.163220766847306 (T) = (0 -0.6854875831232605) / Math.Sqrt((0.9793435720598734 / (299)) + (0.07624318849628375 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7609366283813431 = (2.8673885860554145 - 0.6854875831232605) / 2.8673885860554145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.6428290097306435 < 5.307788137705861.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 62.71314427012969 (T) = (0 -0.9385053617618606) / Math.Sqrt((0.1888393070288684 / (299)) + (0.09943941885030178 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.8219835497940601 = (5.272014809171525 - 0.9385053617618606) / 5.272014809171525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.0217762235008 < 13.119043800528374.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 123.05956422529526 (T) = (0 -8.238842931111787) / Math.Sqrt((0.07630438709303498 / (299)) + (0.042610901091843206 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4023001180614598 = (13.784247211812172 - 8.238842931111787) / 13.784247211812172 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8740306402452047 < 3.349258931255742.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.36088717578545 (T) = (0 -1.016293868787583) / Math.Sqrt((0.33744062585462203 / (299)) + (0.07555396357046437 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7316028017285781 = (3.786529350279714 - 1.016293868787583) / 3.786529350279714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.3734033552111 < 42.21205990880771.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.273336368883086 (T) = (0 -40.744865430281564) / Math.Sqrt((0.6288358430021727 / (299)) + (0.15783650358011422 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06273540094283013 = (43.47210539186946 - 40.744865430281564) / 43.47210539186946 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.643739095881315 < 40.231148840933855.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.470518849709293 (T) = (0 -39.21696823479406) / Math.Sqrt((0.5233952364891367 / (299)) + (0.2174789913529717 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05386150347022824 = (41.44950065834261 - 39.21696823479406) / 41.44950065834261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.06389856282004 < 15.292857317300834.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 127.89681313597568 (T) = (0 -8.247800952964976) / Math.Sqrt((0.40618299156407317 / (299)) + (0.04424636562913351 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.46737101007782583 = (15.485077059305606 - 8.247800952964976) / 15.485077059305606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.8148452003075889 < 21.97402057613532.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 138.505415102147 (T) = (0 -0.9009579824648447) / Math.Sqrt((7.8015382148980095 / (299)) + (0.07152183209630496 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.963251574861283 = (24.516914100779378 - 0.9009579824648447) / 24.516914100779378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.6048733354456389 < 3.135977610446099.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.80801581269736 (T) = (0 -1.2090152007209192) / Math.Sqrt((0.3576835472029317 / (299)) + (0.09140960901117169 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.6769317100234007 = (3.7422899065968096 - 1.2090152007209192) / 3.7422899065968096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.350520568685576 < 8.365123699035292.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.116431243426916 (T) = (0 -6.330522135600525) / Math.Sqrt((0.8322069196096854 / (299)) + (0.1043470789754851 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.21977527489868973 = (8.113716384441062 - 6.330522135600525) / 8.113716384441062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.427914600595342 < 14.659739411182398.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 110.06523190635052 (T) = (0 -8.143255999491046) / Math.Sqrt((0.4076324081717099 / (299)) + (0.07752401998562207 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.47453690426052714 = (15.497293845215177 - 8.143255999491046) / 15.497293845215177 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.527789263070897 < 15.29593906893335.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 130.5658375313294 (T) = (0 -8.316712311098858) / Math.Sqrt((0.4437094986024355 / (299)) + (0.037669652105108226 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4645308986538239 = (15.53163812849432 - 8.316712311098858) / 15.53163812849432 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.047039061554068 < 12.974677125200353.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 119.14088963402281 (T) = (0 -8.165611189897442) / Math.Sqrt((0.20867144876460036 / (299)) + (0.03982433149147997 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4146547243606682 = (13.950076185339865 - 8.165611189897442) / 13.950076185339865 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.642881170687545 < 57.05846044981243.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 102.19007379343685 (T) = (0 -32.53251717638804) / Math.Sqrt((15.996312266715092 / (299)) + (0.022388099652166166 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.4228345387806111 = (56.366015228381734 - 32.53251717638804) / 56.366015228381734 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7020239773248663 < 2.096777911973468.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.996283790625665 (T) = (0 -0.7415656127997913) / Math.Sqrt((0.963234198038485 / (299)) + (0.12266751268774612 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.741484403359482 = (2.8685527002495883 - 0.7415656127997913) / 2.8685527002495883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.045987225356368 < 13.146688024132681.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 128.57363012664183 (T) = (0 -8.186145381063438) / Math.Sqrt((0.0503270023630336 / (299)) + (0.041380117740407105 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.40591647503978895 = (13.779451940889471 - 8.186145381063438) / 13.779451940889471 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 83.14987194440874 < 87.02778605238194.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/10/2023 12:53:36 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.48643460885977 (T) = (0 -83.93685400753256) / Math.Sqrt((1.4873040158566462 / (299)) + (0.19693382266001827 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.07266736008688626 = (90.51428839536696 - 83.93685400753256) / 90.51428839536696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.396461225341194 < 7.980606758072394.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.01154359190461 (T) = (0 -6.545908048746128) / Math.Sqrt((0.22055913735417937 / (299)) + (0.0764331707840294 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.25092373900462134 = (8.738640362261476 - 6.545908048746128) / 8.738640362261476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAllBenchmark - Duration of single invocation 5.44 ns 0.45 ns 0.08 0.18 False
BitwiseOrOperatorBenchmark - Duration of single invocation 13.79 ns 8.06 ns 0.58 0.03 False
EqualityOperatorBenchmark - Duration of single invocation 1.95 ns 0.37 ns 0.19 0.40 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 13.86 ns 8.05 ns 0.58 0.02 False
ConditionalSelectBenchmark - Duration of single invocation 60.57 ns 32.55 ns 0.54 0.28 False
OneBenchmark - Duration of single invocation 11.18 ns 0.80 ns 0.07 0.22 False
SubtractBenchmark - Duration of single invocation 13.40 ns 11.71 ns 0.87 0.02 False
BitwiseAndOperatorBenchmark - Duration of single invocation 13.87 ns 8.01 ns 0.58 0.04 False
GreaterThanAnyBenchmark - Duration of single invocation 8.30 ns 6.97 ns 0.84 0.10 False
LessThanAnyBenchmark - Duration of single invocation 1.99 ns 0.45 ns 0.23 0.44 False
LessThanAllBenchmark - Duration of single invocation 8.06 ns 7.03 ns 0.87 0.10 False
AndNotBenchmark - Duration of single invocation 24.34 ns 15.34 ns 0.63 0.02 False
BitwiseOrBenchmark - Duration of single invocation 16.15 ns 8.04 ns 0.50 0.31 False
OnesComplementOperatorBenchmark - Duration of single invocation 8.41 ns 6.38 ns 0.76 0.05 False
AddBenchmark - Duration of single invocation 13.44 ns 11.73 ns 0.87 0.01 False
MultiplyBenchmark - Duration of single invocation 13.60 ns 12.29 ns 0.90 0.02 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 3.50 ns 0.95 ns 0.27 0.44 False
XorBenchmark - Duration of single invocation 16.36 ns 8.04 ns 0.49 0.28 False
EqualsBenchmark - Duration of single invocation 8.90 ns 5.48 ns 0.62 0.08 False
BitwiseAndBenchmark - Duration of single invocation 16.08 ns 8.33 ns 0.52 0.29 False
OnesComplementBenchmark - Duration of single invocation 8.38 ns 6.35 ns 0.76 0.06 False
DivideBenchmark - Duration of single invocation 15.58 ns 12.00 ns 0.77 0.02 False
GreaterThanAllBenchmark - Duration of single invocation 2.37 ns 1.26 ns 0.53 0.45 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 3.46 ns 1.25 ns 0.36 0.42 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt32>.EqualsAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.44652098957608716 < 5.150393002096178.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 71.42873808603915 (T) = (0 -0.7672634116163123) / Math.Sqrt((0.19572365319373317 / (299)) + (0.07396374086523806 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.8505360121063746 = (5.133433293392247 - 0.7672634116163123) / 5.133433293392247 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.058909548208797 < 12.972988025700076.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 112.54870003278276 (T) = (0 -8.230245514235685) / Math.Sqrt((0.21536310087442861 / (299)) + (0.045393227633361156 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.41136776289599053 = (13.9819822895317 - 8.230245514235685) / 13.9819822895317 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.37378490997817526 < 2.0861486804184732.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.583701311394638 (T) = (0 -0.7723882125530958) / Math.Sqrt((0.778290918654201 / (299)) + (0.140558713288954 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.6998825252553436 = (2.5736195908294013 - 0.7723882125530958) / 2.5736195908294013 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.052070639593602 < 13.166614102026585.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 143.275512289987 (T) = (0 -8.24370227777069) / Math.Sqrt((0.07463464254191472 / (299)) + (0.02984146124609651 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4017542169154706 = (13.779791702444633 - 8.24370227777069) / 13.779791702444633 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.5530655488175 < 57.31324773527244.
IsChangePoint: Marked as a change because one of 1/5/2023 9:29:47 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 91.31254397775528 (T) = (0 -32.447100789648466) / Math.Sqrt((19.126832036327016 / (299)) + (0.0859159017235978 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4224380998795905 = (56.17943424399 - 32.447100789648466) / 56.17943424399 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OneBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.7954301798844134 < 10.035625285967964.
IsChangePoint: Marked as a change because one of 12/15/2022 5:36:17 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 136.09153349082905 (T) = (0 -0.9294590049901665) / Math.Sqrt((1.2180171461397167 / (299)) + (0.042356113509950354 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.9174924825520655 = (11.265143271056386 - 0.9294590049901665) / 11.265143271056386 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.710803978974168 < 12.763136614621914.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 59.38496984842192 (T) = (0 -11.753672567534203) / Math.Sqrt((0.16356163265688436 / (299)) + (0.013318824629548896 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14363171994804602 = (13.725020930038573 - 11.753672567534203) / 13.725020930038573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.014929652050018 < 13.113734562884133.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 97.97951344186524 (T) = (0 -8.213551761041193) / Math.Sqrt((0.22133164663945035 / (299)) + (0.06424517270881236 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.41083723762817875 = (13.941057184224437 - 8.213551761041193) / 13.941057184224437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.970700942507417 < 7.525399875312005.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 10.754532629852296 (T) = (0 -6.896732832774521) / Math.Sqrt((0.050690492965803045 / (299)) + (0.2097927713208164 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12831269279324053 = (7.91193444685395 - 6.896732832774521) / 7.91193444685395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.4528399105271268 < 1.936432192228187.
IsChangePoint: Marked as a change because one of 1/5/2023 9:23:01 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.484807474275755 (T) = (0 -0.6453065002539501) / Math.Sqrt((0.7399802480495085 / (299)) + (0.09971774547353655 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.7531149818211224 = (2.6137936802078476 - 0.6453065002539501) / 2.6137936802078476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.031116099851796 < 7.516556976705794.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 15.082810936581941 (T) = (0 -6.918762581374704) / Math.Sqrt((0.048170374099494984 / (299)) + (0.11233094491108984 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1317080543700664 = (7.96824457050012 - 6.918762581374704) / 7.96824457050012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.34016697708412 < 23.118975433332675.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 105.80465954520362 (T) = (0 -15.542228711955472) / Math.Sqrt((0.27727946379536067 / (299)) + (0.12417516242143253 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.34714983331389193 = (23.80673162855791 - 15.542228711955472) / 23.80673162855791 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.04267376499011 < 15.311607134551423.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 99.36679571228619 (T) = (0 -8.111562844225904) / Math.Sqrt((0.39730344852118754 / (299)) + (0.09844329911500492 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.474440454133701 = (15.434146155323502 - 8.111562844225904) / 15.434146155323502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.377235757786069 < 8.016847561145248.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 47.176213753667675 (T) = (0 -6.439745953853273) / Math.Sqrt((0.14913723122558575 / (299)) + (0.04237910699511606 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.25849958632371434 = (8.68475031851385 - 6.439745953853273) / 8.68475031851385 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.731505959467293 < 12.749029590483751.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 52.00335598719035 (T) = (0 -11.736552585601848) / Math.Sqrt((0.34594665036659605 / (299)) + (0.013122771411612407 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.15461569892489405 = (13.883097392128109 - 11.736552585601848) / 13.883097392128109 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.294288589447016 < 12.978319551573358.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.484680568016024 (T) = (0 -12.354079542333947) / Math.Sqrt((0.3214033225395826 / (299)) + (0.012268133625738108 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.12119907951114951 = (14.05788188690306 - 12.354079542333947) / 14.05788188690306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.953920315659316 < 3.171219464758179.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 41.23198405390144 (T) = (0 -0.8857170810227074) / Math.Sqrt((0.43290046442700697 / (299)) + (0.05590288561932497 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.7410003741836247 = (3.4197620101994293 - 0.8857170810227074) / 3.4197620101994293 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.04358880476355 < 14.050181685525017.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 97.96405834561979 (T) = (0 -8.265857087330271) / Math.Sqrt((0.39506957812112525 / (299)) + (0.09709971800384683 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.46474254760748523 = (15.442768802906375 - 8.265857087330271) / 15.442768802906375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.480608579955632 < 8.471574982652783.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 26.059144059833887 (T) = (0 -5.66893670417998) / Math.Sqrt((1.4626105395746272 / (299)) + (0.09665868895824556 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3027130118997061 = (8.12999066514144 - 5.66893670417998) / 8.12999066514144 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.333249096500618 < 14.001792215076298.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 96.02983981100117 (T) = (0 -8.148062012055716) / Math.Sqrt((0.4100340726049704 / (299)) + (0.10663379600219292 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4733184403608148 = (15.47056634684101 - 8.148062012055716) / 15.47056634684101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.351337654453274 < 8.019213842803655.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 26.315331785699655 (T) = (0 -6.426190826147759) / Math.Sqrt((0.15147267888901403 / (299)) + (0.16557998078224562 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2605756805096794 = (8.690802637621227 - 6.426190826147759) / 8.690802637621227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.00380098445845 < 14.822633535938902.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 38.264329678783554 (T) = (0 -13.36106134762836) / Math.Sqrt((0.29546334066684327 / (299)) + (0.0939401094914019 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.16702675636932526 = (16.040204712208514 - 13.36106134762836) / 16.040204712208514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2583177367804104 < 2.04157050299217.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 6:03:21 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 15.624664292184551 (T) = (0 -0.8665773272546454) / Math.Sqrt((0.8249244426118123 / (299)) + (0.2284642231294002 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.6629217187323068 = (2.5708488959763227 - 0.8665773272546454) / 2.5708488959763227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2539955014569382 < 2.9599679225016344.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 36.26985617983505 (T) = (0 -0.8665069957495676) / Math.Sqrt((0.4423965133880132 / (299)) + (0.08071284210688186 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.7417409165076938 = (3.3551849717432356 - 0.8665069957495676) / 3.3551849717432356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MaxBenchmark - Duration of single invocation 21.17 ns 19.36 ns 0.91 0.01 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 8.67 ns 7.03 ns 0.81 0.11 False
DivideBenchmark - Duration of single invocation 22.13 ns 19.60 ns 0.89 0.02 False
AddOperatorBenchmark - Duration of single invocation 18.99 ns 17.46 ns 0.92 0.01 False
EqualsStaticBenchmark - Duration of single invocation 19.04 ns 16.30 ns 0.86 0.02 False
LessThanOrEqualBenchmark - Duration of single invocation 19.03 ns 16.30 ns 0.86 0.02 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.55 ns 12.78 ns 0.88 0.01 False
ConditionalSelectBenchmark - Duration of single invocation 16.64 ns 15.36 ns 0.92 0.04 False
BitwiseOrBenchmark - Duration of single invocation 15.46 ns 12.83 ns 0.83 0.04 False
EqualsAllBenchmark - Duration of single invocation 8.14 ns 6.57 ns 0.81 0.10 False
SubtractionOperatorBenchmark - Duration of single invocation 18.79 ns 17.30 ns 0.92 0.02 False
GreaterThanOrEqualBenchmark - Duration of single invocation 18.87 ns 16.61 ns 0.88 0.02 False
MultiplyBenchmark - Duration of single invocation 20.88 ns 18.27 ns 0.87 0.01 False
AndNotBenchmark - Duration of single invocation 14.57 ns 13.16 ns 0.90 0.05 False
XorBenchmark - Duration of single invocation 15.50 ns 12.85 ns 0.83 0.02 False
GreaterThanBenchmark - Duration of single invocation 19.02 ns 16.25 ns 0.85 0.03 False
LessThanBenchmark - Duration of single invocation 17.57 ns 16.28 ns 0.93 0.03 False
MinBenchmark - Duration of single invocation 21.09 ns 19.08 ns 0.90 0.02 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 9.56 ns 6.43 ns 0.67 0.14 False
GreaterThanAnyBenchmark - Duration of single invocation 8.47 ns 6.85 ns 0.81 0.12 False
BitwiseAndBenchmark - Duration of single invocation 15.54 ns 12.87 ns 0.83 0.03 False
EqualsAnyBenchmark - Duration of single invocation 8.27 ns 6.96 ns 0.84 0.13 False
AddBenchmark - Duration of single invocation 20.31 ns 17.28 ns 0.85 0.03 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.42 ns 12.87 ns 0.89 0.02 False
SubtractBenchmark - Duration of single invocation 20.41 ns 17.38 ns 0.85 0.00 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.MaxBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.36230696827983 < 20.070155941013386.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 36.799711497158285 (T) = (0 -19.177234929734816) / Math.Sqrt((0.08902629301534594 / (299)) + (0.052755543417832916 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08748094656701481 = (21.015709050225524 - 19.177234929734816) / 21.015709050225524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.026805146257023 < 8.783936888943455.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.65879386555762 (T) = (0 -6.907103713158889) / Math.Sqrt((0.08872777505618702 / (299)) + (0.1573518729661205 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.25197084476320547 = (9.233735964438967 - 6.907103713158889) / 9.233735964438967 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.60013572732997 < 21.037931640604373.
IsChangePoint: Marked as a change because one of 11/21/2022 10:59:14 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 85.02176229387837 (T) = (0 -19.627506845423856) / Math.Sqrt((0.12707912257174278 / (299)) + (0.008149822278320881 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10696631442394054 = (21.978461912960153 - 19.627506845423856) / 21.978461912960153 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.462475705063984 < 17.98466352225514.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 53.55362953973174 (T) = (0 -17.40225065325651) / Math.Sqrt((0.042375725592348604 / (299)) + (0.012729069073333439 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07388656852278855 = (18.790625491197964 - 17.40225065325651) / 18.790625491197964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.297527859058327 < 18.099965501550102.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 41.59268497355321 (T) = (0 -16.34989804596084) / Math.Sqrt((0.3308095316228958 / (299)) + (0.04439081877560524 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12150488921231718 = (18.61125673346215 - 16.34989804596084) / 18.61125673346215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.29835405463674 < 18.252931549307473.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 57.52371897171539 (T) = (0 -16.421945206843407) / Math.Sqrt((0.4092276396866683 / (299)) + (0.005428708910176449 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.12241536711408968 = (18.71266267828818 - 16.421945206843407) / 18.71266267828818 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.783931449299349 < 13.637617269412992.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.74909728545574 (T) = (0 -12.97153219439595) / Math.Sqrt((0.16900193837500282 / (299)) + (0.020028900519513032 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10976577838103126 = (14.570920640195212 - 12.97153219439595) / 14.570920640195212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.363824936475444 < 15.810353365391357.
IsChangePoint: Marked as a change because one of 12/15/2022 1:21:30 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 30.101833073350285 (T) = (0 -15.285504247851582) / Math.Sqrt((0.11096449282402299 / (299)) + (0.04556546673456942 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08577280392063837 = (16.719590396569966 - 15.285504247851582) / 16.719590396569966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.83047442490318 < 14.701789133982585.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 58.77583153706824 (T) = (0 -12.96619385037053) / Math.Sqrt((0.5721395349962712 / (299)) + (0.0164734422286684 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18774110460531918 = (15.963129396164002 - 12.96619385037053) / 15.963129396164002 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.574628184760863 < 8.255407797331358.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 30.08075600561711 (T) = (0 -6.5826382270672035) / Math.Sqrt((0.1109882855584844 / (299)) + (0.09845636128943927 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.23409397304634386 = (8.594576874201186 - 6.5826382270672035) / 8.594576874201186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.30040442354895 < 17.863923292414093.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 12/16/2022 5:41:28 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.08418823449564 (T) = (0 -17.33564470666529) / Math.Sqrt((0.04611425598127245 / (299)) + (0.03225003172298408 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08588778797520184 = (18.964460247463585 - 17.33564470666529) / 18.964460247463585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.605510792468817 < 18.061575199467196.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 39.750539591927506 (T) = (0 -16.382837959433164) / Math.Sqrt((0.1282258710139225 / (299)) + (0.0837908040856561 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1318813426038344 = (18.871657485823235 - 16.382837959433164) / 18.871657485823235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.266006014748495 < 19.836769484020966.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 60.564290705417626 (T) = (0 -18.290040794150617) / Math.Sqrt((0.04600617312084073 / (299)) + (0.03922059641616105 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12282308324240913 = (20.851028389755378 - 18.290040794150617) / 20.851028389755378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.16240557462708 < 13.847711957686938.
IsChangePoint: Marked as a change because one of 11/24/2022 1:04:57 PM, 12/5/2022 9:44:53 PM, 12/15/2022 5:36:17 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 30.202777303785165 (T) = (0 -13.130912869411917) / Math.Sqrt((0.20980877455010444 / (299)) + (0.019515947844996308 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08216792369338978 = (14.306443638635065 - 13.130912869411917) / 14.306443638635065 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.850074011982631 < 14.677028965081274.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 51.38373053916248 (T) = (0 -12.898112551504717) / Math.Sqrt((0.24386293904616269 / (299)) + (0.05958352758587601 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18619258343281536 = (15.849096836585417 - 12.898112551504717) / 15.849096836585417 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.248428102416007 < 18.015395364549818.
IsChangePoint: Marked as a change because one of 12/15/2022 5:36:17 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 39.98997060696605 (T) = (0 -16.345518038442535) / Math.Sqrt((0.28228854769194406 / (299)) + (0.06223751103010468 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12702584467431166 = (18.72394267198479 - 16.345518038442535) / 18.72394267198479 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.281619589434957 < 18.067758956156403.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.607664796453044 (T) = (0 -16.30973402305283) / Math.Sqrt((0.20100980163696225 / (299)) + (0.057507362383604155 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12900034981449027 = (18.725304906355706 - 16.30973402305283) / 18.725304906355706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.07984124619197 < 20.042607900762558.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 93.73582675494167 (T) = (0 -19.18126109532314) / Math.Sqrt((0.07281679148628888 / (299)) + (0.004073160043767146 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09036528273348449 = (21.08677332914855 - 19.18126109532314) / 21.08677332914855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.433824275840003 < 8.969945475739872.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 16.474923801059646 (T) = (0 -6.974479907473586) / Math.Sqrt((0.09504776907866061 / (299)) + (0.43502272054019303 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2428842904541533 = (9.211907532148823 - 6.974479907473586) / 9.211907532148823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.854399398023598 < 8.182496381372296.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 13.025909490987413 (T) = (0 -6.522490249030238) / Math.Sqrt((0.057490816266700516 / (299)) + (0.5164179496067117 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.22735375945638012 = (8.441754980184893 - 6.522490249030238) / 8.441754980184893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.874369518291868 < 14.731253051343925.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 56.34228790478465 (T) = (0 -12.965132443104673) / Math.Sqrt((0.6160408285347271 / (299)) + (0.02171265261184021 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.19135152141512562 = (16.03308827810263 - 12.965132443104673) / 16.03308827810263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.957536221665172 < 7.99467446137607.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 17.789156058097483 (T) = (0 -6.703770275841719) / Math.Sqrt((0.06495177494965908 / (299)) + (0.22275027956135546 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.20547955694299205 = (8.437505081742389 - 6.703770275841719) / 8.437505081742389 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.280609234286924 < 19.38568417231124.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 53.856996471221784 (T) = (0 -17.331791507281096) / Math.Sqrt((0.06404496941939014 / (299)) + (0.06776006399788469 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14621953707641344 = (20.300056349300995 - 17.331791507281096) / 20.300056349300995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.873933942505568 < 13.64107849375354.
IsChangePoint: Marked as a change because one of 11/24/2022 9:42:44 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 45.81117233821024 (T) = (0 -13.019099378078133) / Math.Sqrt((0.06471237945947578 / (299)) + (0.01909566508793398 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.09923640778494013 = (14.453403191022607 - 13.019099378078133) / 14.453403191022607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.380037453573358 < 19.332481012072094.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 90.74188492581789 (T) = (0 -17.388069209066824) / Math.Sqrt((0.04283745751551181 / (299)) + (0.020907849074476532 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14252437568489032 = (20.278208168255713 - 17.388069209066824) / 20.278208168255713 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanAllBenchmark - Duration of single invocation 7.74 ns 5.85 ns 0.76 0.17 False
AddBenchmark - Duration of single invocation 17.79 ns 14.01 ns 0.79 0.04 False
GreaterThanBenchmark - Duration of single invocation 15.76 ns 13.64 ns 0.87 0.03 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 8.64 ns 6.96 ns 0.80 0.21 False
BitwiseOrBenchmark - Duration of single invocation 15.60 ns 12.87 ns 0.82 0.02 False
LessThanBenchmark - Duration of single invocation 15.10 ns 13.62 ns 0.90 0.01 False
EqualsAllBenchmark - Duration of single invocation 8.44 ns 4.81 ns 0.57 0.33 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.42 ns 13.04 ns 0.90 0.04 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.20 ns 12.81 ns 0.90 0.02 False
GreaterThanOrEqualBenchmark - Duration of single invocation 16.18 ns 14.19 ns 0.88 0.04 False
DivideBenchmark - Duration of single invocation 23.10 ns 19.29 ns 0.83 0.01 False
SubtractionOperatorBenchmark - Duration of single invocation 15.74 ns 13.79 ns 0.88 0.02 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.42 ns 12.77 ns 0.89 0.03 False
EqualsAnyBenchmark - Duration of single invocation 7.74 ns 6.43 ns 0.83 0.17 False
BitwiseAndBenchmark - Duration of single invocation 15.50 ns 13.88 ns 0.90 0.02 False
AddOperatorBenchmark - Duration of single invocation 15.71 ns 13.82 ns 0.88 0.02 False
EqualsBenchmark - Duration of single invocation 11.01 ns 7.52 ns 0.68 0.13 False
LessThanOrEqualBenchmark - Duration of single invocation 15.46 ns 13.75 ns 0.89 0.04 False
MultiplyBenchmark - Duration of single invocation 18.52 ns 14.65 ns 0.79 0.07 False
SubtractBenchmark - Duration of single invocation 17.87 ns 13.91 ns 0.78 0.05 False
ConditionalSelectBenchmark - Duration of single invocation 16.67 ns 15.25 ns 0.91 0.03 False
XorBenchmark - Duration of single invocation 15.61 ns 12.99 ns 0.83 0.01 False
MinBenchmark - Duration of single invocation 15.68 ns 14.68 ns 0.94 0.03 False
MaxBenchmark - Duration of single invocation 15.71 ns 14.33 ns 0.91 0.03 False
EqualsStaticBenchmark - Duration of single invocation 15.13 ns 13.51 ns 0.89 0.04 False
MultiplyOperatorBenchmark - Duration of single invocation 16.57 ns 14.69 ns 0.89 0.02 False
LessThanAnyBenchmark - Duration of single invocation 7.95 ns 6.13 ns 0.77 0.20 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.8469913919647105 < 7.588597294609138.
IsChangePoint: Marked as a change because one of 12/14/2022 12:01:27 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.33154716762044 (T) = (0 -6.253725840821047) / Math.Sqrt((0.24788876342720018 / (299)) + (0.16127882452525305 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2286129687903085 = (8.107118201110971 - 6.253725840821047) / 8.107118201110971 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.009351931677164 < 16.728920623792796.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 90.46020724059188 (T) = (0 -13.945257108300003) / Math.Sqrt((0.07116203718340271 / (299)) + (0.03440323721334576 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.20961365045761252 = (17.64359558635335 - 13.945257108300003) / 17.64359558635335 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.644934691435388 < 14.922338767746176.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.61517078384404 (T) = (0 -13.88880816698864) / Math.Sqrt((0.6309863000563106 / (299)) + (0.018838359280967557 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14454784918558397 = (16.235634165822226 - 13.88880816698864) / 16.235634165822226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.955594051745868 < 8.183124660001466.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 12.864745789571245 (T) = (0 -7.339275603093431) / Math.Sqrt((0.12775538341759188 / (299)) + (0.17381684358332533 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1330804903166765 = (8.465925061225569 - 7.339275603093431) / 8.465925061225569 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.867976091690364 < 14.717546347007286.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 46.16706454079624 (T) = (0 -12.94272095130934) / Math.Sqrt((0.4702809078527165 / (299)) + (0.06659703352724644 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1904145971795619 = (15.986850684584255 - 12.94272095130934) / 15.986850684584255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.622362624047424 < 14.296283174741214.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.48068583371721 (T) = (0 -13.438050932108581) / Math.Sqrt((0.42538853773607177 / (299)) + (0.021627383189701588 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.13493172613180296 = (15.534092901152933 - 13.438050932108581) / 15.534092901152933 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.807507968919453 < 7.798182125937259.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 26.4939213485919 (T) = (0 -4.57449639823528) / Math.Sqrt((0.3636378891556802 / (299)) + (0.34717353412898494 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.42038081434206126 = (7.89224461754603 - 4.57449639823528) / 7.89224461754603 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.038151269444914 < 13.55407302636986.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 27.77935278589623 (T) = (0 -12.949748614184529) / Math.Sqrt((0.29740568954958135 / (299)) + (0.06785584513499245 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11709058499719685 = (14.667131637896754 - 12.949748614184529) / 14.667131637896754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.805375419662404 < 13.633532119358009.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 39.63795313790131 (T) = (0 -12.99587769946326) / Math.Sqrt((0.2772718887436569 / (299)) + (0.021640446374195405 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.11437693187213734 = (14.674276413029215 - 12.99587769946326) / 14.674276413029215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.185688693873782 < 15.3661104382425.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 33.500689529331815 (T) = (0 -14.243225697078044) / Math.Sqrt((0.4261264245381693 / (299)) + (0.0801745763873656 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1396905558125707 = (16.555933209045392 - 14.243225697078044) / 16.555933209045392 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.285630991269937 < 21.992253328611035.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 74.94236590778263 (T) = (0 -19.327689701681653) / Math.Sqrt((0.07018979901644364 / (299)) + (0.05647426793467039 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1647523692845685 = (23.140071268596735 - 19.327689701681653) / 23.140071268596735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.791366065632458 < 14.816012188133916.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 38.67262442469498 (T) = (0 -13.886562491572084) / Math.Sqrt((0.19034801888342615 / (299)) + (0.05381364603681038 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12999853228238767 = (15.961539154643617 - 13.886562491572084) / 15.961539154643617 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.769532756511122 < 13.672826209660697.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 22.190179632701298 (T) = (0 -12.899048766171754) / Math.Sqrt((0.13085424418790517 / (299)) + (0.12106837845597725 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11298245648980479 = (14.54204469860465 - 12.899048766171754) / 14.54204469860465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.433656322023201 < 7.435003437446762.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 18.59112851149577 (T) = (0 -6.3411458203139395) / Math.Sqrt((0.31375148698592653 / (299)) + (0.16739145947940562 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2079980214990347 = (8.006477246832043 - 6.3411458203139395) / 8.006477246832043 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.882665492435027 < 14.720576221174163.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 47.671454767095014 (T) = (0 -13.027289620373462) / Math.Sqrt((0.4908483230295786 / (299)) + (0.05097874025425535 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.18520039042820355 = (15.988335619379745 - 13.027289620373462) / 15.988335619379745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.815361540533642 < 14.942390437839862.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 45.4285677941016 (T) = (0 -13.989715688930795) / Math.Sqrt((0.10121967701314782 / (299)) + (0.03047410084745471 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1152220240948398 = (15.811555067946628 - 13.989715688930795) / 15.811555067946628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.5248116255996464 < 10.367043352524014.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.381872328480075 (T) = (0 -7.71398458899571) / Math.Sqrt((1.1845862449062747 / (299)) + (0.2418188692645573 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2293996691834639 = (10.010357224765128 - 7.71398458899571) / 10.010357224765128 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.751996776767472 < 14.637766341069693.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.282682008519437 (T) = (0 -13.850883967424375) / Math.Sqrt((0.5696458200319656 / (299)) + (0.05793641224023468 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12924752033007256 = (15.906798189853875 - 13.850883967424375) / 15.906798189853875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.653884606349717 < 17.633572056312207.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 46.553700127752656 (T) = (0 -14.914007023081473) / Math.Sqrt((0.06112327091536339 / (299)) + (0.15568808550653893 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.20340323062319304 = (18.722153536661953 - 14.914007023081473) / 18.722153536661953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.914717544065043 < 16.68303024973676.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 90.58598409971718 (T) = (0 -13.89596606302032) / Math.Sqrt((0.1534261807100764 / (299)) + (0.03075567872714845 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2163982041301591 = (17.733453568205565 - 13.89596606302032) / 17.733453568205565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.24655777234091 < 15.839664632929393.
IsChangePoint: Marked as a change because one of 12/15/2022 5:36:17 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.1753012110788 (T) = (0 -15.422564823303567) / Math.Sqrt((0.25921905648871046 / (299)) + (0.04782969716474922 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09187100682294036 = (16.982790924170615 - 15.422564823303567) / 16.982790924170615 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.987271706466165 < 14.776056839048355.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 51.7138328095309 (T) = (0 -13.019383392617534) / Math.Sqrt((0.16424109447654006 / (299)) + (0.057027568253775286 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.17684549796140678 = (15.816451662931106 - 13.019383392617534) / 15.816451662931106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.676603408544269 < 14.977581388211503.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.973850683377762 (T) = (0 -14.587225749095879) / Math.Sqrt((0.2812535267014646 / (299)) + (0.03854007387913899 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09394914982638208 = (16.099787055329916 - 14.587225749095879) / 16.099787055329916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.332006710211033 < 14.920374982159117.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.243231928891113 (T) = (0 -14.359110790668488) / Math.Sqrt((0.22305493546045865 / (299)) + (0.10480020846297033 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10373592833784336 = (16.021071517503717 - 14.359110790668488) / 16.021071517503717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.513210511094854 < 14.330249839058952.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.305816395227247 (T) = (0 -13.323447803921752) / Math.Sqrt((0.5288438604361646 / (299)) + (0.1622925556871543 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14419943006507827 = (15.56840258348381 - 13.323447803921752) / 15.56840258348381 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.691031630060925 < 15.723930686556109.
IsChangePoint: Marked as a change because one of 11/25/2022 10:05:08 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 38.16820830793088 (T) = (0 -14.882288000395835) / Math.Sqrt((0.18532938793041726 / (299)) + (0.06137484300047688 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.12450728638600075 = (16.99875712153256 - 14.882288000395835) / 16.99875712153256 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.125103358648826 < 7.517103401187634.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 23.340833331947923 (T) = (0 -6.168850157546557) / Math.Sqrt((0.29527649038470183 / (299)) + (0.12193953461178107 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.22764770334677978 = (7.987093693224712 - 6.168850157546557) / 7.987093693224712 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Mar 14, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsBenchmark - Duration of single invocation 11.35 ns 8.20 ns 0.72 0.09 False
GreaterThanBenchmark - Duration of single invocation 15.69 ns 13.96 ns 0.89 0.03 False
AndNotBenchmark - Duration of single invocation 14.54 ns 13.15 ns 0.90 0.05 False
MaxBenchmark - Duration of single invocation 15.70 ns 14.28 ns 0.91 0.02 False
MultiplyBenchmark - Duration of single invocation 18.62 ns 14.82 ns 0.80 0.04 False
MultiplyOperatorBenchmark - Duration of single invocation 16.43 ns 14.56 ns 0.89 0.04 False
LessThanBenchmark - Duration of single invocation 15.04 ns 13.28 ns 0.88 0.03 False
XorBenchmark - Duration of single invocation 15.54 ns 12.85 ns 0.83 0.02 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.39 ns 12.86 ns 0.89 0.02 False
EqualsStaticBenchmark - Duration of single invocation 15.08 ns 13.23 ns 0.88 0.02 False
EqualsAnyBenchmark - Duration of single invocation 7.64 ns 6.05 ns 0.79 0.16 False
ConditionalSelectBenchmark - Duration of single invocation 16.78 ns 15.48 ns 0.92 0.05 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.22 ns 12.86 ns 0.90 0.02 False
MinBenchmark - Duration of single invocation 15.88 ns 14.36 ns 0.90 0.01 False
DivideBenchmark - Duration of single invocation 19.09 ns 15.79 ns 0.83 0.02 False
AddBenchmark - Duration of single invocation 17.39 ns 13.99 ns 0.80 0.07 False
EqualsAllBenchmark - Duration of single invocation 7.94 ns 4.03 ns 0.51 0.37 False
AddOperatorBenchmark - Duration of single invocation 15.75 ns 13.71 ns 0.87 0.04 False
LessThanOrEqualBenchmark - Duration of single invocation 15.39 ns 13.68 ns 0.89 0.04 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.33 ns 12.81 ns 0.89 0.02 False
SubtractBenchmark - Duration of single invocation 17.33 ns 13.71 ns 0.79 0.08 False
SubtractionOperatorBenchmark - Duration of single invocation 15.75 ns 14.08 ns 0.89 0.04 False
GreaterThanOrEqualBenchmark - Duration of single invocation 16.22 ns 14.12 ns 0.87 0.06 False
LessThanAnyBenchmark - Duration of single invocation 7.94 ns 6.53 ns 0.82 0.21 False
BitwiseOrBenchmark - Duration of single invocation 14.64 ns 12.97 ns 0.89 0.03 False
BitwiseAndBenchmark - Duration of single invocation 15.50 ns 12.83 ns 0.83 0.02 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualsBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.200015675088284 < 10.421441565102525.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.02730669933758 (T) = (0 -7.810214299046476) / Math.Sqrt((1.505600347159905 / (299)) + (0.17546483756007272 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2302673321098516 = (10.14665821635766 - 7.810214299046476) / 10.14665821635766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.963239779105617 < 14.901986477954214.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 37.975401682004595 (T) = (0 -13.826899963910018) / Math.Sqrt((0.6142300335288363 / (299)) + (0.048134293814442064 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14893498081865286 = (16.246584752373362 - 13.826899963910018) / 16.246584752373362 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.145098831233582 < 13.83000990437817.
IsChangePoint: Marked as a change because one of 11/24/2022 8:21:24 PM, 12/5/2022 5:12:51 PM, 12/15/2022 5:36:17 AM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 57.67028713046489 (T) = (0 -13.102364650153781) / Math.Sqrt((0.13553030200081803 / (299)) + (0.005355949720549903 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10337628372895431 = (14.613002547651762 - 13.102364650153781) / 14.613002547651762 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.27943164668524 < 14.915640655317123.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 35.906871681522816 (T) = (0 -14.295148393769063) / Math.Sqrt((0.2524317755239559 / (299)) + (0.035717091739443305 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10818594357789335 = (16.0292925311361 - 14.295148393769063) / 16.0292925311361 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.821278253489636 < 17.686149279414188.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 82.70895456079097 (T) = (0 -14.777282488354437) / Math.Sqrt((0.07782516888035473 / (299)) + (0.0492529399024713 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.21207244352304117 = (18.754620734966725 - 14.777282488354437) / 18.754620734966725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.56182252584918 < 15.881254990501864.
IsChangePoint: Marked as a change because one of 11/25/2022 6:33:31 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 21.772223573683117 (T) = (0 -14.856019724007757) / Math.Sqrt((0.09094458805278793 / (299)) + (0.1915139918008825 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.11566511428266134 = (16.799088177955483 - 14.856019724007757) / 16.799088177955483 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.278768611495455 < 14.296018922333054.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 27.119792371164454 (T) = (0 -13.371736056634411) / Math.Sqrt((0.46319497390060993 / (299)) + (0.11646123728267913 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.13961691589168618 = (15.541607341679256 - 13.371736056634411) / 15.541607341679256 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.849005794048177 < 14.746697745867063.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 72.90631249412047 (T) = (0 -12.970318574278437) / Math.Sqrt((0.18878936364306229 / (299)) + (0.020828668726407065 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.17874420038489963 = (15.793274860716068 - 12.970318574278437) / 15.793274860716068 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.85517595836382 < 13.629029613635836.
IsChangePoint: Marked as a change because one of 12/5/2022 3:49:45 AM, 1/5/2023 4:05:11 PM, 3/6/2023 6:03:21 AM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 30.458356634426035 (T) = (0 -12.953644500217491) / Math.Sqrt((0.09370729374336689 / (299)) + (0.06043230617285613 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10774961582536642 = (14.5179477980277 - 12.953644500217491) / 14.5179477980277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.227535064920072 < 14.319853778432824.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 49.28334567295586 (T) = (0 -13.46383452502875) / Math.Sqrt((0.23081629823299094 / (299)) + (0.018562983902144333 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1257961094081694 = (15.40125212198932 - 13.46383452502875) / 15.40125212198932 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.0523331430048986 < 7.278503510916273.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.900010605779215 (T) = (0 -6.3358023129201735) / Math.Sqrt((0.279578238598325 / (299)) + (0.07656809862353016 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.20795912769550748 = (7.999337577725957 - 6.3358023129201735) / 7.999337577725957 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.476837984162602 < 15.892993951392567.
IsChangePoint: Marked as a change because one of 12/14/2022 8:02:51 PM, 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 29.757446668432223 (T) = (0 -15.397286657766685) / Math.Sqrt((0.2912879398502072 / (299)) + (0.043399354602249564 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09251450094230854 = (16.966978176240627 - 15.397286657766685) / 16.966978176240627 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.860493091630046 < 13.696768222581387.
IsChangePoint: Marked as a change because one of 1/3/2023 9:28:36 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 31.26522069571393 (T) = (0 -12.944402817409623) / Math.Sqrt((0.15514038835597185 / (299)) + (0.050494664813130606 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11008120010093136 = (14.545599911899524 - 12.944402817409623) / 14.545599911899524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.358770760606566 < 14.954499867817796.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 33.06906081515443 (T) = (0 -14.567874946644979) / Math.Sqrt((0.2692879437432484 / (299)) + (0.03191776379222843 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09682794356663746 = (16.129678551143062 - 14.567874946644979) / 16.129678551143062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.785458425249377 < 18.507539938528854.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 46.00217199344628 (T) = (0 -15.688928693671714) / Math.Sqrt((0.12383408411776987 / (299)) + (0.1560632579732296 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.19604990652105642 = (19.514804240871236 - 15.688928693671714) / 19.514804240871236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.990156348964154 < 16.60888275438415.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 43.879710598179706 (T) = (0 -13.958443822504876) / Math.Sqrt((0.10632513723228905 / (299)) + (0.15656750604202488 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.206811113404426 = (17.59788123408481 - 13.958443822504876) / 17.59788123408481 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.033784899779511 < 7.640871552789316.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.652600492582376 (T) = (0 -4.578247157230529) / Math.Sqrt((0.3433879201466459 / (299)) + (0.39970601372983633 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.41808696166690296 = (7.867579613519266 - 4.578247157230529) / 7.867579613519266 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.709243706922168 < 14.896256617552728.
IsChangePoint: Marked as a change because one of 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 45.31157729760718 (T) = (0 -13.877621927963247) / Math.Sqrt((0.1173105379955183 / (299)) + (0.035063121844457486 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12323919722112742 = (15.828287354975785 - 13.877621927963247) / 15.828287354975785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.682603977323012 < 14.678282214462259.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 27.779866147644434 (T) = (0 -13.833852361275683) / Math.Sqrt((0.23333788736779987 / (299)) + (0.0945868185926547 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12125273279957256 = (15.742697448549123 - 13.833852361275683) / 15.742697448549123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.814319670612509 < 13.643702459990823.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 47.821883281972895 (T) = (0 -12.945402832064964) / Math.Sqrt((0.09756105605223124 / (299)) + (0.017379500983371182 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10692605346693448 = (14.495331413842413 - 12.945402832064964) / 14.495331413842413 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.708122573965525 < 16.46176874848641.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 87.1224209987901 (T) = (0 -13.95136915617303) / Math.Sqrt((0.1632094585162487 / (299)) + (0.03199617108291366 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2130287545655545 = (17.727927465089543 - 13.95136915617303) / 17.727927465089543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.075633239154905 < 14.944438074502708.
IsChangePoint: Marked as a change because one of 12/30/2022 6:15:13 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 39.8894064399324 (T) = (0 -13.931496439960497) / Math.Sqrt((0.10259860248433245 / (299)) + (0.045102884591429464 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11892817958648083 = (15.811987305895151 - 13.931496439960497) / 15.811987305895151 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.123007607430532 < 15.323611488626295.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 26.79227843112542 (T) = (0 -14.25887982906272) / Math.Sqrt((0.24097173255862614 / (299)) + (0.13622686514348575 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1314012395209452 = (16.415956915709362 - 14.25887982906272) / 16.415956915709362 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.530544090789055 < 7.388036407578781.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/30/2022 6:06:02 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.59137892554157 (T) = (0 -6.270509018414366) / Math.Sqrt((0.34196648184794626 / (299)) + (0.08889692813954023 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.21448782028777622 = (7.982701198486314 - 6.270509018414366) / 7.982701198486314 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.969125493235673 < 14.772748194567988.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 71.14465083132806 (T) = (0 -12.981986568145409) / Math.Sqrt((0.21374500905418814 / (299)) + (0.02021790691692942 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.17781010913211076 = (15.789523457217227 - 12.981986568145409) / 15.789523457217227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.827959253395086 < 14.726423144251479.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 44.032025409078905 (T) = (0 -12.822719231381685) / Math.Sqrt((0.2079547334984926 / (299)) + (0.09147373504554422 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1901063917638151 = (15.832597147306124 - 12.822719231381685) / 15.832597147306124 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6123cb00bbec8ae3500c970890f5b0efcf7950e8
Compare 795a2875908e26369011100f9882e188f2de8fab
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyOperatorBenchmark - Duration of single invocation 55.97 ns 51.37 ns 0.92 0.00 False
AddOperatorBenchmark - Duration of single invocation 55.68 ns 48.81 ns 0.88 0.01 False
GreaterThanOrEqualBenchmark - Duration of single invocation 67.98 ns 62.84 ns 0.92 0.01 False
DotBenchmark - Duration of single invocation 44.03 ns 39.16 ns 0.89 0.01 False
EqualsStaticBenchmark - Duration of single invocation 60.25 ns 52.74 ns 0.88 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 39.90 ns 34.25 ns 0.86 0.02 False
BitwiseAndOperatorBenchmark - Duration of single invocation 14.40 ns 13.15 ns 0.91 0.02 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 43.08 ns 38.87 ns 0.90 0.01 False
LessThanBenchmark - Duration of single invocation 59.26 ns 52.78 ns 0.89 0.01 False
EqualsAnyBenchmark - Duration of single invocation 40.15 ns 34.98 ns 0.87 0.01 False
BitwiseAndBenchmark - Duration of single invocation 15.46 ns 12.76 ns 0.83 0.03 False
UnaryNegateOperatorBenchmark - Duration of single invocation 50.43 ns 46.76 ns 0.93 0.01 False
SubtractBenchmark - Duration of single invocation 55.79 ns 50.03 ns 0.90 0.01 False
DivideBenchmark - Duration of single invocation 104.40 ns 95.03 ns 0.91 0.00 False
MultiplyBenchmark - Duration of single invocation 56.95 ns 51.86 ns 0.91 0.00 False
XorBenchmark - Duration of single invocation 15.49 ns 12.84 ns 0.83 0.02 False
DivisionOperatorBenchmark - Duration of single invocation 106.33 ns 96.42 ns 0.91 0.00 False
BitwiseOrBenchmark - Duration of single invocation 15.56 ns 12.75 ns 0.82 0.02 False
GreaterThanBenchmark - Duration of single invocation 62.89 ns 56.95 ns 0.91 0.00 False
InequalityOperatorBenchmark - Duration of single invocation 6.25 ns 4.98 ns 0.80 0.26 False
BitwiseOrOperatorBenchmark - Duration of single invocation 14.40 ns 12.77 ns 0.89 0.03 False
LessThanOrEqualBenchmark - Duration of single invocation 64.33 ns 59.66 ns 0.93 0.01 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 14.25 ns 13.00 ns 0.91 0.01 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 43.80 ns 38.76 ns 0.88 0.01 False
EqualsAllBenchmark - Duration of single invocation 8.13 ns 4.70 ns 0.58 0.28 False
ConditionalSelectBenchmark - Duration of single invocation 16.81 ns 15.48 ns 0.92 0.04 False
LessThanAnyBenchmark - Duration of single invocation 39.85 ns 34.79 ns 0.87 0.02 False
SubtractionOperatorBenchmark - Duration of single invocation 55.65 ns 49.23 ns 0.88 0.01 False
AddBenchmark - Duration of single invocation 55.73 ns 49.22 ns 0.88 0.00 False
AndNotBenchmark - Duration of single invocation 14.51 ns 13.04 ns 0.90 0.05 False
EqualsBenchmark - Duration of single invocation 10.64 ns 8.57 ns 0.81 0.18 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

  • Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
  • CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
  • AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
  • Dotnet SDK installed for dotnet commands
  • Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.MultiplyOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.369292207672586 < 53.18156166399078.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 33.66067151124074 (T) = (0 -51.275329325268864) / Math.Sqrt((2.349152109127683 / (299)) + (0.16734597928999495 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07402462338192607 = (55.374398304781046 - 51.275329325268864) / 55.374398304781046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.80563534108698 < 53.040315311610314.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 38.97065907256369 (T) = (0 -49.67052509745498) / Math.Sqrt((2.435422767017126 / (299)) + (0.12335006346517297 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08293131992995918 = (54.16227396803193 - 49.67052509745498) / 54.16227396803193 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 62.83554273171722 < 64.61238786839365.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 16.8726305306538 (T) = (0 -63.1844284788463) / Math.Sqrt((2.705856290244962 / (299)) + (0.882104907486908 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.05494503741566857 = (66.85794052238319 - 63.1844284788463) / 66.85794052238319 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.16440154094248 < 41.74655041918369.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 36.241068582525486 (T) = (0 -39.47668487226775) / Math.Sqrt((2.778173569590402 / (299)) + (0.15940996937239757 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10384826971663343 = (44.051340345886594 - 39.47668487226775) / 44.051340345886594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.742224335806235 < 56.034377517522955.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 38.47246334573788 (T) = (0 -53.405696244645696) / Math.Sqrt((2.02033124232193 / (299)) + (0.27064745646165417 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.08719382480121525 = (58.50715923675184 - 53.405696244645696) / 58.50715923675184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.251583153961896 < 37.71967391007565.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.524793485297977 (T) = (0 -34.72570200912495) / Math.Sqrt((3.8525215538475135 / (299)) + (0.08588135479115289 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09534739277829522 = (38.385676150065706 - 34.72570200912495) / 38.385676150065706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.145587000252279 < 13.677079231830483.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 42.49009881327481 (T) = (0 -12.975717917082221) / Math.Sqrt((0.27943364897766504 / (299)) + (0.015150102450562858 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11471253575056342 = (14.657067270328152 - 12.975717917082221) / 14.657067270328152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.874334061039576 < 41.570971535262544.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.341090376257917 (T) = (0 -38.721794435894545) / Math.Sqrt((3.911080059971061 / (299)) + (0.044772928602144046 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08233492443620165 = (42.195998809374444 - 38.721794435894545) / 42.195998809374444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.77956139404402 < 56.15454914474728.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 39.00666987016367 (T) = (0 -53.30948151212214) / Math.Sqrt((2.720851015511354 / (299)) + (0.17747369645960742 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08590099564422034 = (58.319154990976635 - 53.30948151212214) / 58.319154990976635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.97611063222269 < 37.81877190079954.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.15755212528993 (T) = (0 -34.55965809895789) / Math.Sqrt((4.199520380629903 / (299)) + (0.08866498053851302 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0978937520075568 = (38.3099642374358 - 34.55965809895789) / 38.3099642374358 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.760880404369837 < 14.70283974989786.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 72.68050222480784 (T) = (0 -12.992656184115896) / Math.Sqrt((0.20642878074336893 / (299)) + (0.020118466518516537 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1778356924076411 = (15.802992253657703 - 12.992656184115896) / 15.802992253657703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.75509854217727 < 47.6558957096803.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.433596182527328 (T) = (0 -46.7182737126229) / Math.Sqrt((2.0967021205017375 / (299)) + (0.06124119269294493 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05054932955956313 = (49.20558294087143 - 46.7182737126229) / 49.20558294087143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.03394541436698 < 52.588592745808235.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 55.834441855458174 (T) = (0 -49.55544445708759) / Math.Sqrt((0.30179785375241763 / (299)) + (0.22071244278250007 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10219178225335837 = (55.19602458247042 - 49.55544445708759) / 55.19602458247042 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 95.02522809540855 < 99.17643301317595.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 24.302437128225872 (T) = (0 -96.713577287044) / Math.Sqrt((1.5259696254679893 / (299)) + (2.4346246426239246 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07580455962848276 = (104.64623938002347 - 96.713577287044) / 104.64623938002347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.86463284334812 < 54.03077450758108.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 60.39708684099996 (T) = (0 -51.39675613006924) / Math.Sqrt((0.13400621264556625 / (299)) + (0.16584389493860852 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09157164103337827 = (56.57766583656126 - 51.39675613006924) / 56.57766583656126 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.XorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.8396284555758 < 14.72168108480358.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 51.44249259121696 (T) = (0 -12.95232659102313) / Math.Sqrt((0.42391122708295464 / (299)) + (0.04764215578266488 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18810336360202123 = (15.953171882182927 - 12.95232659102313) / 15.953171882182927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 96.41875253251698 < 99.30484317941578.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 15.872495937060737 (T) = (0 -96.98820963349625) / Math.Sqrt((8.327599250000672 / (299)) + (3.5442487343414872 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06416516991091542 = (103.63817044965477 - 96.98820963349625) / 103.63817044965477 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.75331383236653 < 14.680368728949599.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 51.81453547379375 (T) = (0 -12.912159318897512) / Math.Sqrt((0.4490599402125833 / (299)) + (0.04979710432710194 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.19171767564764206 = (15.974813415898304 - 12.912159318897512) / 15.974813415898304 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 56.950164547152596 < 61.21587544362412.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 16.34169555280694 (T) = (0 -58.20082182521611) / Math.Sqrt((2.515919402769772 / (299)) + (1.6979773752929932 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07321647346278107 = (62.79872285028019 - 58.20082182521611) / 62.79872285028019 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.9761153538393925 < 5.9568797836843475.
IsChangePoint: Marked as a change because one of 1/15/2023 2:31:53 AM, 3/5/2023 5:50:49 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 7.506435742207858 (T) = (0 -5.521107604312509) / Math.Sqrt((0.15123145153410372 / (299)) + (0.19128401389847244 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1059064461504952 = (6.17508937464259 - 5.521107604312509) / 6.17508937464259 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.76642234799306 < 13.683537544683059.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 30.474563458308314 (T) = (0 -12.910582453960222) / Math.Sqrt((0.2983819501310992 / (299)) + (0.05512980173683812 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11932599698232495 = (14.659888233014081 - 12.910582453960222) / 14.659888233014081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 59.659231074739594 < 61.11991858378933.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.598022045101818 (T) = (0 -59.68250074308376) / Math.Sqrt((1.980824211997406 / (299)) + (0.3423564086118463 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.058372010851302954 = (63.382250135790095 - 59.68250074308376) / 63.382250135790095 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.002050984060848 < 13.538762762798473.
IsChangePoint: Marked as a change because one of 1/5/2023 11:12:07 AM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 48.13851328510304 (T) = (0 -12.98701965503074) / Math.Sqrt((0.09771400149863856 / (299)) + (0.01641254066367706 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1054159987790155 = (14.517384211326426 - 12.98701965503074) / 14.517384211326426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.76044873946418 < 41.25299809066805.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 25.483912427993445 (T) = (0 -38.61196081379689) / Math.Sqrt((3.889539524817637 / (299)) + (0.12632563824364923 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08190763719703847 = (42.056728035416285 - 38.61196081379689) / 42.056728035416285 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.702623032382997 < 7.840929227842613.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 37.11368035132895 (T) = (0 -4.896031915382687) / Math.Sqrt((0.23789188347025778 / (299)) + (0.16795861943141266 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.40091685885874634 = (8.172541637636044 - 4.896031915382687) / 8.172541637636044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.484349109586313 < 15.768201948401508.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 49.0602513190427 (T) = (0 -15.421162605820244) / Math.Sqrt((0.10636874185865727 / (299)) + (0.01066109274744676 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08255213638071424 = (16.80876180253397 - 15.421162605820244) / 16.80876180253397 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.79151423148453 < 37.961325831208626.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 28.891539741340548 (T) = (0 -34.42413376534631) / Math.Sqrt((4.148615436516157 / (299)) + (0.1033390393194815 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10166091097492524 = (38.31975496324579 - 34.42413376534631) / 38.31975496324579 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.22568123190136 < 52.011700272560844.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 36.181941151982585 (T) = (0 -49.57930751252449) / Math.Sqrt((2.4967198382481457 / (299)) + (0.17647455932151426 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08378827890003192 = (54.113373984128366 - 49.57930751252449) / 54.113373984128366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.22105538205954 < 52.902329318395736.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 71.28429893548545 (T) = (0 -49.491134415988036) / Math.Sqrt((0.21946259038445767 / (299)) + (0.13510206149888204 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10305551318264186 = (55.17747769608149 - 49.491134415988036) / 55.17747769608149 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.AndNotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.042478910979693 < 13.785238617776836.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 49.67618159143723 (T) = (0 -13.11152806135832) / Math.Sqrt((0.10646512873014 / (299)) + (0.014387856538615653 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10483969715578204 = (14.647128586576832 - 13.11152806135832) / 14.647128586576832 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.56672757154913 < 10.386925167946758.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 3/6/2023 4:10:52 PM, 3/14/2023 1:07:29 AM falls between 3/5/2023 5:11:19 AM and 3/14/2023 1:07:29 AM.
IsImprovementStdDev: Marked as improvement because 19.25286533157259 (T) = (0 -8.666096073705008) / Math.Sqrt((1.030682163901027 / (299)) + (0.06177808098542472 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14704173364940498 = (10.16004699829352 - 8.666096073705008) / 10.16004699829352 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@fanyang-mono fanyang-mono mentioned this issue Mar 17, 2023
Merged
@kotlarmilos kotlarmilos mentioned this issue Apr 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-improvement runtime-mono
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos