[Perf -25%] PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj

[DrewScoggins]

Run Information

Architecture	x64
OS	Windows 10.0.18362
Changes	diff

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark	Baseline	Test	Test/Base	Modality	Baseline Outlier
ScalarValueTypeObj	374.67 μs	467.74 μs	1.25		True

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*';

Histogram

PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj

[365755.713 ; 383072.269) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[383072.269 ; 399898.343) | 
[399898.343 ; 416724.417) | 
[416724.417 ; 433550.491) | 
[433550.491 ; 450376.565) | 
[450376.565 ; 458866.791) | 
[458866.791 ; 476163.417) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[DrewScoggins]

This will likely need further investigation. As can be seen from the graph we moved into a mode where we started moving between two stable points in the benchmark. It would probably be interesting to see if we have a assembly difference here or if this is just an alignment issue.

[stephentoub]

cc: @VSadov

[mangod9]

could be related to #37803

[mangod9]

Assuming this is similar to 37803, and the perf difference is reasonable.

[VSadov]

This is a trivial Unbox. JIT inlines the type check and if there is a match (which is the case for the benchmark), it copies the value out without calling any helpers since the value here contains no GC references. I checked the codegen to be sure.

I do not think the code changed. The perf has not technically regressed, just went bimodal.
It could be some kind of alignment influence that depends on time when the code is JITted or the location of the static variables.

The perf seems acceptable. This is a very fast scenario similar to 37803.

[adamsitnik]

I don't have enough data to be 100% sure that the benchmark is not just multimodal, but the data that I have so far tells that it might be Windows x64 specific issue. I am going to update it later this week when I have data from more runs and machines

PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj

Conclusion	Base	Diff	Base/Diff	Modality	Operating System	Bit	Processor Name	Base Runtime	Diff Runtime
Same	621504.33	591798.80	1.05		Windows 10.0.18363.1016	Arm	Microsoft SQ1 3.0 GHz	.NET Core 3.1.6	5.0.100-rc.1.20413.9
Slower	191036.55	238074.53	0.80		Windows 10.0.18363.959	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20404.3
Slower	193446.84	232128.75	0.83		Windows 10.0.18363.959	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20418.3
Slower	222340.49	266385.96	0.83		Windows 10.0.19041.450	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	.NET Core 3.1.6	5.0.100-rc.1.20413.9
Same	229034.67	228314.68	1.00		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20403.23
Same	265719.35	265708.50	1.00		macOS Mojave 10.14.5	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	.NET Core 3.1.6	5.0.100-rc.1.20404.2

[mangod9]

@VSadov is there anything which needs to be investigated for .net 5 here?

[VSadov]

@mangod9 - I do not think there is something actionable further.