Stabilize performance measurement

[kunalspathak]

The code quality and performance of RyuJIT is tracked internally by running MicroBenchmarks in our performance lab. We regularly triage the performance issues opened by the .NET performance team. After going through these issues for past several months, we have identified some key points.

Stability

Many times, the set of commits that are flagged as introducing regression in a benchmark, do not touch the code that is tested in the benchmark. In fact, the assembly code generated for the .NET code that is being tested is often identical and yet the measurements show differences. Some of our investigation reveals that the fluctuation in the benchmark measurements happen because of the misalignment of generated JIT code in process memory. Below is an example of LoopReturn benchmark that shows such behavior.

It is very time consuming for .NET developers to do the analysis of benchmarks that regressed because of things that are out of control of .NET runtime. In the past, we have closed several issues like #13770, #39721 and #39722 because they were regressions because of code alignment. A great example that we found out while investigating those issues was the change introduced in #38586 eliminated a test instruction and should have showed improvement in the benchmarks, but introduced regression because the code (loop code inside method) now gets misaligned and the method runs slower.

Alignment issues was brought up few times in #9912 and #8108 and this issue tracks the progress towards the goal of stabilizing and possibly improving the performance of .NET apps that are heavily affected because of code alignment.

Performance lab infrastructure

Once we address the code alignment issue, the next big thing will be to identify and make required infrastructure changes in our performance lab to make sure that it can easily flag such issues without needing much interaction from .NET developers. For example, dotnet/BenchmarkDotNet#1513 proposes to make memory alignment in the benchmark run random to catch these issues early and once we address the underlying problem in .NET, we should never see bimodal behavior of those benchmarks. After that, if the performance lab does find a regression in the benchmark, we need to have robust tooling support to get possible metrics from performance runs so that a developer doing the investigation can easily identify the cause of regression. For example, identifying the time spent in various phases of .NET runtime like Jitting, Jit interface, Tier0/Tier1 JIT code, hot methods, instructions retired during benchmark execution and so forth.

Reliable benchmarks collection

Lastly, for developers working on JIT, we want to identify set of benchmarks that are stable enough and can be trusted to give us reliable measurement whenever there is a need to verify the performance for changes done to the JIT codebase. This will help us conduct performance testing ahead of time and identify potential regressions rather than waiting it to happen in performance lab.

Here are set of work items that we have identified to achieve all the above:

Code alignment work

• Make method entries of JIT code having loops aligned at 32-byte boundary for xarch. (Done in Fix condition to align methods to 32 bytes boundary #42909)
• Devise a mechanism to identify inner loops (the ones that don't have more loops inside) present in a method. (Done: Align inner loops #44370)
• Experiment with heuristics that should decide if a particular inner loop needs alignment or not. (Done: Align inner loops #44370)
• Calculate the padding that needs to be added to align the identified inner loop. (Done: Align inner loops #44370)
• Basic: Add the padding near the loop header (either at the end of previous block or at the beginning of the loop header). (Done: Align inner loops #44370)
• Before merging above work, measure the impact of loop alignment on Microbenchmarks. Tracked in Measure loop alignment's performance impact on Microbenchmarks #44051. Also in Stabilize performance measurement #43227 (comment)
• Identify benchmarks that get stabilized or whose performance is improved by the above work. (Update: See the analysis starting from Stabilize performance measurement #43227 (comment))

Future work

• Make method entries of JIT code having loops aligned at relevant boundary for arm32/arm64. See section 4.8 of https://developer.arm.com/documentation/swog309707/a.
• Combine branch tightening and loop alignment adjustments in single phase.
• Advanced: If padding proves to be costly, have a way to spread the padding throughout the method so the loop header gets aligned. Account the padding while doing branch tensioning.
  • Add padding at the blind spot like after jmp or ret instruction that comes before align instruction.
  • Add padding at the end of blocks that has lower weight than the block that precedes the loop header block.
  • Explore option to have a method misaligned such that we can skip the padding needed for loops and they get auto-aligned or aligned with minimal padding.
• Perform loop alignment similar to Align inner loops #44370 for R2R code.
• For x86, we should improve the encoding we use for multiple size NOP instructions. Today, we just output repeated single byte 90 , but could do better like we do for x64.
• Explore option to align enclosing loops if it can borrow some of the padding needed for inner loop. E.g. in a nested loop, if there is an inner loop that needs padding of 10 bytes and the outer loop can be aligned by adding padding of 4 bytes, then add padding of 4 bytes to outer loop and 6 bytes to inner loop. That way, both the loops are aligned.

Performance tooling work

• Add random memory alignment. Tracked by Make memory alignment more random BenchmarkDotNet#1513 and Refactor initialization logic to allow for enabling Memory Randomization performance#1587
• Add various tooling in PerfView, BenchmarkDotNet analyzers to add the required metric. (WIP)
• Perform superpmi collection of microbenchmarks so codegen team can easily test the impact of their change on generated code of various benchmarks. (Superpmi on Microbenchmarks #47900)
• Add a new event to track memory allocated for JIT code that can be surfaced in PerfView through JITStats. PR: New event to track memory allocation for JIT code #44030 and Display JIT Allocated Heap Size microsoft/perfview#1289

[kunalspathak]

@dotnet/jit-contrib , @danmosemsft

[danmoseley]

cc @DrewScoggins @billwert @adamsitnik

[danmoseley]

To state the obvious (I think) -- I believe we have good data to suggest that alignment is the dominant reason for bimodality. I'm not sure though that we can be sure there aren't other common causes for bimodality -- my assumption is that we'll find out how much is left when you've completed some of this work.

[kunalspathak]

To state the obvious (I think) -- I believe we have good data to suggest that alignment is the dominant reason for bimodality. I'm not sure though that we can be sure there aren't other common causes for bimodality -- my assumption is that we'll find out how much is left when you've completed some of this work.

Yes, there will definitely be more reason for bimodality, but alignment will fix most of the obvious ones that we know of and then, it will be easier for us to focus one remaining ones. Currently, there are just too many bimodal benchmarks.

[JulieLeeMSFT]

Bolded WIP texts in the plan.

[danmoseley]

As an aside, is there more than BDN can do to help regularize data alignment, when the buffer is allocated by the test and reused?

Never mind, I forgot about dotnet/performance#1587 etc. we're attacking both sides. 😊

[kunalspathak]

Another round of analysis.

Stability

I went through the stability data for windows x64 and linux x64 that @DrewScoggins shared with me. In this report, "regression" means that loop alignment introduced more variance to the measurements and thus made the benchmark instable, while "improvements" meant that the loop alignment work actually stabilized the benchmark and reduced the variance. I analyzed around 150 benchmarks to see if the regressions were real by checking at the historical data of Windows x64 and Linux x64 for each of the 150 benchmark. If I see that the benchmark had lot of variance historically, I concluded that the loop alignment didn't added to the variance. There were few exceptions like Perf_Enumerable.Count and Single.Max where post-loop alignment we saw some variance, but other than that, most of the regression pointed out in report were not related to the loop alignment work.

Performance

Here are some of the take away points:

• JIT maintains a loop table to track all the loops it has seen and use it to perform various optimizations and analysis. It turns out that there is a bug (or a missing feature, whatever you call) where we do not record the cloned loop inside loop table. With that, we miss out opportunities to do the required analysis on cloned loops. Tracking issue: Certain loops do not get recorded in optLoopTable #43713

• We sometimes add "NOP compensation" for the over-estimated instruction. Most of the over-estimated instructions are jumps and that that happens because branch tightening happens before loop alignment adjustment phase. In branch tightening we adjust the size of jump instructions depending on the target offset, but later, the loop alignment adjustment can further shorten the IG offsets resulting in over-estimated jump instruction. To address this, we need to combine branch tightening and loop alignment adjustment in same phase rather than separating them. I have updated the work item list to capture this explicitly. After doing this, we might further think about adding the compensation behind jmp or in cold block.

• Often the encoding of jump instruction before vs. after loop alignment might increase because of added padding instructions. I do not think we can do much about it.

FannkuchRedux_9

The regression happens because of extra padding we added to align some of the loops inside benchmark code. Asmdiffs: https://www.diffchecker.com/ryZqqO6I.

MulMatrix

The regression happens because of addition of 5 "NOP overestimate compensation" in hot blocks. The overestimation happened for jump instructions. The way to address is by combining branch tightening with loop alignment adjustments (already planned work). Asmdiffs: https://www.diffchecker.com/IcktyNPi

TryGetUInt16:

In this method, TryParseUInt16D() is the hot method as seen from VTune screenshot:

With loop alignment, we ended up adding padding at the end of hot block that VTune pointed above (Block 2).
Asmdiffs: https://www.diffchecker.com/jHt5pv99

GetTypeCode

Although this shows regression, I verified that there is no loop inside the benchmark, nor a loop in the libraries method that it test. Further, I also compared the asm diffs and there is no "align" instruction.

ContainsTrue/ContainsFalse

This one is interesting. The hot method for all these benchmarks is GenericEqualityComparer`1[__Canon][System.__Canon]:IndexOf. In this method, we clone a loop, but later decide to not align it because it had call. However, we do not record that fact in the cloned loop because of which we have to align the cloned loop containing call. Additionally, until we emit the padding in the cloned loop, we have to also compensate the over-estimated instructions. One of the place where such overestimated instruction is compensated happen to be the actual hot loop that the method executes. You can see my detail analysis in #43713 (comment).

[JulieLeeMSFT]

Good analysis that loop alignment work didn’t introduce regression. What is cloned loop in the loop table? Thanks, Julie

…

________________________________ From: Kunal Pathak <notifications@github.com> Sent: Wednesday, February 3, 2021 4:04:47 PM To: dotnet/runtime <runtime@noreply.github.com> Cc: Julie Lee <Julie.Lee@microsoft.com>; State change <state_change@noreply.github.com> Subject: Re: [dotnet/runtime] Stabilize performance measurement (#43227) Another round of analysis. Stability I went through the stability data for windows x64<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Freports%2FallTestHistory%2Frefs%2Fheads%2Fmaster_x64_Windows%252010.0.18362%2FAllTestindex.html&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906853966%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JzoKgDiI6s4aPuzmEKLXLKYP4p6D3%2FFBUcfTHMqQTh8%3D&reserved=0> and linux x64<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Freports%2F01_18_2021%2Freport_Weekly_ca%3Dx64_cb%3Drefs-heads-master_co%3DUbuntu1804_cr%3Ddotnetruntime_cc%3DCompliationMode%3Dtiered-RunKind%3Dmicro_2021-01-18.html%23Improvements&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906863969%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0Sh17Vl9rR2A1VHzGPvVg6o4sdok3Oo8xHLIGOE1n68%3D&reserved=0> that @DrewScoggins<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FDrewScoggins&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906873964%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jC4vx7YwI%2BkeytbEAgtoYZ5olJYzSwRrMRIXtuxH9Yw%3D&reserved=0> shared with me. In this report, "regression" means that loop alignment introduced more variance to the measurements and thus made the benchmark instable, while "improvements" meant that the loop alignment work actually stabilized the benchmark and reduced the variance. I analyzed around 150 benchmarks to see if the regressions were real by checking at the historical data of Windows x64<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Freports%2FallTestHistory%2Frefs%2Fheads%2Fmaster_x64_Windows%252010.0.18362%2FAllTestindex.html&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906873964%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=a0d01xOt9ES%2FDK47JR%2Bkte69X56LDC8eFrZigjIkED8%3D&reserved=0> and Linux x64<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Freports%2FallTestHistory%2Frefs%2Fheads%2Fmaster_x64_ubuntu%252018.04%2FAllTestindex.html&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906883953%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=iQCzzCL2iCnirJk6AdsVNDHx9F3roGI1ooUe5yOwflY%3D&reserved=0> for each of the 150 benchmark. If I see that the benchmark had lot of variance historically, I concluded that the loop alignment didn't added to the variance. There were few exceptions like Perf_Enumerable.Count and Single.Max where post-loop alignment we saw some variance, but other than that, most of the regression pointed out in report were not related to the loop alignment work. Performance Here are some of the take away points: * JIT maintains a loop table to track all the loops it has seen and use it to perform various optimizations and analysis. It turns out that there is a bug (or a missing feature, whatever you call) where we do not record the cloned loop inside loop table. With that, we miss out opportunities to do the required analysis on cloned loops. Tracking issue: #43713<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdotnet%2Fruntime%2Fissues%2F43713&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906893953%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=J5enp6slL3eR8bofOKJRQ%2FPWD%2FcytuF5%2BJ%2FdZeAO%2FpE%3D&reserved=0> * We sometimes add "NOP compensation" for the over-estimated instruction. Most of the over-estimated instructions are jumps and that that happens because branch tightening happens before loop alignment adjustment phase. In branch tightening we adjust the size of jump instructions depending on the target offset, but later, the loop alignment adjustment can further shorten the IG offsets resulting in over-estimated jump instruction. To address this, we need to combine branch tightening and loop alignment adjustment in same phase rather than separating them. I have updated the work item list to capture this explicitly. After doing this, we might further think about adding the compensation behind jmp or in cold block. * Often the encoding of jump instruction before vs. after loop alignment might increase because of added padding instructions. I do not think we can do much about it. FannkuchRedux_9 [image]<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F12488060%2F106822899-dacafa00-6634-11eb-8cb1-b04ddde9263f.png&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906893953%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Q54HzNgvPNQTXtwAQuPmzfTRqjVfhARvx1IAfrLJqNQ%3D&reserved=0> The regression happens because of extra padding we added to align some of the loops inside benchmark code. Asmdiffs: https://www.diffchecker.com/ryZqqO6I<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.diffchecker.com%2FryZqqO6I&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906903947%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XFuE8AbjSsS%2B1%2Bpz4%2FrC9WrMKjw1s%2FvR8T6M9X6jSUg%3D&reserved=0>. MulMatrix [image]<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F12488060%2F106822997-0a7a0200-6635-11eb-8ba8-ef14f24e69b5.png&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906913945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=f9kEpNalgN%2BN97G1T%2FoizEOzW0RFIlhxPqz7MT3%2Bnf4%3D&reserved=0> The regression happens because of addition of 5 "NOP overestimate compensation" in hot blocks. The overestimation happened for jump instructions. The way to address is by combining branch tightening with loop alignment adjustments (already planned work). Asmdiffs: https://www.diffchecker.com/IcktyNPi<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.diffchecker.com%2FIcktyNPi&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906913945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rN6BDyNV4K0SxjzZspVE0HiLjWSTnNFxwg9qc72mFic%3D&reserved=0> TryGetUInt16: [image]<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F12488060%2F106823269-7b211e80-6635-11eb-8b4b-ac6e37c361bb.png&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906923938%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Hguv6A8Lir65jIlpbWgYcqk2n5ZClcGBnbkP8r9aoHU%3D&reserved=0> In this method, TryParseUInt16D() is the hot method as seen from VTune screenshot: [image]<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F12488060%2F106823363-a9066300-6635-11eb-9af3-66c59035bc18.png&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906933933%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=dDmX2yMppcyPM74kCdYxNOqxleIveSFuOrdGM9NNwGg%3D&reserved=0> [image]<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F12488060%2F106823367-ab68bd00-6635-11eb-91c3-7d0d4d80a5ac.png&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906933933%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8BT8fuTpmy05kjuMYqijLaCQvci84j9HcWexcFpXSw4%3D&reserved=0> With loop alignment, we ended up adding padding at the end of hot block that VTune pointed above (Block 2). Asmdiffs: https://www.diffchecker.com/jHt5pv99<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.diffchecker.com%2FjHt5pv99&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906943929%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=H0s2zpi8O9CZmNbx4LzRh%2BdCKb%2Fo8JaIkBc42CN81Us%3D&reserved=0> GetTypeCode [image]<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F12488060%2F106823797-62fdcf00-6636-11eb-88f6-c1158cdf8cd8.png&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906953929%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mFhfJ7Gil4MqAWvEbHRQp2Y1S2iHq1etBCHqYcYj%2F%2BU%3D&reserved=0> Although this shows regression, I verified that there is no loop inside the benchmark, nor a loop in the libraries method that it test. Further, I also compared the asm diffs and there is no "align" instruction. ContainsTrue/ContainsFalse [image]<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F12488060%2F106800888-e529cb80-6615-11eb-82d1-d6c35d93cbd8.png&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906953929%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=DmNJfqNMuKephBI0ZUD0V2Uj2US%2FN%2BMeKZqtNcah%2FkA%3D&reserved=0> [image]<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F12488060%2F106800925-f07cf700-6615-11eb-8363-9f4d4bf541d3.png&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906963923%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rumBw9wW%2FIhj%2BA%2BogbUobw%2BGY6v1YvFvMWajnDjdgfQ%3D&reserved=0> [image]<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F12488060%2F106800954-fb378c00-6615-11eb-9ae1-cbd1e33a749d.png&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906973917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6JSCGxq6Hdv7TJWfo3AZB3qxMd9dzkaOXbEG0IVGZJ0%3D&reserved=0> This one is interesting. The hot method for all these benchmarks is GenericEqualityComparer`1[__Canon][System.__Canon]:IndexOf<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsource.dot.net%2F%23System.Private.CoreLib%2Fsrc%2FSystem%2FCollections%2FGeneric%2FEqualityComparer.CoreCLR.cs%2C19&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906973917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=toPQAni6Sdcvh%2FhulUnwGD4NQDMT6uOHyPNo62h0VLo%3D&reserved=0>. In this method, we clone a loop, but later decide to not align it because it had call. However, we do not record that fact in the cloned loop because of which we have to align the cloned loop containing call. Additionally, until we emit the padding in the cloned loop, we have to also compensate the over-estimated instructions. One of the place where such overestimated instruction is compensated happen to be the actual hot loop that the method executes. You can see my detail analysis in #43713 (comment)<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdotnet%2Fruntime%2Fissues%2F43713%23issuecomment-772781240&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906983912%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=AKs3N1ZEhFiuSi0Dq%2BQ9Siw6pGkwZpX6gWduhLDE4jE%3D&reserved=0>. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdotnet%2Fruntime%2Fissues%2F43227%23issuecomment-772914110&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906983912%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=k0HxgSTDJXOxNGULPrXMlejqz%2FymJc%2Fy5AMHDYFvhcA%3D&reserved=0>, or unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPELRBZH4L5R756AJOYEI73S5HQB7ANCNFSM4SKMJJGA&data=04%7C01%7Cjulie.lee%40microsoft.com%7Cfe1560c80afe4e308df708d8c8a07c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637479938906993908%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Lrq9LPegknla%2BZBF8NfSKIb30FWp0pJkGAAevYlr5GA%3D&reserved=0>.

[kunalspathak]

What is cloned loop in the loop table? Thanks, Julie

Sometimes, if a loop has checks like array's bound check, we will clone a loop. One variant of the loop will have no bound checks (and hence will be fast loop) while other variant will have bound checks (and will be a slow loop). Depending on the input, we decide whether to execute fast loop or the cloned (slow) loop. You can see an example here. Here, fast loop is L003e ~ L0058 while cloned slow loop is L005c ~ L0080. During JIT, we track all the loops in a data structure called optLoopTable. Whenever we want to do any analysis on loops, we iterate over this table to decide whether the given loop needs any update/optimization. When we clone a loop, we do not inserted an entry of cloned loop inside optLoopTable because of which further analysis doesn't happen on cloned loop. Towards the end of my comment in #43713 (comment), I have listed what type of analysis we miss out on cloned loop because of lack of recording its entry inside optLoopTable.

[JulieLeeMSFT]

Moved future since .NET 6 scope items are complete now.

[MithrilMan]

I read your article here about loop alignment here https://devblogs.microsoft.com/dotnet/loop-alignment-in-net-6/ and it mentions that a loop like this

for (int l = 0; l < M; l++) {
        // body
        OtherMethod();
    }

isn't a candidate for inlining because contains a call to a method
What about methods that are inlined?
Shouldn't inline method be considered like loop body and be eligible for alignment if their body is small enough?

[kunalspathak]

What about methods that are inlined?

Inlining decision happens in early phase of compilation and when we decide whether to align a loop or not, we already know that it was inlined or not. We won't align the loop if we know for sure that the method will not be inlined.