JIT: Run new block layout only in backend

[amanasifkhalid]

Follow-up to #107483. This had a lot of churn in both directions locally, though layout doesn't seem to have changed.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[amanasifkhalid]

cc @dotnet/jit-contrib, @AndyAyersMS PTAL. Churn is quite big, and varies between platforms.

This churn seems to be driven by LSRA visiting blocks in a different order. Would it make sense to instead visit blocks in RPO to make it layout-agnostic? cc @kunalspathak

[amanasifkhalid]

Now that LSRA is using its own block order, the diffs seem to be contained to methods with switch statements. I see two broad sources of diffs:

• Switch lowering has some pretty gnarly lexical dependencies. JIT: Remove fallthrough checks in Compiler::TryLowerSwitchToBitTest #108106 removes a trivial one, though removing the fallthrough checks in Lowering::LowerSwitch doesn't seem simple.
• Since switch lowering can change flow by converting switch blocks to conditional branches, we're also churning layout (though this churn in particular is probably an improvement, since we can now optimize these changes in flow).

I think modernizing switch lowering is a worthy goal, though since we already don't place as much emphasis on layout quality for switch blocks, I think we should take this churn for now to expedite enabling 3-opt. @AndyAyersMS are you ok with this?

[AndyAyersMS]

I think modernizing switch lowering is a worthy goal, though since we already don't place as much emphasis on layout quality for switch blocks, I think we should take this churn for now to expedite enabling 3-opt. @AndyAyersMS are you ok with this?

Yes

[amanasifkhalid]

SPMI failure is a timeout, and libraries test failure is known.