Add hot/cold splitting test job to jit-runtime-experimental

[amanasifkhalid]

Adding a new rolling test job to JIT-runtime-experimental can help ensure the new fake- and stress-splitting modes in the JIT do not regress. The proposed test, titled jitosr_stress_splitting, runs a Checked build of the JIT on OS- and platform-specific test suites with the following environment:

• COMPlus_GCgen0size=1000000
• COMPlus_JitFakeProcedureSplitting=1
• COMPlus_JitStressprocedureSplitting=1

This environment forces the JIT to fake-split every method after its first basic block, assuming the method consists of more than one block. COMPlus_GCgen0size=1000000 sets a large memory allocation threshold before the GC runs; recall that COMPlus_JitFakeProcedureSplitting currently doesn't generate unwind information for cold code, thus breaking the GC's stack walks. COMPlus_GCgen0size provides a patchwork fix for this limitation in cases where memory usage is not excessive (i.e. >= 0x1000000 B).

In my local testing, all of the x64 tests in src/tests/ pass with this environment.

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Adding a new rolling test job to JIT-runtime-experimental can help ensure the new fake- and stress-splitting modes in the JIT do not regress. The proposed test, titled jitosr_stress_splitting, runs a Checked build of the JIT on OS- and platform-specific test suites with the following environment:

• COMPlus_GCgen0size=1000000
• COMPlus_JitFakeProcedureSplitting=1
• COMPlus_JitStressprocedureSplitting=1

This environment forces the JIT to fake-split every method after its first basic block, assuming the method consists of more than one block. COMPlus_GCgen0size=1000000 sets a large memory allocation threshold before the GC runs; recall that COMPlus_JitFakeProcedureSplitting currently doesn't generate unwind information for cold code, thus breaking the GC's stack walks. COMPlus_GCgen0size provides a patchwork fix for this limitation in cases where memory usage is not excessive (i.e. >= 0x1000000 B).

In my local testing, all of the x64 tests in src/tests/ pass with this environment.

Author:	amanasifkhalid
Assignees:	amanasifkhalid
Labels:	area-CodeGen-coreclr
Milestone:	-

[BruceForstall]

Looks good. However, this isn't related to OSR, so remove "osr" from the name. Use jit_stress_splitting or jit_stress_procedure_splitting.

[BruceForstall]

LGTM.

You should trigger jit-runtime-experimental on this PR to verify.

[amanasifkhalid]

/azp run runtime-jit-experimental

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[amanasifkhalid]

I ran and re-ran runtime-jit-experimental and, like all the other failing checks, it fails to send tests to Helix due to a bad System.AccessToken value. Is the fact that runtime-jit-experimental makes it all the way to the "Send to Helix" step sufficient verification to merge the PR?

[BruceForstall]

I ran and re-ran runtime-jit-experimental and, like all the other failing checks, it fails to send tests to Helix due to a bad System.AccessToken value.

You hit #69854, which I believe is now fixed. Try triggering the job again.

Is the fact that runtime-jit-experimental makes it all the way to the "Send to Helix" step sufficient verification to merge the PR?

No -- It's the "Send to Helix" step that actually runs tests, and since it failed, no tests were run.

[BruceForstall]

You also hit the "remainingSize == 4" assert which was fixed with #69905

Also, some wasm failures which I don't see issues for, but which are obviously unrelated. You can always try "Re-run" on those.

[amanasifkhalid]

/azp run runtime-jit-experimental

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[amanasifkhalid]

I pulled from main and re-ran runtime-jit-experimental, but it's still failing with 401 responses from Azure DevOps, per #69854 .

[BruceForstall]

I pulled from main and re-ran runtime-jit-experimental, but it's still failing with 401 responses from Azure DevOps, per #69854 .

The scheduled run on Sunday did not have those issues (it had other issues, but not those: https://dev.azure.com/dnceng/public/_build/results?buildId=1795838&view=ms.vss-test-web.build-test-results-tab).

My suggestion:

1. rebase on top of main (don't merge main)
2. push the updated PR

It shouldn't be necessary, but if that doesn't work, try closing and re-opening the PR, or create an entirely new PR.

[amanasifkhalid]

/azp run runtime-jit-experimental

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[BruceForstall]

/azp run runtime-jit-experimental

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[BruceForstall]

/azp run runtime-jit-experimental

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[amanasifkhalid]

/azp run runtime-jit-experimental

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[amanasifkhalid]

I have pushed a commit with a fix for unwind info generation when fake-splitting: When calling eeAllocUnwindInfo, we manually set the start offset to point at the beginning of the hot section, and the end offset to point at the end of the cold section, effectively treating them as a single "hot" section. However, for reasons under investigation, this implementation breaks the GC when there is a buffer between the sections; thus, I've adjusted the fake-splitting implementation to leave the hot/cold sections adjacent in memory, while still behaving as if they're arbitrarily far apart (for example, the JIT still generates long branches between sections).

I've tested the x64 implementation locally, and will adjust x86 implementation/add ARM implementation as needed.

[BruceForstall]

LGTM. If the tests pass, you're good to merge.

[amanasifkhalid]

Other than jit_osr_stress_random (see issue), runtime-jit-experimental is passing. Build windows arm64 Release NativeAOT is failing due to a Helix job timing out, but the build otherwise succeeds.