Fix win-arm64 native varargs ABI

[BruceForstall]

SIMD vector types should be passed in integer registers for win-arm64 native varargs. This might require splitting a Vector128 between x7 and stack.

See https://learn.microsoft.com/en-us/cpp/build/arm64-windows-abi-conventions?view=msvc-170#addendum-variadic-functions

Fixes #89595

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

SIMD vector types should be passed in integer registers. This might require splitting a Vector128 between x7 and stack.

Author:	BruceForstall
Assignees:	BruceForstall
Labels:	area-CodeGen-coreclr
Milestone:	-

[BruceForstall]

Only failure is known failure #90593

[BruceForstall]

No spmi asm diffs -- negligible TP change on arm64.

[BruceForstall]

@jakobbotsch PTAL
cc @dotnet/jit-contrib

[jkotas]

Does VM-side implementation of calling conventions (https://github.com/dotnet/runtime/blob/main/src/coreclr/vm/callingconvention.h) work as expected for this case?

[jakobbotsch]

Is the size mismatch here fine?

[BruceForstall]

It turns out the size doesn't matter; the only thing that matters is INT (use general-purpose registers) or non-INT (use floaing-point/SIMD registers).

[BruceForstall]

Does VM-side implementation of calling conventions (https://github.com/dotnet/runtime/blob/main/src/coreclr/vm/callingconvention.h) work as expected for this case?

It looks like it might already be covered:

runtime/src/coreclr/vm/callingconvention.h

Lines 1618 to 1636 in 64243bb

	#ifdef _WIN32
	if (this->IsVarArg() && m_idxGenReg < 8)
	{
	// Address the Windows ARM64 varargs case where an arg is split between regs and stack.
	// This can happen in the varargs case because the first 64 bytes of the stack are loaded
	// into x0-x7, and any remaining stack arguments are placed normally.
	int argOfs = TransitionBlock::GetOffsetOfArgumentRegisters() + m_idxGenReg * 8;
	// Increase m_ofsStack to account for the space used for the remainder of the arg after
	// registers are filled.
	m_ofsStack += cbArg + (m_idxGenReg - 8) * TARGET_POINTER_SIZE;
	// We used up the remaining reg slots.
	m_idxGenReg = 8;
	return argOfs;
	}
	else
	#endif

One thing that wasn't handled on the JIT is passing/receiving any Vector to/in win-arm64 varargs functions. How could I test this case through the callingconvention.h code?

[jkotas]

callingconvention.h is only used for GC stackroot reporting.

It may be useful to add a variant of the test with interleaving scalars and object refences in the vararg signature, so that we would catch the situation where the GC stackroot reporting does not report the varargs arguments correctly.

[BruceForstall]

callingconvention.h is only used for GC stackroot reporting.

It may be useful to add a variant of the test with interleaving scalars and object refences in the vararg signature, so that we would catch the situation where the GC stackroot reporting does not report the varargs arguments correctly.

I stepped through callingconvention.h and saw the correct behavior occuring with the current code.

I added a few addtional test cases to the test.