[MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 1, ArraySubType = UnmanagedType.LPUTF8Str)] results in "Cannot marshal 'parameter #3': Unknown error."

[dlech]

I am trying out the UTF8 pinvoke marshalling. I have the following that works as expected:

    [DllImport ("gio-2.0", CallingConvention = CallingConvention.Cdecl)]
    static extern bool g_application_id_is_valid([MarshalAs(UnmanagedType.LPUTF8Str)] string applicationId);

However, this one does not...

    [DllImport ("gio-2.0", CallingConvention = CallingConvention.Cdecl)]
    static extern void g_application_run (SafeApplicationHandle application,
        int argc, [MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 1, ArraySubType = UnmanagedType.LPUTF8Str)] string[] argv);

It throws a MarshalDirectiveException with the message "Cannot marshal 'parameter dotnet/coreclr#3': Unknown error."

I see that in tests/src/Interop/StringMarshalling/UTF8/UTF8Test.cs, there is not a test for such a case, so I am guessing that perhaps it was overlooked and has not been implemented yet. (Or, if there is a mistake on my part, the error message could be improved.)

[yizhang82]

@dlech Thanks for reporting the issue. This is indeed a missing feature. /cc @tijoytom

[ChrisBirkett]

I observe the same MarshalDirectiveException (message Cannot marshal 'parameter dotnet/coreclr#1': Unknown error.) when I attempt to marshal parameters as UnmanagedType.AnsiBStr:

[DllImport(@"_Libs\vcmowr64.dll")]
public static extern Int32 wcmo_init([MarshalAs(UnmanagedType.AnsiBStr), In] ref string WCMOarg_Handle,
    [MarshalAs(UnmanagedType.AnsiBStr), In] ref string WCMOarg_User,
    [MarshalAs(UnmanagedType.AnsiBStr), Out] out string WCMOarg_DataOut,
    [MarshalAs(UnmanagedType.AnsiBStr), Out] out string WCMOarg_ErrOut);

The marshalling and call above work on Windows (1.0.3 and 1.1.0 tested):
[DllImport(@"_Libs\vcmowr64.dll")]
But produce the error when running in a Linux docker container (base image microsoft/dotnet running 1.0.4)
[DllImport(@"_Libs/libvcmowrap.so")]

I also looked at the tests/src/Interop/StringMarshalling and do not see any ANSI tests, so I am wondering if this is a missing feature as well, and curious to know why it works on Windows but not Linux?

[yizhang82]

@ChrisBirkett This is a different issue. We support "ANSI" marshaling in linux - in that context it simply means UTF-8 marshaling ("ANSI" is such an overloaded term...). We don't support BSTR/AnsiBSTR in linux, though. They are very Windows/COM specific (especially AnsiBSTR which is a VB6 thing). Would it make sense for you to simply switch to UnmanagedType.LPStr?

[ro-jo]

@yizhang82 So hopefully UnmanagedType.LPStr is UTF-8 on both Linux and Windows, can you clarify please?
I am writing common native hosting code for both Windows and Linux, the same code compiles on both - it won't be possible to use multilingual char arrays otherwise (with automatic marshalling).

[yizhang82]

@ro-jo LPStr is UTF-8 in linux but in windows it still needs to be "ANSI" (a misnomer). Changing it to UTF-8 in Windows is a breaking change for everybody else. You probably should use LPUTF8Str.
/cc @luqunl @jeffschwMSFT

[AaronRobinsonMSFT]

@GrabYourPitchforks How passionate are you about this support? I am just trying to understand if this is something we will ever update the built-in system to support or if we should defer to whatever happens with source generated P/Invokes.

[AaronRobinsonMSFT]

Details on what needs to happen in order to support this. The crux of the issue is we rely on OLE VARENUM to indicate the element type of arrays containing strings. There a several layers of indirection but that is how we are conveying the information through at present.

The starting point for this logic would be below. Notice the switch statement doesn't handle NATIVE_TYPE_LPUTF8STR.

runtime/src/coreclr/src/vm/mlinfo.cpp

Lines 5147 to 5189 in 103fa2c

	switch (ntElement)
	{
	case NATIVE_TYPE_DEFAULT:
	#ifdef FEATURE_COMINTEROP
	if (arrayNativeType == NATIVE_TYPE_SAFEARRAY || ms == MarshalInfo::MARSHAL_SCENARIO_COMINTEROP)
	{
	m_vtElement = VT_BSTR;
	}
	else
	#endif // FEATURE_COMINTEROP
	{
	m_vtElement = static_cast<VARTYPE>(isAnsi ? VT_LPSTR : VT_LPWSTR);
	}
	break;
	case NATIVE_TYPE_BSTR:
	m_vtElement = VT_BSTR;
	break;
	case NATIVE_TYPE_LPSTR:
	m_vtElement = VT_LPSTR;
	break;
	case NATIVE_TYPE_LPWSTR:
	m_vtElement = VT_LPWSTR;
	break;
	case NATIVE_TYPE_LPTSTR:
	{
	#ifdef FEATURE_COMINTEROP
	if (ms == MarshalInfo::MARSHAL_SCENARIO_COMINTEROP)
	{
	// We disallow NATIVE_TYPE_LPTSTR for COM or if we are exporting.
	ReportInvalidArrayMarshalInfo(IDS_EE_BADMARSHALPARAM_NO_LPTSTR);
	}
	else
	#endif // FEATURE_COMINTEROP
	{
	// We no longer support Win9x so LPTSTR always maps to a Unicode string.
	m_vtElement = VT_LPWSTR;
	}
	break;
	}
	default:
	ReportInvalidArrayMarshalInfo(IDS_EE_BADMARSHAL_STRINGARRAY);
	}

[raffaeler]

@AaronRobinsonMSFT

I am just trying to understand if this is something we will ever update the built-in system to support or if we should defer to whatever happens with source generated P/Invokes.

Since the code-generators do not currently work on custom native code, I do expect a fix on this.

• consider that the C Abi is something used for many languages, so I don't expect you will support parsing all of the programming languages
• the win32metadata repo does not even allow (afaik, please tell me if I am wrong) to decorate custom native components in order to automatically create pinvokes
• when talking of interop, reverse pinvoke is also extremely precious and I expect to be able to use it as well.

Therefore I feel scared when @yizhang82 says:

LPStr is UTF-8 in linux but in windows it still needs to be "ANSI" (a misnomer)

Since .NET is now a first-class x-platform thing, I do expect the same behavior on every platform.
The simplest solution is probably to support LPUTF8Str which is why I hit this issue.

[AaronRobinsonMSFT]

The simplest solution is probably to support LPUTF8Str which is why I hit this issue.

@raffaeler I agree it is the "simplest" but that is relative. Plumbing this support through the various layers with confidence of not impacting any other behavior is non-trivial and a serious concern for us. Examples of where we've tried to enable new functionality or enrich code is #48193 and the sequence #50137, #50655, and #50735. This isn't to say we won't consider this but our strong desire is to focus on source generators which provide better performance and stability overall. Given the relative low interest on this issue since it was opened in 2017 it is unlikely it will be addressed in the built-in system.

Manually marshalling arrays is always an option albeit less than ideal for most people. If this becomes an acute problem for the community it is absolutely something we will consider though. However, even if it were fixed it is unlikely we would backport this to previous runtimes so at this point focusing our effort on source generators seems to be the path forward here.

[raffaeler]

@AaronRobinsonMSFT I understand the difficulties but the source generators solve just one of the many use-cases.
More we move towards multiple OSes, greater is the need to interop among different languages.
Please don't reduce the need of interoperability to just accessing Win32 because it is just one very specific use case.

I currently had to start a project interoperating with Rust, and I am studying the language from scratch. There are tons of python libraries and mountains of bad interop code that crashes. And I can continue a long list of cases where there is no Win32 or WinRT as possible solutions.

For sure, if you asked me the same question 15 years ago, I would have answered that it could be sufficient for the eternity, but the reality (and companies) hugely changed :)

This is the reason why I asked you (maybe you remember) a cross-platform ABI. WinRT would be perfect, but it has been restricted to Windows, and doesn't fit the current needs for many projects.

[AaronRobinsonMSFT]

Please don't reduce the need of interoperability to just accessing Win32 because it is just one very specific use case.

@raffaeler That was not stated nor was it intended to be implied in my statement above. Source generators are a platform agnostic Roslyn extension that can be used on all platforms. The Interop team is prototyping such a generator in runtimelab. The prototype works on all .NET Core platforms (i.e., Windows, macOS, Linux) which can be observed from the CI definitions in the repo.

This is the reason why I asked you (maybe you remember) a cross-platform ABI. WinRT would be perfect, but it has been restricted to Windows, and doesn't fit the current needs for many projects.

Yes, I do recall talking together at the MVP summit. At present, the WinRT ABI is still relegated to Windows and supported in .NET 5+ through the C#/WinRT tooling. The C#/WinRT tool is built upon the .NET 5 ComWrappers API, which was designed to be supported in a cross-platform manner - see #36582 for the request.

This means that the IUnknown ABI is closer than one may think but not using the built-in system. The .NET interop team's focus going forward is to support and maintain the built-in system for high performance scenarios (e.g., all blittable arguments) and enable non-blittable scenarios through Roslyn source generation on all platforms in a stable and version resilient way. It should be noted that teams that extend/enable .NET interop scenarios through source generation may not have such a focus on cross-platform scenarios. However, all Interop tooling produced under the dotnet organization on GitHub will continue to prioritize cross-platform scenarios.

To further reinforce this statement see the following issues that should demonstrate our intense focus on the cross-platform scenario support:

• C long support - Expose CLong, CULong, and NFloat interchange types #13788
• Member function calling convention - Introduce CallConvMemberFunction to represent the member-function variants of Cdecl, Stdcall, etc. on Windows #46775
• Objective-C interop - Low level API support for Objective-C scenarios #44659

[raffaeler]

@AaronRobinsonMSFT

In the Interop team is prototyping one runtimelab.

Great. I will take a look to the lab. I hope there is some language-neutral way to instruct the code generator, so that I can create the interop layer to any language just "suggesting" the call with a high-level metadata.

The C#/WinRT tool is built upon the .NET 5 ComWrappers API, which was designed to be supported in a cross-platform manner - see #36582 for the request.
[...]

I remember you telling me about the ComWrappers. I stay tuned and hope to see it in action soon.
The reason why I still hoped about WinRT is because the WinUI team told several times there is nothing preventing (apart from the effort) porting their api x-plat. I see a big value in that, but I understand the COM abi is indeed needed by many "players".
As I wrote a C# debugger for .NET Core x-plat, I had to implement the COM ABI by myself on Linux for a customer. I wrote it by generating code from the IDL with many difficulties. I would love to replace that piece with something more mainstream.

Thanks for the other links, I am going to read them :)

[AaronRobinsonMSFT]

Now that the LibraryImport generator (i.e., source generated DllImports) has been exposed as a publicly consumable tool, this issue should be explored relative to that effort. This specific issue is being close due to lower priority relative to other interop needs and should be solved using the source generator.