Cross-Platform COM Interop

[hypersw]

Goal: enable interop with native libraries with COM ABI and a reasonable subset of COM features on all supported platforms.

Related issue with generic discussion: https://github.com/dotnet/coreclr/issues/11279
This issue is to list specific work areas required to achieve the functionality:

• RCW (Runtime Callable Wrappers).

  • Managed proxy for a native VTable.
  • AddRef on native IUnknown until managed proxy is finalized (or Marshal Release/FinalRelease).
  • Turn managed casts into QI calls to native IUnknown.
  • Turn reflection/dynamic into IDispatch(Ex) calls to native object.

• CCW (COM Callable Wrappers).

  • Generate VTable, expose native pointer to it.
  • Keep GC root until final release from native side.
  • Turn IDispatch calls into reflection/dynamic calls on the managed object.

• Marshalling.

  • Mostly the same as in PInvoke.
  • Non-preserve-sig handling:
    • HRESULT into Exception for RCW; Exception into HRESULT for CCW.
    • Out parameter vs. return value transform.
    • Pointer<->RCW and object<->CCW transform.

Supposedly, most of the implementation should already be available from the .NET Framework code. WinAPI implementations should be substituted with crude fallbacks for trivial cases (no activation scenarios, no cross-apartment call marshalling, no proxy/stubs).

Activation: manual from a DLL file, or possibly side-by-side scenarios. At first, activation is out of scope, suppose we create the first COM object with external means (e.g. Pinvoke + GetObjectForIUnknown).

Here are example scenarios for possible application of this functionality.

• CLR Debugger interfaces (ICorDebug et al). We have a success story of running the cross-platform implementation of debugger with native COM interop from C#, with Mono on Linux and Mac. Mono has huge problems with correctness of the implementation, yet if you're using only core features and trivial marshalling it's working enough to have a functioning debugger. The problems do not look fundamental, just of the quality sort. Obviously, we would now like to run the same code on CoreCLR.
• Visual Studio COM interfaces have proven a working example of a large-scale native-managed interop with native and managed parts mixed together with no clear separation. That's not exactly about porting, but that's a good example of a multi-technology app with native/managed interop API.
• A few of our own helper libraries on Windows are exposing COM interfaces for object-oriented interaction with the app. This experience is nice to have on any platform.

[jkotas]

Have you looked at https://github.com/SharpGenTools/SharpGenTools ? We think independent tools like this is much better way to solve this. It has better performance and it is independent from the runtime so it is much easier to customize to your unique needs.

[jkotas]

cc @jkoritzinsky

[hypersw]

@jkotas: Yes we looked at various tools, during the initial assessment when considering a Mono-based COM interop implementation, and later on when checking the NetCore migration scenarios. Though it looks much more promising to have these objects working first-class in the runtime, especially as the runtime has all of the required features on board but #ifdef'ed out for all the non-Windows platforms. It was astonishing to see COM interop running successfully on Mono (!) on Linux (!). After seeing that, running custom adapter generation tools feels like a major drawback. Especially on the runtime which has derived the original support for the feature in terms of internal structures and JIT features.

[hypersw]

Another point is that rather than customizing to our unique needs (which with open-source software often means fixing and patching) we'd rather have something which runs against the clear and well-defined COM interop rules, common for all. If the effort is roughly the same, we'd rather invest in making the standard interop work (for us and for everyone).

It's hard to estimate how deep the rabbit hole is, actually. What pieces are missing on Linux? I wrote this issue to learn this in the first place. From reading the Mono code it seems that the big deal is to hack the type system and JIT and wherever you have to plug in CCW/RCW correctly, all of these mostly cross-platform — while the WinAPI-specific calls are not so mixed up with the logic. How true is this?

[AaronRobinsonMSFT]

It's hard to estimate how deep the rabbit hole is, actually. What pieces are missing on Linux? I wrote this issue to learn this in the first place. From reading the Mono code it seems that the big deal is to hack the type system and JIT and wherever you have to plug in CCW/RCW correctly, all of these mostly cross-platform — while the WinAPI-specific calls are not so mixed up with the logic. How true is this?

@hypersw I think you are basically right in the sense that WinAPI use is orthogonal to CCW/RCW which is really nice from "Should be easy to port to non-Windows" perspective. What isn't obvious in the approach is the cost of maintaining this logic in the runtime. Your general suggestion of bringing out CCW/RCW support is and has been debated before. In the near future I am going to be starting that conversation in the open so more people can contribute. There is a strong opinion to have a pay-for-play mentality with IUnknown style interop and marshalling in general on non-Windows platforms. Again the debate is ongoing, not even close to done, and we want to hear from people about their needs. Look for something in the next few months regarding this work.

cc: @jeffschwMSFT @luqunl

[jkotas]

running custom adapter generation tools feels like a major drawback

We are interested in having first class support for running custom adapter generation tools in the build tooling so that it does not feel like a major drawback. The custom adapter generation tools were always part of the COM interop story actually: tlbexp, tlbimp, etc. Unfortunately, they were not designed to generate the actual code and there was still a lot of complex logic required in the runtime.

runtime which has derived the original support for the feature in terms of internal structures

This architecture choice makes the COM and WinRT interop (the two are not cleanly segregated) very expensive to maintain. It is a lot of complex low-level code, our legacy baggage on Windows. We would like to avoid maintaining it on multiple platforms.

the big deal is to hack the type system and JIT and wherever you have to plug in CCW/RCW correctly

It is not required to hack the type system and JIT at all to plug in CCW/RCW correctly. The basic COM interop can be done just fine as SharpGen demonstrated.
Full feature parity requires ICastable hook. We would like to make it officially documented and supported. If there are other similar low-level hooks required to enable great interop with other language environments, we will be happy to look at adding those too.

FWIW, Visual Studio parts that run on Unix have a pretty complete COM interop build on top of stock CoreCLR using the ICastable hook. The interop code is generated using MCG tool that is closed source equivalent of SharpGen - if we were doing it today, we would just use SharpGen instead.

[jkoritzinsky]

Hey! I'm the SharpGenTools maintainer. Just wanted to pop in to the conversation since I was tagged. As @jkotas mentioned, SharpGenTools enables COM interop relatively seamlessly. The 1.1 release (coming out after I get back from vacation) will make it even easier by adding support for auto-generating the C# stubs that enable passing .NET objects to native code.

@hypersw: re customization: SharpGenTools supports an insane amount of customization. When SharpGenTools were just an internal tool for SharpDX, tons of customization features were added for obscure features or deficiencies in the mapping software at the time. Some examples are multiple methods of custom marshalling as well as runtime customizable vtable indices when needed.

Full feature parity requires ICastable hook. We would like to make it officially documented and supported. If there are other similar low-level hooks required to enable great interop with other language environments, we will be happy to look at adding those too.

I am very much looking forward to seeing low level runtime hooks like these exposed for interop libraries like SharpGenTools! I never realized there were hooks like ICastable in the runtime.

[migueldeicaza]

(Addendum: Mono's COM support was never productized, it got a few contributions from the community, but the core team was never involved with it. I am sure that there are many things missing there. You really will be better off using something like SharpGenTools, even in Mono)

[Const-me]

I’ve just implemented something very similar: https://github.com/Const-me/ComLightInterop

[AaronRobinsonMSFT]

Now that the ComWrappers API has been enabled on all platforms where CoreCLR runs, I am considering this issue closed as it is now possible. What is still missing is a source generation tool similar to C#/WinRT but for COM.