Low level API support for RCW and CCW management

[AaronRobinsonMSFT]

Work has begun to provide support for WinUI 3.0. This support is expected to manifest in a way similar to the CppWinRT tool by way of a new source generation tool (e.g. CsWinRT). In order to support this new tool, APIs for integrating and coordinating with the runtime object lifetime are necessary.

Rationale and Usage

The below API surface provides a way for a third party tool to generate what are colloquially known as Runtime Callable Wrappers (RCW) and COM Callable Wrappers (CCW) in a way that allows safe interaction with managed object lifetime and identity.

A specific example of the need for lifetime coordination is in WinRT scenarios involving UI (e.g. WinUI 3.0) via the IReferenceTrackerManager interface.

Goals:

• Enable source generation of interop code in WinRT/WinUI scenarios.
• An API that generally aligns with how existing 3rd party source generators work (e.g. SharpGenTools).
• Limit exposing APIs that manage lifetime in a micro way (e.g. avoid GC hooks at dangerous times).
• Semantics in .NET Framework/.NET Core for WinRT scenarios should be able to be replicated.
• Provide a mechanism for 3rd parties to be able to support Reference Tracker scenarios.
• Ensure AOT scenarios are considered.

Non-Goals:

• Replace the existing built-in RCW/CCW infrastructure.
• Change anything related to P/Invoke Interop.
• Hide WinRT and/or COM concepts.

Outstanding questions:

• Implement new COM interop API for RCW/CCW creation/management #32091 (comment)
• Implement new COM interop API for RCW/CCW creation/management #32091 (comment)

Proposed API

namespace System.Runtime
{
    public static partial class RuntimeHelpers
    {
        /// <summary>
        /// Allocate memory that is associated with the <paramref name="type"/> and
        /// will be freed if and when the <see cref="System.Type"/> is unloaded.
        /// </summary>
        /// <param name="type">Type associated with the allocated memory.</param>
        /// <param name="size">Amount of memory in bytes to allocate.</param>
        /// <returns>The allocated memory</returns>
        public static IntPtr AllocateTypeAssociatedMemory(Type type, int size);
    }
}

namespace System.Runtime.InteropServices
{
    /// <summary>
    /// Enumeration of flags for <see cref="ComWrappers.GetOrCreateComInterfaceForObject(object, CreateComInterfaceFlags)"/>.
    /// </summary>
    [Flags]
    public enum CreateComInterfaceFlags
    {
        None = 0,

        /// <summary>
        /// The caller will provide an IUnknown Vtable.
        /// </summary>
        /// <remarks>
        /// This is useful in scenarios when the caller has no need to rely on an IUnknown instance
        /// that is used when running managed code is not possible (i.e. during a GC). In traditional
        /// COM scenarios this is common, but scenarios involving <see href="https://docs.microsoft.com/windows/win32/api/windows.ui.xaml.hosting.referencetracker/nn-windows-ui-xaml-hosting-referencetracker-ireferencetrackertarget">Reference Tracker hosting</see>
        /// calling of the IUnknown API during a GC is possible.
        /// </remarks>
        CallerDefinedIUnknown = 1,

        /// <summary>
        /// Flag used to indicate the COM interface should implement <see href="https://docs.microsoft.com/windows/win32/api/windows.ui.xaml.hosting.referencetracker/nn-windows-ui-xaml-hosting-referencetracker-ireferencetrackertarget">IReferenceTrackerTarget</see>.
        /// When this flag is passed, the resulting COM interface will have an internal implementation of IUnknown
        /// and as such none should be supplied by the caller.
        /// </summary>
        TrackerSupport = 2,
    }

    /// <summary>
    /// Enumeration of flags for <see cref="ComWrappers.GetOrCreateObjectForComInstance(IntPtr, CreateObjectFlags, object?)"/>.
    /// </summary>
    [Flags]
    public enum CreateObjectFlags
    {
        None = 0,

        /// <summary>
        /// Indicate if the supplied external COM object implements the <see href="https://docs.microsoft.com/windows/win32/api/windows.ui.xaml.hosting.referencetracker/nn-windows-ui-xaml-hosting-referencetracker-ireferencetracker">IReferenceTracker</see>.
        /// </summary>
        TrackerObject = 1,

        /// <summary>
        /// Ignore any internal caching and always create a unique instance.
        /// </summary>
        UniqueInstance = 2,
    }

    /// <summary>
    /// Class for managing wrappers of COM IUnknown types.
    /// </summary>
    [CLSCompliant(false)]
    public abstract partial class ComWrappers
    {
        /// <summary>
        /// Interface type and pointer to targeted VTable.
        /// </summary>
        public struct ComInterfaceEntry
        {
            /// <summary>
            /// Interface IID.
            /// </summary>
            public Guid IID;

            /// <summary>
            /// Memory must have the same lifetime as the memory returned from the call to <see cref="ComputeVtables(object, CreateComInterfaceFlags, out int)"/>.
            /// </summary>
            public IntPtr Vtable;
        }

        /// <summary>
        /// ABI for function dispatch of a COM interface.
        /// </summary>
        public struct ComInterfaceDispatch
        {
            public IntPtr vftbl;

            /// <summary>
            /// Given a <see cref="System.IntPtr"/> from a generated VTable, convert to the target type.
            /// </summary>
            /// <typeparam name="T">Desired type.</typeparam>
            /// <param name="dispatchPtr">Pointer supplied to VTable function entry.</param>
            /// <returns>Instance of type associated with dispatched function call.</returns>
            public static unsafe T GetInstance<T>(ComInterfaceDispatch* dispatchPtr) where T : class;
        }

        /// <summary>
        /// Create an COM representation of the supplied object that can be passed to an non-managed environment.
        /// </summary>
        /// <param name="instance">A GC Handle to the managed object to expose outside the .NET runtime.</param>
        /// <param name="flags">Flags used to configure the generated interface.</param>
        /// <returns>The generated COM interface that can be passed outside the .NET runtime.</returns>
        public IntPtr GetOrCreateComInterfaceForObject(object instance, CreateComInterfaceFlags flags);

        /// <summary>
        /// Compute the desired VTables for <paramref name="obj"/> respecting the values of <paramref name="flags"/>.
        /// </summary>
        /// <param name="obj">Target of the returned VTables.</param>
        /// <param name="flags">Flags used to compute VTables.</param>
        /// <param name="count">The number of elements contained in the returned memory.</param>
        /// <returns><see cref="ComInterfaceEntry" /> pointer containing memory for all COM interface entries.</returns>
        /// <remarks>
        /// All memory returned from this function must either be unmanaged memory, pinned managed memory, or have been
        /// allocated with the <see cref="System.Runtime.CompilerServices.RuntimeHelpers.AllocateTypeAssociatedMemory(Type, int)"/> API.
        ///
        /// If the interface entries cannot be created and <code>null</code> is returned, the call to <see cref="ComWrappers.GetOrCreateComInterfaceForObject(object, CreateComInterfaceFlags)"/> will throw a <see cref="System.ArgumentNullException"/>.
        /// </remarks>
        protected unsafe abstract ComInterfaceEntry* ComputeVtables(object obj, CreateComInterfaceFlags flags, out int count);

        /// <summary>
        /// Get the currently registered managed object or creates a new managed object and registers it.
        /// </summary>
        /// <param name="externalComObject">Object to import for usage into the .NET runtime.</param>
        /// <param name="flags">Flags used to describe the external object.</param>
        /// <param name="wrapper">An optional <see cref="object"/> to be used as the wrapper for the external object</param>
        /// <returns>Returns a managed object associated with the supplied external COM object.</returns>
        /// <remarks>
        /// Providing a <paramref name="wrapper"/> instance means <see cref="ComWrappers.GetOrCreateObjectForComInstance(IntPtr, CreateObjectFlags, object?)"/>
        /// will not be called.
        ///
        /// If the <paramref name="wrapper"/> instance already has an associated external object a <see cref="System.NotSupportedException"/> will be thrown.
        /// </remarks>
        public object GetOrCreateObjectForComInstance(IntPtr externalComObject, CreateObjectFlags flags, object? wrapper = null);

        /// <summary>
        /// Create a managed object for the object pointed at by <paramref name="externalComObject"/> respecting the values of <paramref name="flags"/>.
        /// </summary>
        /// <param name="externalComObject">Object to import for usage into the .NET runtime.</param>
        /// <param name="flags">Flags used to describe the external object.</param>
        /// <returns>Returns a managed object associated with the supplied external COM object.</returns>
        /// <remarks>
        /// If the object cannot be created and <code>null</code> is returned, the call to <see cref="ComWrappers.GetOrCreateObjectForComInstance(IntPtr, CreateObjectFlags, object?)"/> will throw a <see cref="System.ArgumentNullException"/>.
        /// </remarks>
        protected abstract object? CreateObject(IntPtr externalComObject, CreateObjectFlags flags);

        /// <summary>
        /// Called when a request is made for a collection of objects to be released.
        /// </summary>
        /// <param name="objects">Collection of objects to release.</param>
        /// <remarks>
        /// The default implementation of this function throws <see cref="System.NotImplementedException"/>.
        /// </remarks>
        protected virtual void ReleaseObjects(IEnumerable objects);

        /// <summary>
        /// Register this class's implementation to be used as the single global instance.
        /// </summary>
        /// <remarks>
        /// This function can only be called a single time. Subsequent calls to this function will result
        /// in a <see cref="System.InvalidOperationException"/> being thrown.
        ///
        /// Scenarios where the global instance may be used are:
        ///  * Object tracking via the <see cref="CreateComInterfaceFlags.TrackerSupport" /> and <see cref="CreateObjectFlags.TrackerObject" /> flags.
        ///  * Usage of COM related Marshal APIs.
        /// </remarks>
        public void RegisterAsGlobalInstance();

        /// <summary>
        /// Get the runtime provided IUnknown implementation.
        /// </summary>
        /// <param name="fpQueryInterface">Function pointer to QueryInterface.</param>
        /// <param name="fpAddRef">Function pointer to AddRef.</param>
        /// <param name="fpRelease">Function pointer to Release.</param>
        protected static void GetIUnknownImpl(out IntPtr fpQueryInterface, out IntPtr fpAddRef, out IntPtr fpRelease);
    }
}

Example usage

The below example is merely for illustrative purposes. In a production ready consumption of the API many of the Marshal APIs would not be used and the VTable layouts should be done in a static manner for optimal efficiency.

[Guid("197BC142-7A71-4637-B504-894DE79C4A22")]
interface IPrint
{
    public void PrintInt(int i);
}

class Print : IPrint
{
    public void PrintInt(int i)
    {
        Console.WriteLine($"{nameof(IPrint.PrintInt)} - 0x{i:x}");
    }
}

struct IUnknownVftbl
{
    public IntPtr QueryInterface;
    public IntPtr AddRef;
    public IntPtr Release;
}

struct IPrintVftbl
{
    public IUnknownVftbl IUnknownImpl;
    public IntPtr PrintInt;

    public delegate int _PrintInt(IntPtr thisPtr, int i);
    public static _PrintInt pPrintInt = new _PrintInt(PrintIntInternal);

    public static int PrintIntInternal(IntPtr dispatchPtr, int i)
    {
        unsafe
        {
            try
            {
                ComWrappers.ComInterfaceDispatch.GetInstance<IPrint>((ComWrappers.ComInterfaceDispatch*)dispatchPtr).PrintInt(i);
            }
            catch (Exception e)
            {
            return e.HResult;
            }
        }

        return 0; // S_OK;
    }
}

struct VtblPtr
{
    public IntPtr Vtbl;
}

class IExternalObject
{
    private struct IExternalObjectVftbl
    {
        public IntPtr QueryInterface;
        public _AddRef AddRef;
        public _Release Release;
        public _AddObjectRef AddObjectRef;
        public _RemoveObjectRef DropObjectRef;
    }

    private delegate int _AddRef(IntPtr This);
    private delegate int _Release(IntPtr This);
    private delegate int _AddObjectRef(IntPtr This, IntPtr o);
    private delegate int _RemoveObjectRef(IntPtr This, IntPtr o);

    private readonly IntPtr instance;
    private readonly IExternalObjectVftbl vtable;

    public IExternalObject(IntPtr instance)
    {
        var inst = Marshal.PtrToStructure<VtblPtr>(instance);
        this.vtable = Marshal.PtrToStructure<IExternalObjectVftbl>(inst.Vtbl);
        this.instance = instance;
    }

    ~IExternalObject()
    {
        if (this.instance != IntPtr.Zero)
        {
            this.vtable.Release(this.instance);
        }
    }

    public void AddObjectRef(object inst)
    {
        ...
    }

    public void DropObjectRef(object inst)
    {
        ...
    }
}

class MyComWrappers : ComWrappers
{
    protected unsafe override ComInterfaceEntry* ComputeVtables(object obj, CreateComInterfaceFlags flags, out int count)
    {
        IntPtr fpQueryInteface = default;
        IntPtr fpAddRef = default;
        IntPtr fpRelease = default;
        ComWrappers.GetIUnknownImpl(out fpQueryInteface, out fpAddRef, out fpRelease);

        var tables = new List<ComInterfaceEntry>();

        var vtbl1 = new IPrintVftbl()
        {
            IUnknownImpl = new IUnknownVftbl()
            {
                QueryInterface = fpQueryInteface,
                AddRef = fpAddRef,
                Release = fpRelease
            },
            PrintInt = Marshal.GetFunctionPointerForDelegate(IPrintVftbl.pPrintInt)
        };
        var vtblRaw1 = RuntimeHelpers.AllocateTypeAssociatedMemory(typeof(IPrintVftbl), sizeof(IPrintVftbl));
        Marshal.StructureToPtr(vtbl1, vtblRaw1, false);
        tables.Add(new ComInterfaceEntry { IID = IID_IPrint, Vtable = vtblRaw1 });

        if (flags.HasFlag(CreateComInterfaceFlags.CallerDefinedIUnknown))
        {
            var vtbl2 = new IUnknownVftbl()
            {
                QueryInterface = fpQueryInteface,
                AddRef = fpAddRef,
                Release = fpRelease
            };
            var vtblRaw2 = RuntimeHelpers.AllocateTypeAssociatedMemory(typeof(IUnknownVftbl), sizeof(IUnknownVftbl));
            Marshal.StructureToPtr(vtbl2, vtblRaw2, false);
            tables.Add(new ComInterfaceEntry { IID = IID_IUnknown, Vtable = vtblRaw2 });
        }

        // Return pointer to memory containing ComInterfaceEntry collection
    }

    protected override object CreateObject(IntPtr externalComObject, CreateObjectFlags flags)
    {
        return new IExternalObject(externalComObject);
    }
}

/cc @jkotas @Scottj1s @dunhor @jkoritzinsky @davidwrighton @terrajobst @tannergooding @jeffschwMSFT

[jkoritzinsky]

Would it be possible to make the return value of ComputeVtables pinned by the caller instead of by the implementation? That would remove a "pit of failure" around using a managed array as storage for the vtables and using the implicit conversion to ReadOnlySpan<T>.

[jkotas]

There is no way to pin byrefs via GCHandles (it would be expensive for the GC to allow it). If you believe that ReadOnlySpan<T> in the signature is a pit of failure, it may be better to fix it by making ComputeVtables return unmanaged pointer + count explicitly.

[jkotas]

Marshal.AllocCoTaskMem

This assumes that the vtables won't be ever unloaded. I think it is ok for v1, but I wondering whether we need to do any tweaks in the design now to potentially allow unloading these in future.

[AaronRobinsonMSFT]

This assumes that the vtables won't be ever unloaded. I think it is ok for v1, but I wondering whether we need to do any tweaks in the design now to potentially allow unloading these in future.

That is just an example. I hope the preface for the Example section wasn't missed. That example is for conceptual understanding for the API review. It is not how I would expect source generators to consume the API.

but I wondering whether we need to do any tweaks in the design now to potentially allow unloading these in future.

I don't think so. The source generator here can drive that decision. Once all the CCWs the source generated are unused, that memory can be reclaimed. I would argue that is something the source generator code can track themselves.

[AaronRobinsonMSFT]

That would remove a "pit of failure" around using a managed array as storage for the vtables and using the implicit conversion to ReadOnlySpan.

Completely forgot about the implicit conversion. That seems bad actually. I think I am inclined to change that to one of the original designs and return a tuple with a pointer and count - similar to what @jkotas is suggesting above.

[jkotas]

Once all the CCWs the source generated are unused, that memory can be reclaimed. I would argue that is something the source generator code can track themselves.

What is the code that the source generator would generate to track this?

[AaronRobinsonMSFT]

What is the code that the source generator would generate to track this?

Boo. I just re-realized the whole point of this is for WinUI and in that scenario the caller isn't going to be implementing any part of IUnknown. I was originally going to say the CCW could wrap the Release() call and when that returns 0 for all CCWs with the associated memory it could be released. However, that won't work for cases when the source generator code can't provide any Release() implementation due to the aforementioned GC issues.

This is something we should put thought into now. I am loathe to request a callback for when an object is destroyed. Perhaps some kind of enumerating mechanism for the CCW caches? Open to other suggestions as well.

[AaronRobinsonMSFT]

Thinking more about this, I believe I got distracted from my example and @jkotas's very specific question.

@jkotas Users should be able to track this information by returning their own version of Release() that proxies to the default one supplied. Of course this will not work in WinUI scenarios. Perhaps that makes it worth the effort to try and think of a solution, but in general I hope not for the following reason.

I hope that no one will use Marshal.AllocCoTaskMem(). Instead I expect users would pin some static class memory and provide the address to the array in the callback. This would allow the memory to be coupled to the assembly loader which will live as long as the object stays around and the object itself will be extended to as long as the CCW.

This means that when the CCW is freed, the managed object can be collected, which means the assembly and all associated memory can be collected.

Is that sound or am I missing something?

[jkotas]

This would allow the memory to be coupled to the assembly loader

How does this coupling happen? Does this mean that all structures involved in this have to be allocated on GC heap?

Also, note that infinite pinning is not good for GC performance - it prevents GC from compacting the heap and getting rid of the fragmentation.

A different way to solve this would be to introduce method that allocates unmanaged piece of memory that has lifetime attached to lifetime of given Type. E.g.IntPtr AllocateMemoryAssociatedWithType(Type type, int size). Implementing this would be cheap - we do have a facility like this internally in the runtime (LoaderAllocator). All calls of AllocCoTaskMem in your example would be replaced with this method.

[AaronRobinsonMSFT]

How does this coupling happen?

Since the data would be the same for all instances of a type, I assume it would something along the lines of the definition below. Then the LongLivedData would be associated with the assembly that it lives in. The trick is ensuring the underlying data for Type1 never moves. My assumption here was pinned memory or some other mechanism I don't know about but was sure the runtime had could be used to efficiently make that memory position stable/static.

static class LongLivedData
{
    public static ComInterfaceEntry[] Type1= ...
}

If there isn't any such efficient mechanism then your allocator API seems reasonable.

[AaronRobinsonMSFT]

Chatting offline with various people the following updates will be made to the API to reflect the lifetime issues mentioned in #1845 (comment). /cc @jkotas

namespace System.Runtime
{
    public static partial class RuntimeHelpers
    {
        /// <summary>
        /// Allocate memory that is associated with the <paramref name="type"/> and
        /// will be freed if and when the <see cref="System.Type"/> is unloaded.
        /// </summary>
        /// <param name="type">Type associated with the allocated memory.</param>
        /// <param name="size">Amount of memory in bytes to allocate.</param>
        /// <returns>The allocated memory</returns>
        public static IntPtr AllocateTypeAssociatedMemory(Type type, int size);
    }
}

In order to mitigate issues mentioned in #1845 (comment), the the ComputeVtables() API will be updated as follows. /cc @jkoritzinsky

/// <summary>
/// Compute the desired VTables for <paramref name="obj"/> respecting the values of <paramref name="flags"/>.
/// </summary>
/// <param name="obj">Target of the returned VTables.</param>
/// <param name="flags">Flags used to compute VTables.</param>
/// <param name="count">The number of elements contained in the returned memory.</param>
/// <returns><see cref="ComInterfaceEntry*" /> containing memory for all COM interface entries.</returns>
/// <remarks>
/// All memory returned from this function must either be unmanaged memory, pinned managed memory, or have been
/// allocated with the <see cref="System.Runtime.RuntimeHelpers.AllocateTypeAssociatedMemory(Type, int)"/> API.
/// </remarks>
protected unsafe abstract ComInterfaceEntry* ComputeVtables(object obj, CreateComInterfaceFlags flags, out int count);

[mjsabby]

Will we support this xplat? That would be awesome.

[AaronRobinsonMSFT]

@mjsabby That is a good question. For right now the implementation is being written to target only Windows. However, the implementation has a single dependency on a Windows API and that is only because of COM apartments. On non-Windows platforms there isn't any concept of COM apartments so that could easily be severed with little fanfare.

My plan is to ensure the API exists on Windows for future WinUI/WinRT scenario tooling. If it proves to be valuable outside of that scenario it would be a small amount of work to enable this xplat if demand exists.

Edit: The COM Apartment API usage has been removed.

[mjsabby]

@AaronRobinsonMSFT Right, if we can cull that it would be a very useful addition to cross platform COM porting. I've seen many libraries benefit from the conventions of COM that don't need registry based activation or COM and would be a breeze to support with the right tooling support, and I say that even if the tooling continues to be windows-based.

Consider a +1 from me :)

[terrajobst]

Video

• Looks good as proposed
• We should align the naming Vtable vs. vftbl
• Some comments are out sync
• The design should be validated by people who are wrapping COM objects

API

namespace System.Runtime
{
    public static partial class RuntimeHelpers
    {
        /// <summary>
        /// Allocate memory that is associated with the <paramref name="type"/> and
        /// will be freed if and when the <see cref="System.Type"/> is unloaded.
        /// </summary>
        /// <param name="type">Type associated with the allocated memory.</param>
        /// <param name="size">Amount of memory in bytes to allocate.</param>
        /// <returns>The allocated memory</returns>
        public static IntPtr AllocateTypeAssociatedMemory(Type type, int size);
    }
}

namespace System.Runtime.InteropServices
{
    /// <summary>
    /// Enumeration of flags for <see cref="ComWrappers.GetOrCreateComInterfaceForObject(object, CreateComInterfaceFlags)"/>.
    /// </summary>
    [Flags]
    public enum CreateComInterfaceFlags
    {
        None = 0,

        /// <summary>
        /// The caller will provide an IUnknown Vtable.
        /// </summary>
        /// <remarks>
        /// This is useful in scenarios when the caller has no need to rely on an IUnknown instance
        /// that is used when running managed code is not possible (i.e. during a GC). In traditional
        /// COM scenarios this is common, but scenarios involving <see href="https://docs.microsoft.com/windows/win32/api/windows.ui.xaml.hosting.referencetracker/nn-windows-ui-xaml-hosting-referencetracker-ireferencetrackertarget">Reference Tracker hosting</see>
        /// calling of the IUnknown API during a GC is possible.
        /// </remarks>
        CallerDefinedIUnknown = 1,

        /// <summary>
        /// Flag used to indicate the COM interface should implement <see href="https://docs.microsoft.com/windows/win32/api/windows.ui.xaml.hosting.referencetracker/nn-windows-ui-xaml-hosting-referencetracker-ireferencetrackertarget">IReferenceTrackerTarget</see>.
        /// When this flag is passed, the resulting COM interface will have an internal implementation of IUnknown
        /// and as such none should be supplied by the caller.
        /// </summary>
        TrackerSupport = 2,
    }

    /// <summary>
    /// Enumeration of flags for <see cref="ComWrappers.GetOrCreateObjectForComInstance(IntPtr, CreateObjectFlags, object?)"/>.
    /// </summary>
    [Flags]
    public enum CreateObjectFlags
    {
        None = 0,

        /// <summary>
        /// Indicate if the supplied external COM object implements the <see href="https://docs.microsoft.com/windows/win32/api/windows.ui.xaml.hosting.referencetracker/nn-windows-ui-xaml-hosting-referencetracker-ireferencetracker">IReferenceTracker</see>.
        /// </summary>
        TrackerObject = 1,

        /// <summary>
        /// Ignore any internal caching and always create a unique instance.
        /// </summary>
        UniqueInstance = 2,
    }

    /// <summary>
    /// Class for managing wrappers of COM IUnknown types.
    /// </summary>
    [CLSCompliant(false)]
    public abstract partial class ComWrappers
    {
        /// <summary>
        /// Interface type and pointer to targeted VTable.
        /// </summary>
        public struct ComInterfaceEntry
        {
            /// <summary>
            /// Interface IID.
            /// </summary>
            public Guid IID;

            /// <summary>
            /// Memory must have the same lifetime as the memory returned from the call to <see cref="ComputeVtables(object, CreateComInterfaceFlags, out int)"/>.
            /// </summary>
            public IntPtr Vtable;
        }

        /// <summary>
        /// ABI for function dispatch of a COM interface.
        /// </summary>
        public struct ComInterfaceDispatch
        {
            public IntPtr vftbl;

            /// <summary>
            /// Given a <see cref="System.IntPtr"/> from a generated VTable, convert to the target type.
            /// </summary>
            /// <typeparam name="T">Desired type.</typeparam>
            /// <param name="dispatchPtr">Pointer supplied to VTable function entry.</param>
            /// <returns>Instance of type associated with dispatched function call.</returns>
            public static unsafe T GetInstance<T>(ComInterfaceDispatch* dispatchPtr) where T : class;
        }

        /// <summary>
        /// Create an COM representation of the supplied object that can be passed to an non-managed environment.
        /// </summary>
        /// <param name="instance">A GC Handle to the managed object to expose outside the .NET runtime.</param>
        /// <param name="flags">Flags used to configure the generated interface.</param>
        /// <returns>The generated COM interface that can be passed outside the .NET runtime.</returns>
        public IntPtr GetOrCreateComInterfaceForObject(object instance, CreateComInterfaceFlags flags);

        /// <summary>
        /// Compute the desired VTables for <paramref name="obj"/> respecting the values of <paramref name="flags"/>.
        /// </summary>
        /// <param name="obj">Target of the returned VTables.</param>
        /// <param name="flags">Flags used to compute VTables.</param>
        /// <param name="count">The number of elements contained in the returned memory.</param>
        /// <returns><see cref="ComInterfaceEntry" /> pointer containing memory for all COM interface entries.</returns>
        /// <remarks>
        /// All memory returned from this function must either be unmanaged memory, pinned managed memory, or have been
        /// allocated with the <see cref="System.Runtime.CompilerServices.RuntimeHelpers.AllocateTypeAssociatedMemory(Type, int)"/> API.
        ///
        /// If the interface entries cannot be created and <code>null</code> is returned, the call to <see cref="ComWrappers.GetOrCreateComInterfaceForObject(object, CreateComInterfaceFlags)"/> will throw a <see cref="System.ArgumentNullException"/>.
        /// </remarks>
        protected unsafe abstract ComInterfaceEntry* ComputeVtables(object obj, CreateComInterfaceFlags flags, out int count);

        /// <summary>
        /// Get the currently registered managed object or creates a new managed object and registers it.
        /// </summary>
        /// <param name="externalComObject">Object to import for usage into the .NET runtime.</param>
        /// <param name="flags">Flags used to describe the external object.</param>
        /// <param name="wrapper">An optional <see cref="object"/> to be used as the wrapper for the external object</param>
        /// <returns>Returns a managed object associated with the supplied external COM object.</returns>
        /// <remarks>
        /// Providing a <paramref name="wrapper"/> instance means <see cref="ComWrappers.GetOrCreateObjectForComInstance(IntPtr, CreateObjectFlags, object?)"/>
        /// will not be called.
        ///
        /// If the <paramref name="wrapper"/> instance already has an associated external object a <see cref="System.NotSupportedException"/> will be thrown.
        /// </remarks>
        public object GetOrCreateObjectForComInstance(IntPtr externalComObject, CreateObjectFlags flags, object? wrapper = null);

        /// <summary>
        /// Create a managed object for the object pointed at by <paramref name="externalComObject"/> respecting the values of <paramref name="flags"/>.
        /// </summary>
        /// <param name="externalComObject">Object to import for usage into the .NET runtime.</param>
        /// <param name="flags">Flags used to describe the external object.</param>
        /// <returns>Returns a managed object associated with the supplied external COM object.</returns>
        /// <remarks>
        /// If the object cannot be created and <code>null</code> is returned, the call to <see cref="ComWrappers.GetOrCreateObjectForComInstance(IntPtr, CreateObjectFlags, object?)"/> will throw a <see cref="System.ArgumentNullException"/>.
        /// </remarks>
        protected abstract object? CreateObject(IntPtr externalComObject, CreateObjectFlags flags);

        /// <summary>
        /// Called when a request is made for a collection of objects to be released.
        /// </summary>
        /// <param name="objects">Collection of objects to release.</param>
        /// <remarks>
        /// The default implementation of this function throws <see cref="System.NotImplementedException"/>.
        /// </remarks>
        protected virtual void ReleaseObjects(IEnumerable objects);

        /// <summary>
        /// Register this class's implementation to be used as the single global instance.
        /// </summary>
        /// <remarks>
        /// This function can only be called a single time. Subsequent calls to this function will result
        /// in a <see cref="System.InvalidOperationException"/> being thrown.
        ///
        /// Scenarios where the global instance may be used are:
        ///  * Object tracking via the <see cref="CreateComInterfaceFlags.TrackerSupport" /> and <see cref="CreateObjectFlags.TrackerObject" /> flags.
        ///  * Usage of COM related Marshal APIs.
        /// </remarks>
        public void RegisterAsGlobalInstance();

        /// <summary>
        /// Get the runtime provided IUnknown implementation.
        /// </summary>
        /// <param name="fpQueryInterface">Function pointer to QueryInterface.</param>
        /// <param name="fpAddRef">Function pointer to AddRef.</param>
        /// <param name="fpRelease">Function pointer to Release.</param>
        protected static void GetIUnknownImpl(out IntPtr fpQueryInterface, out IntPtr fpAddRef, out IntPtr fpRelease);
    }
}

[jkoritzinsky]

Design has been validated by the CsWinRT team.

[hughbe]

@AaronRobinsonMSFT Would this API allow for converting an IntPtr into a managed ComImport interface? I'm thinking about the following use case as an example

[ComImport]
[Guid("0000010C-0000-0000-C000-000000000046")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public unsafe interface IPersist
{
    [PreserveSig]
    int GetClassID(
        Guid* pClassID);
}

public unsafe void Method()
{
    var control = new Control();
    IntPtr pUnk = Marshal.GetIUnknownForObject(control);
    Guid iid = typeof(IPersist).GUID;
    int hr = Marshal.QueryInterface(pUnk, ref iid, out IntPtr pPersist);
    Debug.Assert(hr == 0);
    IPersist persist = ???

    // Do something with persist. E.g.
    Guid classId;
    hr = persist.GetClassID(&classId);
    Debug.Assert(hr == 0);
}

The class Control implements IPersist so obviously its com callable wrapper can query the IPersist interface. This means I can pass control to interop/PInvoke where the native code for example expects an IPersist object.

However, what I actually want to do is to call methods on the IPersist interface and call methods against it from c# just as I could do from native code.

I can't directly cast control to the interface here, as the interface that Control implements is internal to its own assembly, but is exposed to COM.

Any ideas?

[AaronRobinsonMSFT]

@hughbe I am going to assume we are talking about the WinForms Control.

This API could perform the desired actions, but that would be a decent amount of work and isn't scalable. VTable layouts would need to be defined and allocated and most of that code is monotonous and easy to get wrong. There is a test that shows an example of consuming a trivial IUnknown based interface.

Instead I would recommend using the dynamic keyword and simply call the method. The COM support for dynamic uses reflection and doesn't respect access modifiers so you should be able to call the desired function as follows:

public unsafe void Method()
{
    dynamic control = new Control();

    Guid classId;
    hr = control.GetClassID(&classId);
    Debug.Assert(hr == 0);
}

If the dynamic keyword is too much magic, it is possible to use reflection manually and call the desired function, but that code is very verbose.

The ComWrappers API is intended for a much lower level scenario - the entire replacement of the built-in CCW/RCW support. This API coupled with a source generation process can be used to avoid any code generation at run time and thus be more AOT friendly. Additionally it permits consumers to define marshaling semantics instead of relying on runtime changes. This API is intended for building tools like SharpGenTools.