Mono linker failed with function pointer delegates

[hez2010]

Description

Mono linker failed with function pointer delegates.
Code:

#define UNICODE
#define WIN32
#include <cstring>

extern "C" __declspec(dllexport) char* __cdecl InvokeFun(char* (*foo)(int)) {
    return foo(5);
}

using System;
using System.Diagnostics;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;

namespace Test
{
    unsafe class Program
    {
        [DllImport("./Test.dll")]
        static extern string InvokeFun(delegate* cdecl<int, IntPtr> fun);

        [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })]
        public static IntPtr Foo(int x)
        {
            var str = Enumerable.Repeat("x", x).Aggregate((a, b) => $"{a}{b}");
            return Marshal.StringToHGlobalAnsi(str);
        }
        static void Main(string[] args)
        {
            var callback = (delegate* cdecl<int, nint>)(delegate*<int, nint>)&Foo;
            Console.WriteLine(InvokeFun(callback));
        }
    }
}

Now run dotnet publish with PublishTrimmed:

dotnet publish -c Release -r win-x64 /p:PublishTrimmed=true

Got exception from mono lnker:

  Mono.Linker.LinkerFatalErrorException: ILLink: error IL1005: Test.Program.InvokeFun(method): Error processing method 'Test.Program.InvokeFun(method)' in assembly 'Test.dll'
   ---> System.NullReferenceException: Object reference not set to an instance of an object.
     at Mono.Linker.Steps.MarkStep.ProcessInteropMethod(MethodDefinition method)
     at Mono.Linker.Steps.MarkStep.ProcessMethod(MethodDefinition method, DependencyInfo& reason)
     at Mono.Linker.Steps.MarkStep.ProcessQueue()
     --- End of inner exception stack trace ---
     at Mono.Linker.Steps.MarkStep.ProcessQueue()
     at Mono.Linker.Steps.MarkStep.ProcessPrimaryQueue()
     at Mono.Linker.Steps.MarkStep.Process()
     at Mono.Linker.Steps.MarkStep.Process(LinkContext context)
     at Mono.Linker.Pipeline.ProcessStep(LinkContext context, IStep step)
     at Mono.Linker.Pipeline.Process(LinkContext context)
     at Mono.Linker.Driver.Run(ILogger customLogger)

Configuration

.NET 5 preview 8

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[danmoseley]

@vitek-karas should this transfer to linker repo?

[EgorBo]

both Android and iOS samples from src/mono/netcore/sample directory fail to build because of this error.

[vitek-karas]

This has been already fixed - see the original linker bug dotnet/linker#1345.

I just tried the repro from this issue with latest 5.0.100-rc.1.20453.7 and I don't see any errors.
(Note that the syntax for function pointers has changed, now it looks like delegate* unmanaged[Cdecl]<int, IntPtr> fun)

Can you please try with the latest RC build?

[akoeplinger]

For the Android/iOS samples we build with the SDK specified in global.json which is still using a preview8 SDK:

runtime/global.json

Line 3 in a196d53

"version": "5.0.100-preview.8.20362.3",

.

[EgorBo]

@akoeplinger @vitek-karas updating global.json to 5.0.100-rc.1.20453.7 fixed the issue! thanks, I'll update it as part of my pr

[akoeplinger]

@EgorBo we can't do that, it needs to happen as part of the monthly infrastructure rollout (#41684)

[marek-safar]

@akoeplinger isn't the rollout scheduled for Monday?

[ghost]

Tagging subscribers to this area: @vitek-karas, @agocke
See info in area-owners.md if you want to be subscribed.

[akoeplinger]

yeah but it looks like it's not as simple as bumping the version, see #41684. So I'd let Viktor handle it.

/cc @ViktorHofer

[ViktorHofer]

Thanks for looping me in. Yes, the plan is that I update the SDK as part of September's monthly rollout. Will send out a mail later today about the rollout. Thanks

[ViktorHofer]

@akoeplinger the plan was to update the P8 final. What's the minimum required version that you need?

[akoeplinger]

@ViktorHofer it looks like the error still happens with 5.0.100-preview.8.20359.7 SDK but is fixed in 5.0.100-rc.1.20453.7 based on the comments in dotnet/linker#1345, so I think a Preview8 SDK is probably not enough.

[akoeplinger]

@EgorBo can you try whether it happens with 5.0.100-preview.8.20417.9 which is the released preview 8 SDK?

[ViktorHofer]

Let's baseline on P8 final (min version) but use an RC1 build as the target version.

[akoeplinger]

I did a quick check with the code from #41866 (comment) and it indeed still fails with 5.0.100-preview.8.20417.9.

[ghost]

Tagging subscribers to this area: @ViktorHofer
See info in area-owners.md if you want to be subscribed.

[ericstj]

I tested this on latest and it's working.

I needed to update the syntax per: https://github.com/dotnet/csharplang/blob/master/proposals/csharp-9.0/function-pointers.md

Now it is

using System;
using System.Diagnostics;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;

namespace Test
{
    unsafe class Program
    {
        [DllImport("./Test.dll")]
        static extern string InvokeFun(delegate* unmanaged[Cdecl]<int, IntPtr> fun);

        [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })]
        public static IntPtr Foo(int x)
        {
            var str = Enumerable.Repeat("x", x).Aggregate((a, b) => $"{a}{b}");
            return Marshal.StringToHGlobalAnsi(str);
        }
        static void Main(string[] args)
        {
            var callback = (delegate* unmanaged[Cdecl]<int, nint>)(delegate* managed<int, nint>)&Foo;
            Console.WriteLine(InvokeFun(callback));
        }
    }
}

Running on

dotnet --version
5.0.100-rc.2.20465.15

C:\scratch\linkerTest>dotnet.exe publish -c Release -r win-x64 /p:PublishTrimmed=true
Microsoft (R) Build Engine version 16.8.0-preview-20464-02+0220c5eae for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  linkerTest -> C:\scratch\linkerTest\bin\Release\net5.0\win-x64\linkerTest.dll
  Optimizing assemblies for size, which may change the behavior of the app. Be sure to test after publishing. See: https://aka.ms/dotnet-illink
  linkerTest -> C:\scratch\linkerTest\bin\Release\net5.0\win-x64\publish\