Failed to resolve libClang or libClangSharp on fresh build on M1/Arm-64 Mac

[kkukshtel]

Trying to jump into this project again and again failing to run the project. Tried as a global tool first but encountered all the "normal" issues that seem to crop up with arm-64.

Decided to build managed from source, and running ClangSharpPinvokeGenerator from the output bin resulted in the following:

Failed to resolve libClang or libClangSharp.

Here's the build output, with libClang and libClangSharp hanging out right next to the output program. Worth noting zstd is installed with brew.

Unhandled exception: System.DllNotFoundException: Unable to load shared library 'libclang' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable:
dlopen(/Users/kyle/Workspace/ClangSharp/artifacts/bin/sources/ClangSharpPInvokeGenerator/Release/net7.0/libclang.dylib, 0x0001): Library not loaded: /opt/homebrew/opt/zstd/lib/libzstd.1.dylib
  Referenced from: /Users/kyle/Workspace/ClangSharp/artifacts/bin/sources/ClangSharpPInvokeGenerator/Release/net7.0/libclang.dylib
  Reason: tried: '/opt/homebrew/opt/zstd/lib/libzstd.1.dylib' (no such file), '/usr/local/lib/libzstd.1.dylib' (no such file), '/usr/lib/libzstd.1.dylib' (no such file)
dlopen(/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.1/libclang.dylib, 0x0001): tried: '/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.1/libclang.dylib' (no such file)
dlopen(libclang.dylib, 0x0001): tried: 'libclang.dylib' (no such file), '/usr/local/lib/libclang.dylib' (no such file), '/usr/lib/libclang.dylib' (no such file), '/Users/kyle/Workspace/clangsharppinvoketest/libclang.dylib' (no such file)
dlopen(/Users/kyle/Workspace/ClangSharp/artifacts/bin/sources/ClangSharpPInvokeGenerator/Release/net7.0/liblibclang.dylib, 0x0001): tried: '/Users/kyle/Workspace/ClangSharp/artifacts/bin/sources/ClangSharpPInvokeGenerator/Release/net7.0/liblibclang.dylib' (no such file)
dlopen(/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.1/liblibclang.dylib, 0x0001): tried: '/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.1/liblibclang.dylib' (no such file)
dlopen(liblibclang.dylib, 0x0001): tried: 'liblibclang.dylib' (no such file), '/usr/local/lib/liblibclang.dylib' (no such file), '/usr/lib/liblibclang.dylib' (no such file), '/Users/kyle/Workspace/clangsharppinvoketest/liblibclang.dylib' (no such file)
dlopen(/Users/kyle/Workspace/ClangSharp/artifacts/bin/sources/ClangSharpPInvokeGenerator/Release/net7.0/libclang, 0x0001): tried: '/Users/kyle/Workspace/ClangSharp/artifacts/bin/sources/ClangSharpPInvokeGenerator/Release/net7.0/libclang' (no such file)
dlopen(/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.1/libclang, 0x0001): tried: '/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.1/libclang' (no such file)
dlopen(libclang, 0x0001): tried: 'libclang' (no such file), '/usr/local/lib/libclang' (no such file), '/usr/lib/libclang' (no such file), '/Users/kyle/Workspace/clangsharppinvoketest/libclang' (no such file)
dlopen(/Users/kyle/Workspace/ClangSharp/artifacts/bin/sources/ClangSharpPInvokeGenerator/Release/net7.0/liblibclang, 0x0001): tried: '/Users/kyle/Workspace/ClangSharp/artifacts/bin/sources/ClangSharpPInvokeGenerator/Release/net7.0/liblibclang' (no such file)
dlopen(/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.1/liblibclang, 0x0001): tried: '/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.1/liblibclang' (no such file)
dlopen(liblibclang, 0x0001): tried: 'liblibclang' (no such file), '/usr/local/lib/liblibclang' (no such file), '/usr/lib/liblibclang' (no such file), '/Users/kyle/Workspace/clangsharppinvoketest/liblibclang' (no such file)

   at ClangSharp.Interop.clang.getClangVersion()
   at ClangSharp.PInvokeGenerator..ctor(PInvokeGeneratorConfiguration config, Func`2 outputStreamFactory) in /Users/kyle/Workspace/ClangSharp/sources/ClangSharp.PInvokeGenerator/PInvokeGenerator.cs:line 75
   at ClangSharp.Program.Run(InvocationContext context) in /Users/kyle/Workspace/ClangSharp/sources/ClangSharpPInvokeGenerator/Program.cs:line 692
   at System.CommandLine.Handler.<>c__DisplayClass1_0`1.<SetHandler>b__0(InvocationContext context)
   at System.CommandLine.Invocation.AnonymousCommandHandler.<>c__DisplayClass2_0.<.ctor>g__Handle|0(InvocationContext context)
   at System.CommandLine.Invocation.AnonymousCommandHandler.InvokeAsync(InvocationContext context)
   at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass18_0.<<UseParseErrorReporting>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass13_0.<<UseHelp>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass20_0.<<UseTypoCorrections>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseSuggestDirective>b__19_0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass17_0.<<UseParseDirective>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<RegisterWithDotnetSuggest>b__6_0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass9_0.<<UseExceptionHandler>b__0>d.MoveNext()

[tannergooding]

As described on other issues (such as #46) this is an issue with .NET Tools and there not being a way to ship RID specific versions of a tool.

Because of that, there is no way to provide libClang or libClangSharp in a way that is guaranteed to "just work". You must manually ensure that the relevant binary is resolvable either by it being installed via your package manager (libClang) or being copied to the folder where the tool was installed (libClangSharp).

If we were to do this in a non-RID specific way, the .NET tool NuGet package (that is the compressed zip) would be 70MB just to support win-x64, osx-x64, and ubuntu-18.04-x64. Adding in -arm64, -x86, or other Linux distros can balloon this further up to 300MB or more.

Please upvote issues such as NuGet/Home#10571 if the scenario is important to you.

[kkukshtel]

Ah okay gotcha. For what it's worth, I'd be happy to pay the price of the full download in the meantime to get a "just works" option. My use case is generating bindings locally and intermittently, so one local 300mb download isn't that bad.

I recognize for cloud/CI scenarios that isn't great, but the issue comes up enough here that it is worth considering releasing a package that is that that people could opt into using, with the necessary caveats.

People with specific RID scenarios can resolve them on their own to reduce the overhead, which is already what's happening now anyways.

[tannergooding]

so one local 300mb download isn't that bad.

It's not possible due to NuGet upload size restrictions.

The easiest thing is to either build locally, which should "just work" or to copy the one required binary from NuGet into the dotnet tool install. Instructions were updated in the readme to try and make this clearer.