Support HLSL -> MSL on MacOS through DXC and Metal Shader Converter

[dafedidi]

Is your feature request related to a problem? Please describe.

I'm currently in the process of writing an application supporting Vulkan, DX12 and Metal where HLSL is the only shader langage used to author pipeline information. This application works on Windows, Linux and MacOS. This allow users to have, on all platforms, an end-to-end pipeline to create materials, shaders and asset without any constraint. I'm specifically talking about using the API, not the command-line here.

Since dxc can output SPIRV, we can easily handle Vulkan and DX12 is of course handled by default. However, while integrating Metal with the new Metal Shader Converter, I found some cracks that I would like to patch out. To be honest, DXC is not far from providing everything needed to support this and HLSL would become the default shader langage on all platforms. I believe that these issues should be easily patchable, but I might be wrong.

The OS/platform I'm using is Mac M1 Pro, with XCode 15 (if this information is useful).

Issue 1 was that DXIL produced by dxc on mac was not signed. I found nowhere how to get a valid dylib for the M series and the one provided for linux is not compatible.

warning: DXIL signing library (dxil.dll,libdxil.so) not found. Resulting DXIL will not be signed for use in release environments.

This issue is not that important because the generated DXIL will not be used directly to create a shader however, that warning is impossible to get rid of which broke the pipeline for error detections. Even using dxc --no-warnings to suppress it does not work.

Issue 2 is that it is impossible to get the reflection info on MacOs. Reflection requires DXC to support types like D3D12_SHADER_DESC but these are not compatible on MacOs because DirectX-Headers, when included creates compilation issues such as "duplicate type" or "unable to find rpc.h". It make sense since WinAdapters.h and basestd.h conflicts and redefine types. But even by fixing those, it would probably not work due to ID3D12ShaderReflection being a stub (I'm not 100% sure).

If Issue 2 was fixed and we could get reflection information, it would be possible to create the root signature for Metal Shader Converter. This data would be merged with the reflection data emitted by Metal Shader Converter and have HLSL be the default shader langage on metal. I believe it would open the door for most application using HLSL almost automatically without any hard dependencies (thanks to DXC and Metal compiler being both clang based).

Describe the solution you'd like

For issue 1, the best would be to provide the missing dylib. However, if that is not possible, we could simply add an option to disable that specifc warning, so end-user would only get "real" errors and warnings for shader they compile.

For issue 2, we would need a way to have the reflection data from DXC or find a way for DirectX-Header to play nice with dxc. I believe the best way would be for DXC to provide the reflection information since it has them during the compilation. We know it can be seen because running a command like dxc file.hlsl -E vs_main -T vs_6_0 spew out the information in the console about the shader compiled with the full reflection info we need (i.e what contains D3D12_SHADER_DESC). We could offer a function in dxc to provide JSON data or offer the full information like D3D12_SHADER_DESC under a different type.

Additional context

Platform used : MacOS Sonoma 14.1.1
Build system : CMake 3.1+
XCode version : 15

If needed, I can provide some sample code.

[dafedidi]

So I have been searching for ways to do a pull request to fix the reflection issue to which I can work on.
I see that we already have everything within the codebase to support such a use case - which already works on mac (build locally). The DXA tool is already able to print out the reflection information from a dxil file.

dxa -dumpreflection my_file.dxil

It has been compiled on MacOS and works without any issue. I'm happy about this discovery, because in some way we should be able to provide it in a different shape to end-users through the dxcapi.h either as LPCSTR (as JSON) or as D3D12_SHADER_DESC.

I'm continuing my search to make it work. I will probably fork soon to start the work once I have every piece needed to offer that functionnality.

[dafedidi]

Alright, I was able to have reflection data on MacOS. I will move my effort to #3686 to fix the issue 1 that I talked about in this ticket.

I will document here how I did it and how I found my way for anyone that would like to do the same. It will be a bit more verbose than it should be.

Like I stated in my previous post, I saw that the dxa tool was able to get the reflection data from a dxil file. Looking at the codebase, I expect the data to come from the tokens and that it made a sensible output from that. To my surprise, it used the structure D3D12_SHADER_DESC by including the DirectX-Header project. This means that I was able to compile and run an executable on mac that at one point was able to get the data correctly. I took the route to get that structure in my code as dxc does it.

By digging deeper, I understood that DXC would use a very specific version of DirectX-Header with git submodule status. I went and got the exact same version that my DXC that I have in my project. However, the release version from august 2023 that my project uses have some massive differences with the one I built (latest from main - at the time novembre 2023).

The first issue I got was that I had duplicates from basestd.h in wsl/stubs. It seems that the project (DirectX-Header) is aware of the issue and in the version from it, the whole file was commented out, which I did. I then found out that interface was not defined and DXC would define it themselves. I copied that with :

• #define interface struct

Which makes sense as MSVC does basically that on windows (and DXC on clang).

From there, the last issue found was a linker issue that some functions were not defined (those with _UUID). Well, it turns out that rpcndr.h had some code that would override the CROSS_PLATFORM_UUIDOF macro. I replaced all that code with a #define __RPCNDR_H_VERSION__ which is expected by dxcapi.h to work correctly.

From there, I could write the same code on Windows, Linux and MacOS to compile and get the reflection info as one would. It a pleasant surprise to have found out that DXC on Linux and MacOS is able to provide the info and now we can have a full shader pipeline with SPIRV-Cross. Wonderful.

I would recommend to simplify the work for consumers of the library either through documentation or a simpler means to get both DirectX-Headers to work well with DXC. At that point, I'm not sure to which repo I should work but my feeling is that it's a DirectX-Header issue more than a DXC issue.

I hope this will help anyone trying to make the flow DXC -> Metal Shader Converter easier.