Don't enable D3D12 debug layers as part of device creation

[aleino-nv]

As [1] shows, creating a D3D12 device and then enabling debug layers causes future device creation to fail.
That means enabling debug layers is a process-wide decision that should be done at startup, and not just before creating an individual device.

This helps to address shader-slang/slang#6172. In the case of the mentioned failure, the WebGPU backend happened to create a D3D12 device before Slang-RHI got to create a device and enable debug layers.

A new interface is added which lets the API user enable debug layers separately, before creating any devices.

After this, the fix for slang-test debug issue is shader-slang/slang#6226.

[1] shader-slang/slang#6172 (comment)

[skallweitNV]

I don't really like how this plays with the enableBackendValidation option in the DeviceDesc.
Maybe we should remove that option from DeviceDesc and have users enable backend debug layers using the new API.
For D3D12 we basically do what is in this PR. For Vulkan we'd have to store a global flag and check that on device creation.

[aleino-nv]

I don't really like how this plays with the enableBackendValidation option in the DeviceDesc. Maybe we should remove that option from DeviceDesc and have users enable backend debug layers using the new API. For D3D12 we basically do what is in this PR. For Vulkan we'd have to store a global flag and check that on device creation.

Could we make enableBackendValidation a parameter to getRHI, so that you have to decide when you create the instance?

[skallweitNV]

getRHI just returns a singleton, so adding a parameter there makes no sense IMO.

[aleino-nv]

getRHI just returns a singleton, so adding a parameter there makes no sense IMO.

I meant that getRHI should get the initial instance using this parameter as well.

[skallweitNV]

I think the current enableDebugLayers method is fine. We could add checks to fail if a D3D12 has already been created. But we can also merge the PR as is, I can remove enableBackendValidation later.

[aleino-nv]

I think the current enableDebugLayers method is fine. We could add checks to fail if a D3D12 has already been created. But we can also merge the PR as is, I can remove enableBackendValidation later.

Ok, I filed #151 for the removal.