Feature/detect dxvk

[Nemirtingas]

No description provided.

[Nemirtingas]

Needs further testing but looks fine on DX9, DX10, DX11.

[Edremon]

Doesn't seem to work. Does rendererToDetect need to be manually set when using dxvk?

[Nemirtingas]

What doesn't work? What issue do you encounter?
I tested on DX9, DX10, DX11, it works fine and DXVK gets detected, DX12 doesn't start, event without the overlay.
rendererToDetect is a new option to target only a set of renderers when you know them.

[Edremon]

I'm using your overlay in gbe_fork, it doesn't show up (I rebuild with feature/detect-dxvk). I wish I could have more logs, but never managed to get ingame_overlay to output logs, I don't really know how. spdlog dependency is not included in the repo, I believe I had managed to build with it once but can't remember where I failed next.

As for dx12, I believe it always worked with dxvk/vkd3d. On dx9/dx11, it fails on almost all games and this change doesn't fix that. As for example included, dx9_app.exe and dx11_app.exe always did work even without that change.

[Nemirtingas]

I'm surprised that dx9_app.exe worked because of the mutex in the dx9 present/presentex calling the swapchain present and locking again the same mutex throwing an exception.
To enable the logs, you need to clone spdlog in the deps directory and enable the cmake option INGAMEOVERLAY_USE_SPDLOG
Also, you should try to set the parameter preferSystemLibraries to false

[Edremon]

And where are those logs supposed to be outputted? I guess they should be in my standard output, but I don't see any. I took a further look at gbe_fork own log and it does seem like ingame_overlay manage to hook with these change:

Steam_Overlay::renderer_hook_proc() got renderer hook 00000000006E9E70 for 'C:\windows\system32\d3d11.dll (DXVK)'
Steam_Overlay::renderer_hook_proc() started renderer hook (result=0)

But, nothing is displayed when hitting the key combination. Setting preferSystemLibraries didn't change anything.

[Nemirtingas]

Vulkan got hooked on my test

[Edremon]

If I force RendererHookType_t::Vulkan, it doesn't detect any. Still didn't manage to get any log from ingame_overlay when used in gbe_fork.

[Nemirtingas]

How did you install dxvk? The version installed by winetricks is old (v0.41), while the latest dxvk version is 2.4.

[Edremon]

With AUR, I also tried using 2.4 binaries next to game files.

[Nemirtingas]

dxvk should not be put next to game files but in wine's system32 directory per the dxvk documentation.

[Edremon]

It is, I'm just talking about another setup that I also tried which is most commonly used by windows user that want to use DXVK on a game.

[Nemirtingas]

On windows, you don't have any other choice but to put it next to the game files. You are not supposed to replace the files in system32 :), but on wine you should.

[Edremon]

I know. I was just testing in case that fixed/changed anything, could be possible that you only tested dxvk on windows and used that setup.

[Nemirtingas]

The screenshot is from ubuntu

[Edremon]

Tested again dx11_app.exe. shift + f2 doesn't open anything, it also auto close after 10 seconds.

(324)[trace] - InGameOverlay::RendererDetector_t::DetectRenderer::<lambda_b9f918266c1c9cbfd2ba5bc3ea0c8862>::operator (){1426} - Started renderer detection.
(324)[info] - InGameOverlay::RendererDetector_t::_HookDX11{1239} - Hooked IDXGISwapChain::Present to detect DX Version
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(216)[info] - InGameOverlay::RendererDetector_t::_MyIDXGISwapChainPresent1{1031} - IDXGISwapChain::Present1
(324)[trace] - InGameOverlay::RendererDetector_t::DetectRenderer::<lambda_b9f918266c1c9cbfd2ba5bc3ea0c8862>::operator (){1498} - Renderer detection done 0x2ab6a20.
(320)[info] - InGameOverlay::DX11Hook_t::~DX11Hook_t{327} - DX11 Hook removed
(324)[trace] - InGameOverlay::RendererDetector_t::DetectRenderer::<lambda_b9f918266c1c9cbfd2ba5bc3ea0c8862>::operator (){1426} - Started renderer detection.
(324)[trace] - InGameOverlay::RendererDetector_t::DetectRenderer::<lambda_b9f918266c1c9cbfd2ba5bc3ea0c8862>::operator (){1498} - Renderer detection done 0x0.

[Edremon]

dx9_app.exe and dx12_app.exe are working correctly.


dx10_app.exe have the same problem as dx11_app.exe

[Nemirtingas]

This version should work on DX11, don't know about DX10

[Edremon]

Well, that fixed the examples.

But still not working when using it from gbe_fork. AFAIK, it fail in WindowsHook_t::StartHook since that function return false but, without log, I'm not exactly sure why it error.

[Edremon]

It's embarrassing but after checking again, I only rebuild the examples with your new commit (forgot to run cmake --install to build dir gbe_fork was using). After fixing that, game I was testing now also have working overlay. Thanks for making your overlay work with dxvk and fixing issue I had.

[Nemirtingas]

dxvk is still not officially 'supported', only detected. (DX12 lacks detection tho). It doesn't always work on Windows either.

[0xD3ADCODE]

@Edremon how did you managed to get working overlay with gbe_fork? i've managed to build latest version of fork with this branch, but on steam deck there is still no overlay in DX11 games

[0xD3ADCODE]

So, i managed where is the problem. For some reason, gbe_fork dll requires zlib1.dll to be in the same folder (dunno why). But i can confirm flawless work of overlay in DX11 games on Steam Deck with DXVK when using this branch. In overlay i see that DXVK is detected correctly

[Edremon]

@Edremon how did you managed to get working overlay with gbe_fork? i've managed to build latest version of fork with this branch, but on steam deck there is still no overlay in DX11 games

I don't have that many games to test, but Dark Souls Remastered for example work correctly when building with this PR but doesn't without.

So, i managed where is the problem. For some reason, gbe_fork dll requires zlib1.dll to be in the same folder (dunno why). But i can confirm flawless work of overlay in DX11 games on Steam Deck with DXVK when using this branch. In overlay i see that DXVK is detected correctly

Some static build failed somewhere in your build, steam_api.dll should be fully static, but that have nothing to do with ingame_overlay and shouldn't be discussed further here.

By the way, @Nemirtingas, I noticed that in some dx11 game, dxvk is not detected, it just says d3d11.dll renderer. Oddly enough, on this particular game, the overlay work just fine without this PR. The game that have this behavior is Dark Souls III.

[Nemirtingas]

Need a way to detect surface color order before merging it:

Overlay is using RGBA, the application uses BGRA