dual 7900 xtx only single vulkan device found

[morphles]

I have desktop system 5950x, with dual 7900 xtx, and I get this:

./bin/main
Log start
main: build = 2464 (d0d5de42)
main: built with cc (Ubuntu 13.2.0-4ubuntu3) 13.2.0 for x86_64-linux-gnu
main: seed  = 1710871188
ggml_vulkan: Found 1 Vulkan devices:
Vulkan0: Radeon RX 7900 XTX | uma: 0 | fp16: 1 | warp size: 64

So it seems it does not consider other card as vulkan device. Any reason why that might be? Possibly that is because this is non workstations class CPU+MB and thus some PCIe lanes go through chipset instead of going directly to CPU? I hope it's not limiting factor and can be solved :). Since so far I was unable to get both cards working at the same time, hip blas keeps producing garbage on dual, and I was unable to make openCL work ether. Had really high hopes for vulkan, alas. Anyone has any idea if there is something I could do to make both cards visible as vulkan device?

[morphles]

Further checking:

==========
VULKANINFO
==========

Vulkan Instance Version: 1.3.250


Instance Extensions: count = 19
===============================
        VK_EXT_acquire_xlib_display            : extension revision 1
        VK_EXT_debug_report                    : extension revision 10
        VK_EXT_debug_utils                     : extension revision 2
        VK_EXT_direct_mode_display             : extension revision 1
        VK_EXT_display_surface_counter         : extension revision 1
        VK_KHR_device_group_creation           : extension revision 1
        VK_KHR_display                         : extension revision 23
        VK_KHR_external_fence_capabilities     : extension revision 1
        VK_KHR_external_memory_capabilities    : extension revision 1
        VK_KHR_external_semaphore_capabilities : extension revision 1
        VK_KHR_get_display_properties2         : extension revision 1
        VK_KHR_get_physical_device_properties2 : extension revision 2
        VK_KHR_get_surface_capabilities2       : extension revision 1
        VK_KHR_portability_enumeration         : extension revision 1
        VK_KHR_surface                         : extension revision 25
        VK_KHR_wayland_surface                 : extension revision 6
        VK_KHR_xcb_surface                     : extension revision 6
        VK_KHR_xlib_surface                    : extension revision 6
        VK_LUNARG_direct_driver_loading        : extension revision 1

Layers: count = 1
=================
VK_LAYER_AMD_switchable_graphics_64 (AMD switchable graphics layer) Vulkan version 1.3.267, layer version 1:
        Layer Extensions: count = 0
        Devices: count = 2
                GPU id = 0 (Radeon RX 7900 XTX)
                Layer-Device Extensions: count = 0

                GPU id = 1 (Radeon RX 7900 XTX)
                Layer-Device Extensions: count = 0

Device Groups:
==============
Group 0:
        Properties:
                physicalDevices: count = 2
                        Radeon RX 7900 XTX (ID: 0)
                        Radeon RX 7900 XTX (ID: 1)
                subsetAllocation = 0

        Present Capabilities:
                Radeon RX 7900 XTX (ID: 0):
                        Can present images from the following devices: count = 2
                                Radeon RX 7900 XTX (ID: 0)
                Radeon RX 7900 XTX (ID: 1):
                        Can present images from the following devices: count = 2
                                Radeon RX 7900 XTX (ID: 1)
                Present modes: count = 1
                        DEVICE_GROUP_PRESENT_MODE_LOCAL_BIT_KHR

So it seems that vulkaninfo sees both cards, just llama.cpp somehow decides one is not valid/available? Hm, I'll try disconnecting monitor totally, as I'm unsing machine over ssh anyway.

[morphles]

Disconnecting monitor completely did not help.

[Engininja2]

Set the environment variable GGML_VK_VISIBLE_DEVICES=0,1

[morphles]

@Engininja2 thanks, that's a bit of progress, and now at the start llama.ccp reports it sees two devices. But from rocm-smi I see it still tries to load on just one (and also due it running out of mem on larger models), I tried -ts 1,1 and GGML_VULKAN_DEVICE=0,1 in addition to GGML_VK_VISIBLE_DEVICES=0,1 but result is the same.

[morphles]

Ah seems I had -sm row in command line and that was the source of problem! It works now. Finally my AMD investment does something! 😂