wine: no GPU acceleration with virtualgl inside xephyr/openbox

[Bluey26]

Hello, i use firejail quite often, and now i am experimenting with a nested X session for games.

This is because i am using Wayland and wine still have some unpolished things in it, (xwayland is used).

For this i have had to install a few packages, because i wanted to have 3d acceleration inside the nested X session (i am using Xephyr with openbox or i3).

A few ones that could give some insight of what i am looking to enable (OS = Archlinux):

lib32-mesa-utils 
lib32-virtualgl
virtualgl

What i am running is something like this:

xterm -hold -bg black -fg yellow -e 'Xephyr -br -ac -noreset -softCursor -screen 1000x600 :2 & sleep 1; DISPLAY=:2 vglrun -display :1 openbox'

Then, inside the openbox, i open a Xterm window and i launch my game, which basically executes:

firejail --profile=~/.config/firejail/wine_offline.profile wine .....

where wine_offline.profile is just the same as the default wine profile, with the addition of net none

if i use,instead:
wine .....

the game is able to detect the hardware acceleration (as desired), but if the firejail command is used, the game does not detect it, defaulting to llvm_pipe (it is not useable at all for this game).

Thanks in advance

[kmk3]

Basic debugging information is missing; please follow the bug report template:

• https://github.com/netblue30/firejail/issues/new?template=bug_report.md

[Bluey26]

Hello @kmk3, here is the information obtained by following those instructions:

I opened the Xephyr window with openbox as follows:

Xephyr -br -ac -noreset -softCursor -screen 1000x600 :2 &
sleep 1
DISPLAY=:2 vglrun -display :1 openbox

I know that is not the problem step, but the following one:

Inside Xephyr, I open xterm and I run:

LC_ALL=C firejail --net=none wine PATH_TO_EXE | tee log1.txt

log1.txt

Then I run:

LC_ALL=C firejail --net=none --noprofile  wine PATH_TO_EXE  | tee log2.txt

log2.txt

Those file does not seem to have the relevant information, that for some reason is not saved into the .txt, so I have taken a picture:

I think I need to load those libraries, but I am not sure on how to do it.

Obviously,running:

wine PATH_TO_EXE

Loads properly the mesa intel graphics using virtualgl:

(Compare the picture with line 7 of log1.txt and log2.txt)

tee has to be used, since I am unable to copy/paste text between the xephyr window and my host WM.

Here is the other asked info of the report:

Environment

• Name/version/arch of the Linux kernel (uname -srm): Linux 6.10.10-arch1-1 x86_64
• Name/version of the Linux distribution (e.g. "Ubuntu 20.04" or "Arch Linux"): Arch Linux
• Version of Firejail (firejail --version): extra/firejail 0.9.72-3

Checklist

Relevant item of the checklist:

• The issues is caused by firejail (i.e. running the program by path (e.g. /usr/bin/vlc) "fixes" it).

[kmk3]

Thanks for the details.

Does the problem happen with the normal wine.profile and without xephyr?

Inside Xephyr, I open xterm and I run:

LC_ALL=C firejail --net=none wine PATH_TO_EXE | tee log1.txt

log1.txt

Then I run:

LC_ALL=C firejail --net=none --noprofile  wine PATH_TO_EXE  | tee log2.txt

log2.txt

Those file does not seem to have the relevant information, that for some
reason is not saved into the .txt

File access is logged by firejail --trace=, so it might be more helpful.

What happens if you re-run with --trace=trace1.txt and --trace=trace2.txt
instead of tee?

Do you notice any meaningful difference in trace1.txt vs trace2.txt?

[kmk3]

LC_ALL=C firejail --net=none wine PATH_TO_EXE [...]

Also, make sure to use the full path to the program, to avoid potentially
running firejail inside firejail (due to firecfg).

Example:

LC_ALL=C firejail --net=none /usr/bin/wine PATH_TO_EXE [...]

[Bluey26]

Thanks for your help.

It does not happen outside of Xephyr, because there is GPU support outside:

firejail --net=none wine file.exe

shows:

[opengl_render_device_t::opengl_render_device_t]:  Driver vendor:  [Intel]
[opengl_render_device_t::opengl_render_device_t]:  Driver renderer:  [Mesa Intel(R) HD Graphics 620 (KBL GT2)]
[opengl_render_device_t::opengl_render_device_t]:  Driver version:  [4.6 (Compatibility Profile) Mesa 24.2.3-arch1.1]

The issue in here is that since I am running a "nested" WM inside xephyr (openbox in this example), the WM needs a GPU for games (for example) so I need to load the libraries.
In the Host WM that is already granted by default (no need for virtualgl in host WM).

Also, make sure to use the full path to the program, to avoid potentially

Ok, so I did the following now:

firejail --noprofile --net=none --trace=trace1.txt wine 'FULL_PATH_TO_EXE'

firejail --noprofile --net=none --trace=trace2.txt wine 'FULL_PATH_TO_EXE'

and the diff trace1.txt trace2.txt > difftrace.txt is a really big file, which does not seem to have anything relevant at first sight. (file has +37k lines).

I may sound stubborn, but maybe the issue could be solved by "whitelisting" libdlfaker.so and libvglfaker.so and maybe ld.so?

Those are mentioned in the 1st picture(black background). Those seem to be vglrun libraries, which may allow the game to detect the "fake" mesa intel card.

PS: I will have a look at those markdown urls, thanks!

[kmk3]

Also, make sure to use the full path to the program, to avoid potentially

Ok, so I did the following now:

firejail --noprofile --net=none --trace=trace1.txt wine 'FULL_PATH_TO_EXE'

firejail --noprofile --net=none --trace=trace2.txt wine 'FULL_PATH_TO_EXE'

By full path I meant the program that firejail executes (as wine might be a
symlink to firejail):

• firejail wine -> firejail /usr/bin/wine

The arguments to wine itself can use the short path.

Does the problem also happen with --noprofile or only when using
wine.profile?

Does it work when not using --net=none?

and the diff trace1.txt trace2.txt > difftrace.txt is a really big file,
which does not seem to have anything relevant at first sight. (file has +37k
lines).

The trace output contains dynamic information like pointer addresses, so
formatting is needed before diffing.

Also log1.txt and log2.txt are identical, so it's not clear if one of the
commands was executed outside xephyr or not.

To be clear, one of the traces should be from a successful execution (such as
outside of xephyr) in order for the comparison of the trace files to be useful.

Please try again and post trace1.txt and trace2.txt.

To simplify debugging, does it work with glxgears instead of wine?

Example:

# inside xephyr
firejail --noprofile --net=none --trace=glxgears_trace1.txt glxgears

# outside xephyr
firejail --noprofile --net=none --trace=glxgears_trace2.txt glxgears

I may sound stubborn, but maybe the issue could be solved by "whitelisting"
libdlfaker.so and libvglfaker.so and maybe ld.so?

There should be nothing blocking these libraries.

If you use private-lib anywhere, try disabling it.

To debug file access, see if you access them with:

# inside xephyr
firejail --noprofile --net=none ls -l /path/to/libdlfaker.so

Alternatively, try exploring the filesystem in the sandbox through a shell:

# inside xephyr
firejail --noprofile --net=none /bin/bash
ls -l /path/to/libdlfaker.so

[Bluey26]

Also log1.txt and log2.txt are identical, so it's not clear if one of the
commands was executed outside xephyr or not.

The commands were both executed inside xephyr

I have done the following:
firejail --noprofile --net=none ls -l /path/to/libdlfaker.so

and

firejail --net=none ls -l /path/to/libdlfaker.so

and the same commands with libvglfaker.so, so, as you said, the libraries are seen and the problem is not in there.

Example output:

> firejail --noprofile --net=none ls -l /usr/lib32/libvglfaker.so 

Parent pid 221435, child pid 221436
Child process initialized in 18.02 ms
-rwxr-xr-x 1 root root 1058644 Mar 10  2024 /usr/lib32/libvglfaker.so

Parent is shutting down, bye...

About the gears, since i am using wine, i also rely in lib32 for the libraries, and i executed(outside xephyr)

>firejail --noprofile --net=none --trace=glxgears_trace2.txt glxgears32

Parent pid 222260, child pid 222261
Child process initialized in 18.27 ms
ERROR: ld.so: object '/run/firejail/lib/libtrace.so' from /etc/ld.so.preload cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
Running synchronized to the vertical refresh.  The framerate should be
approximately the same as the monitor refresh rate.
304 frames in 5.0 seconds = 60.636 FPS
X connection to :1 broken (explicit kill or server shutdown).

The result was the gears moving and displaying.

The same happened with the other command(inside xephyr), so the glxgears seems to be working properly.

glxgears_trace2.txt and glxgears_trace1.txt where empty in the process, no trace generated (seems to work accordingly).

about the private-lib parameter, i have not seen it inside wine.profile both in /etc/firejail or any custom wine profiles i have(they are not used, i have checked them just in case). Although i have seen private-dev inside the default wine.profile,maybe it is not related to this issue.

Regarding the original trace1.txt and trace2.txt, i did the following as indicated:

Outside Xephyr(shows Mesa intel):

firejail --net=none --trace="trace1(outside).txt" /usr/bin/wine game.exe

Inside Xephyr(shows llvm instead):

firejail --net=none --trace="trace2(inside).txt" /usr/bin/wine game.exe

Then i did a diff with the above files, which i add in here( +21k lines):

diff trace1\(outside\).txt trace2\(inside-xephyr\).txt > difftrace-new.txt

I replaced some names because they were not relevant, for privacy reasons(username, exe names, folder name)

difftrace-new.txt

so far, i have seen some lines which may be relevant, like:

6247: < 18:wine64:open64 /usr/share/drirc.d/00-mesa-defaults.conf:14

7395: < 18:wine64:stat64 /home/my_username/.cache/mesa_shader_cache:0

or:

20943: > 6:wineserver:open /home/my_username/.wine/dosdevices/c:/windows/syswow64/opengl32.dll:282

Unfortunately this is far beyond my knowledges, so i only tried to search for keywords as "mesa" "vgl" "virtualgl", besides the names i have replaced.

I have seen some mentions to gamescope, which as you can see in the commands above, was not used in here, but it is installed and helps me with other games, outside Xephyr.

Thanks for your patience, and if you need any other file or command to be run, please tell me.