fix and improve AppImage #324

Samueru-sama · 2025-01-19T13:37:26Z

I have not tested beyond launching the AppImage on alpine linux, which is usually a good enough first test.

Let me know if there are any issues.

edit: Now the AppImage follows the system theme if you use qt6ct for that.

It also can load separate stylesheet files by either placing and renaming the file with the name of the appimage + .stylesheet or directly setting the env variable to the location of the file.

edit2: closes pkgforge-dev/Anylinux-AppImages#3 #323

Rosalie241 · 2025-01-19T18:26:58Z

I'm not a huge fan of the big size increase, it goes from 80MB -> 140MB which is a big jump.

I've also tried it for a little bit and it has a few issues.

Netplay, it fails to initialize TLS, you should be able to reproduce this by opening the Netplay -> Create Session dialog, you can view the qt log by setting RMG_SHOW_DEBUG_QT_MESSAGES=1:

[QT WARNING] No TLS backend is available
[QT WARNING] No functional TLS backend was found
[QT WARNING] QSslSocket::connectToHostEncrypted: TLS initialization failed

Vulkan video plugins do not work, you can try this by going to Settings -> Plugins and changing the RSP and Video plugin to paraLLEl and booting a game, and then you can view the error log in View -> Log:

[GFX]   [ERROR]   Failed to create Vulkan instance.
[GFX]   [ERROR]   Failed to create Vulkan instance.
[GFX]   [ERROR]   Could not start GFX plugin

From a quick look, it seems to be missing some vulkan extensions, on a working AppImage you see the following:

[GFX]   [INFO]    Layer count: 9
[GFX]   [INFO]    Found layer: VK_LAYER_VALVE_steam_overlay_32.
[GFX]   [INFO]    Found layer: VK_LAYER_VALVE_steam_overlay_64.
[GFX]   [INFO]    Found layer: VK_LAYER_VALVE_steam_fossilize_32.
[GFX]   [INFO]    Found layer: VK_LAYER_VALVE_steam_fossilize_64.
[GFX]   [INFO]    Found layer: VK_LAYER_MANGOHUD_overlay_x86.
[GFX]   [INFO]    Found layer: VK_LAYER_MANGOAPP_overlay.
[GFX]   [INFO]    Found layer: VK_LAYER_MANGOHUD_overlay_x86_64.
[GFX]   [INFO]    Found layer: VK_LAYER_MANGOAPP_overlay.
[GFX]   [INFO]    Found layer: VK_LAYER_MESA_device_select.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_device_group_creation.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_display.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_external_fence_capabilities.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_external_memory_capabilities.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_external_semaphore_capabilities.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_get_display_properties2.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_get_physical_device_properties2.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_get_surface_capabilities2.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_surface.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_surface_protected_capabilities.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_wayland_surface.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_xcb_surface.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_xlib_surface.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_acquire_drm_display.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_acquire_xlib_display.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_debug_report.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_debug_utils.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_direct_mode_display.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_headless_surface.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_surface_maintenance1.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_swapchain_colorspace.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_display_surface_counter.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_portability_enumeration.
[GFX]   [INFO]    Enabling instance extension: VK_LUNARG_direct_driver_loading.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_debug_utils.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_get_surface_capabilities2.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_surface_maintenance1.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_swapchain_colorspace.

while on a non-working AppImage like the one you have, you can see:

[GFX]   [INFO]    Layer count: 9
[GFX]   [INFO]    Found layer: VK_LAYER_VALVE_steam_overlay_32.
[GFX]   [INFO]    Found layer: VK_LAYER_VALVE_steam_overlay_64.
[GFX]   [INFO]    Found layer: VK_LAYER_VALVE_steam_fossilize_32.
[GFX]   [INFO]    Found layer: VK_LAYER_VALVE_steam_fossilize_64.
[GFX]   [INFO]    Found layer: VK_LAYER_MANGOHUD_overlay_x86.
[GFX]   [INFO]    Found layer: VK_LAYER_MANGOAPP_overlay.
[GFX]   [INFO]    Found layer: VK_LAYER_MANGOHUD_overlay_x86_64.
[GFX]   [INFO]    Found layer: VK_LAYER_MANGOAPP_overlay.
[GFX]   [INFO]    Found layer: VK_LAYER_MESA_device_select.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_debug_report.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_debug_utils.
[GFX]   [INFO]    Enabling instance extension: VK_KHR_portability_enumeration.
[GFX]   [INFO]    Enabling instance extension: VK_LUNARG_direct_driver_loading.
[GFX]   [INFO]    Enabling instance extension: VK_EXT_debug_utils.

I also have another question, considering you're shipping libGL, libEGL and such, do nonfree drivers like the nvidia drivers still work with this?

Samueru-sama · 2025-01-19T18:42:18Z

I'm not a huge fan of the big size increase, it goes from 80MB -> 140MB which is a big jump.

The current AppImage does not launch on my system nor on the ones of the users that reported the issue.

Netplay, it fails to initialize TLS, you should be able to reproduce this by opening the Netplay -> Create Session dialog, you can view the qt log by setting RMG_SHOW_DEBUG_QT_MESSAGES=1:

mmmm, that's odd I guess the libraries needed for this are only dlopened when such function is used.

lib4bin uses a mode that launches the app with strace to get all the libraries it needs, however I think it is failing to work, likely because the app refuses to run properly when it detects that it is ran as sudo in the CI or something similar.

Vulkan video plugins do not work, you can try this by going to Settings -> Plugins and changing the RSP and Video plugin to paraLLEl and booting a game, and then you can view the error log in View -> Log:

I think I only bundled opengl, wasn't aware this used vulkan as well 👀

I also have another question, considering you're shipping libGL, libEGL and such, do nonfree drivers like the nvidia drivers still work with this?

Yes. Actually that's the reason because I switched to building on 24.04, because otherwise there is a bug on old mesa that prevents the proprietary nvidia driver from being used.

sharun still adds the host glvd to __EGL_VENDOR_LIBRARY_DIRS to let the proprietary nvidia driver be used.

It also does a similar thing for vulkan just in case.

Rosalie241 · 2025-01-19T18:47:23Z

The current AppImage does not launch on my system nor on the ones of the users that reported the issue.

That'd odd, have you tried debugging why it doesn't launch? Maybe try setting RMG_SHOW_DEBUG_QT_MESSAGES=1 or using strace?

sharun still adds the host glvd to __EGL_VENDOR_LIBRARY_DIRS to let the proprietary nvidia driver be used.

That's good

Samueru-sama · 2025-01-19T18:53:18Z

Netplay, it fails to initialize TLS, you should be able to reproduce this by opening the Netplay -> Create Session dialog, you can view the qt log by setting RMG_SHOW_DEBUG_QT_MESSAGES=1:

Alright it is very likely that this is because the qt tls plugin wasn't added at all, likely indicating that the app isn't running in the CI, hence none of the dlopened libs are being picked up.

If this still fails I will start doing proper debugging with strace.

EDIT: Updated artifact link, also putting it here

Rosalie241 · 2025-01-19T19:05:56Z

I'd rather have you look into why the existing AppImage doesn't work and attempt to fix that instead of switching to an entirely different system without a good reason, because I'm currently unconvinced that the new system is better than the existing one, and not knowing why the existing one doesn't work isn't very helpful either.

Samueru-sama · 2025-01-19T19:11:55Z

I'd rather have you look into why the existing AppImage doesn't work and attempt to fix that instead of switching to an entirely different system without a good reason, because I'm currently unconvinced that the new system is better than the existing one, and not knowing why the existing one doesn't work isn't very helpful either.

The existing appimage isn't great, relying on the host libraries is just not reliable you will often see issues of the AppImage not working on certain users. that's the good reason.

I'm currently unconvinced that the new system is better than the existing one

Because I forgot to add vulkan and the tls plugin? Note I'm assuming that fixed those issues, if it didn't I'm very sorry.

and not knowing why the existing one doesn't work isn't very helpful either.

I don't know either, I really don't bother with AppImages that don't bundle all the dependencies because it is just not worth the effort to troubleshoot something that it the end won't still work for all linux users.

Even if the current issue is fixed, that appimage won't still work on systems with a glibc older than that of 20.04, it won't also work on systems without glibc at all, it is just not worth the effort.

Rosalie241 · 2025-01-19T19:23:41Z

The existing appimage isn't great, relying on the host libraries is just not reliable you will often see issues of the AppImage not working on certain users. that's the good reason.

That's a fair point, sure.

EDIT: Updated artifact link, also putting it here

I can sadly confirm that both issues remain:

[QT WARNING] Incompatible version of OpenSSL (built with OpenSSL >= 3.x, runtime version is < 3.x)
[QT WARNING] The backend "cert-only" does not support QSslKey
[QT WARNING] Active TLS backend does not support key creation
[QT WARNING] The backend "cert-only" does not support QSslKey
[QT WARNING] Active TLS backend does not support key creation
[QT WARNING] The backend "cert-only" does not support QSslSocket
[QT WARNING] The backend named "cert-only" does not support TLS
[QT WARNING] QSslSocket::connectToHostEncrypted: TLS initialization failed

And the vulkan video plugin has the same issue, it seems like not all the required instance extensions are being exposed.

Samueru-sama · 2025-01-19T19:29:26Z

The existing appimage isn't great, relying on the host libraries is just not reliable you will often see issues of the AppImage not working on certain users. that's the good reason.

That's a fair point, sure.

EDIT: Updated artifact link, also putting it here

I can sadly confirm that both issues remain:
[QT WARNING] Incompatible version of OpenSSL (built with OpenSSL >= 3.x, runtime version is < 3.x)
[QT WARNING] The backend "cert-only" does not support QSslKey
[QT WARNING] Active TLS backend does not support key creation
[QT WARNING] The backend "cert-only" does not support QSslKey
[QT WARNING] Active TLS backend does not support key creation
[QT WARNING] The backend "cert-only" does not support QSslSocket
[QT WARNING] The backend named "cert-only" does not support TLS
[QT WARNING] QSslSocket::connectToHostEncrypted: TLS initialization failed
And the vulkan video plugin has the same issue, it seems like not all the required instance extensions are being exposed.

Alright I'm very sorry, gotta properly setup tests in that case.

EDIT: yeah...

Samueru-sama · 2025-01-19T19:41:54Z

Just in case @Rosalie241 go with the rest of your day and ignore everything I do here until I notify everything is ready, I will make sure everything is fixed.

I should get all of this sorted out in a few days at most (gotta go just now lol) and I also have something in mind that should reduce the size of the AppImage by about 20 MiB.

lestcape · 2025-01-19T19:56:38Z

The app image works for me and I don't see any issues, but I don't really have many complicated use cases.

@Rosalie241, if some dependency of an app image is not built in and that dependency changes in the next release or is missing in a future state, it's impossible for the app image to work. So the more dependencies are built in, the less chances there are for the appimage to fail in the future or work in an unplanned environment, but of course more built in libraries require more size of the image. So size can't be seen as a problem, but as the solution to the problem, since the main objective of an appimage is precisely to run without attachment to a specific environment.

Having an appimage that works only on the system where it was generated or on systems almost identical to it, is practically the same as compiling for a specific environment and therefore the essence of what it means to have an appimage is lost.

I would like that 20 years later, when nothing resembles what we have today, the appimage that was created today still works. That is the true utility of having a sufficiently robust appimage. But of course, the appimage should be complete and functional today so that it will be functional in the future.

Maybe what you should consider is whether you are going to generate an appimage in all releases or if, on the contrary, you are going to generate it only when there is a really noticeable change in your app. Is preferable to me 1 appimage per year that works for 20 years that 1 appimage per day that works for a week or only works on the distro where it was created.

By the way, I really liked your emulator. Especially the feature of being able to decide whether to let keyboard shortcuts repeat or not is something all emulators should have.

Samueru-sama · 2025-01-19T23:04:37Z

Turns out vulkan drivers were not installed at all in the CI 🤦

Now I'm able to launch a title with vulkan. I can also go into netplay without having an error, although I don't see any session to join to, not sure if I need to configure this.

Updated artifact link, size is now 155 MiB which is very close to the dolphin emu appimage, after all vulkan, opengl and all of qt are bundle in both.

I can reduce the final size to 140 MiB by using a different appimage runtime that uses dwarfs instead of squashfs, however this has the downside that it breaks everything that interacts with squash in appimages, most notably the AppImage thumbnailers that some users may want:

Some might suggest to build on alpine linux, but this will stop people from using the proprietary nvidia driver at all, since this is dlopened on the host means what we ship in the AppImage has to have glibc yes or yes.

This means the only other method to reduce size is to build some dependencies statically, but I really don't know how to do that.

I know it can reduce the size a lot, because I make this AppImage of mpv (bundles everything) and it used to be 80 MIB before I switched to their build script that automatically builds it linking ffmpeg statically, that reduced its size from 80 MiB to 22 MiB.

So to resume so far this PR:

Fixes the issue of the AppImage not opening, although I would want confirmation from @Dangonizer still.
Lets the AppImage follow the system theme if you have qt6ct, I guess more styles could be bundled but this means the size increases as well.
Adds delta updates with AppImageUpdate. Although I haven't modified the CI to also upload the .zsync file yet. (Note that with the dwarfs runtime this would still work just in case).
Adds wayland support since now I'm bundling the wayland plugins. Hasn't been test it though.

Dangonizer · 2025-01-19T23:25:52Z

Fixes the issue of the AppImage not opening, although I would want confirmation from @Dangonizer still.

Can confirm that it launches now. 👍

Rosalie241 · 2025-01-19T23:29:20Z

I can confirm the issues have been fixed, thank you.

Updated artifact link, size is now 155 MiB which is very close to the dolphin emu appimage, after all vulkan, opengl and all of qt are bundle in both.

That's rather big but at this point I'll just live with it, sadly RMG is already quite large due to Qt6 and shipping vosk-api.

I can reduce the final size to 140 MiB by using a different appimage runtime that uses dwarfs instead of squashfs, however this has the downside that it breaks everything that interacts with squash in appimages, most notably the AppImage thumbnailers that some users may want:

I think having a 'standard' runtime makes more sense.

Some might suggest to build on alpine linux, but this will stop people from using the proprietary nvidia driver at all, since this is dlopened on the host means what we ship in the AppImage has to have glibc yes or yes.

I don't even know if RMG will build on alphine linux, though I'm a little curious about the size for it now, might experiment with that in the future but for now glibc is fine also due to the nvidia support.

Lets the AppImage follow the system theme if you have qt6ct, I guess more styles could be bundled but this means the size increases as well.

I'd rather not ship this in the AppImage and have RMG handle it on it's own, RMG supports qt stylesheets by itself but it's an undocumented feature.

Adds delta updates with AppImageUpdate. Although I haven't modified the CI to also upload the .zsync file yet. (Note that with the dwarfs runtime this would still work just in case).

RMG handles updates for AppImage by itself, so the zsync stuff is unrequired.

Adds wayland support since now I'm bundling the wayland plugins. Hasn't been test it though.

I can confirm this works fine on my system by setting RMG_WAYLAND=1, the wayland support is locked behind an environment variable because I've experienced a lot of issues on GNOME last time I tested it on that DE.

Samueru-sama · 2025-01-19T23:43:34Z

I don't even know if RMG will build on alphine linux, though I'm a little curious about the size for it now, might experiment with that in the future but for now glibc is fine also due to the nvidia support.

It usually doesn't reduce the size more than 20%, the only thing that can reduce at lot is static linking some dependencies.

For example this is a Qt5 app that bundles everything, the reason it is that small is because the developers released a portable build that linked Qt5 statically, so I'm pretty much only adding glibc and the dynamic linker to the appimage to make it work on any linux system.

Of course it doesn't contain stuff like vulkan and opengl, but even with that the final size would be under 100 MiB.

I'd rather not ship this in the AppImage and have RMG handle it on it's own

So I remove qt6ct from the CI? Note that this is unlikely to reduce the size by more than 1 MIB. And it is a feature you normally get when it is a native distro package for example. just let me know anyway.

I would not really want to remove this other method that I added to load stylesheets outside the application, basically it works by placing the stylesheet file next to the appimage with the name of the appimage + .stylesheet or directly setting the env variable on your shell profile to point to such file.

EDIT

RMG handles updates for AppImage by itself, so the zsync stuff is unrequired.

It is still the official method for appimages to update, and allows the user to update appimages without having to open the application, I don't think that's a good idea to remove that.

Rosalie241 · 2025-01-19T23:47:18Z

So I remove qt6ct from the CI? Note that this is unlikely to reduce the size by more than 1 MIB. And it is a feature you normally get when it is a native distro package for example. just let me know anyway.
I would not really want to remove this other method that I added to load stylesheets outside the application, basically it works by placing the stylesheet file next to the appimage with the name of the appimage + .stylesheet or directly setting the env variable on your shell profile to point to such file.

Well alright, you can leave it in I suppose, I'll start reviewing your code changes now though.

Package/AppImage/Create.sh

Rosalie241 · 2025-01-20T13:44:40Z

Thank you for the PR ❤️

lestcape · 2025-01-20T13:52:20Z

Thanks so much, for this @Rosalie241 and @Samueru-sama. We finally have a funtional portable version of a good N64 emulator!! Have a good day.

Samueru-sama added 9 commits January 19, 2025 08:05

Update Create.sh

288f410

Delete Package/AppImage/AppRun

d39ba38

kek

93b6eee

Update build.yml

8989c80

kek2

355b481

Update Create.sh

9429826

Update build.yml

212233a

mmmmmmm

ec20f48

allow theming thru qt6ct or directly loading stylesheet files (#1)

1b97064

Samueru-sama mentioned this pull request Jan 19, 2025

An appimage for an N64 emulator pkgforge-dev/Anylinux-AppImages#3

Closed

add vulkan as well

7ad187f

add tls plugin 😅

c8bbcc0

strace shows this as missing when trying netplay

43f689b

add mesa-vulkan-drivers

2af66d7

Rosalie241 requested changes Jan 19, 2025

View reviewed changes

Package/AppImage/Create.sh Outdated Show resolved Hide resolved

Package/AppImage/Create.sh Outdated Show resolved Hide resolved

Package/AppImage/Create.sh Outdated Show resolved Hide resolved

Package/AppImage/Create.sh Outdated Show resolved Hide resolved

add requested changes

fa1fb98

Samueru-sama requested a review from Rosalie241 January 20, 2025 12:35

Rosalie241 merged commit 7f02a76 into Rosalie241:master Jan 20, 2025
4 checks passed

Samueru-sama mentioned this pull request Jan 20, 2025

MelonDS appimage crashes when a specific joystick is selected melonDS-emu/melonDS#2261

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix and improve AppImage #324

fix and improve AppImage #324

Samueru-sama commented Jan 19, 2025 •

edited

Loading

Rosalie241 commented Jan 19, 2025 •

edited

Loading

Samueru-sama commented Jan 19, 2025

Rosalie241 commented Jan 19, 2025

Samueru-sama commented Jan 19, 2025 •

edited

Loading

Rosalie241 commented Jan 19, 2025

Samueru-sama commented Jan 19, 2025 •

edited

Loading

Rosalie241 commented Jan 19, 2025 •

edited

Loading

Samueru-sama commented Jan 19, 2025 •

edited

Loading

Samueru-sama commented Jan 19, 2025

lestcape commented Jan 19, 2025 •

edited

Loading

Samueru-sama commented Jan 19, 2025

Dangonizer commented Jan 19, 2025

Rosalie241 commented Jan 19, 2025

Samueru-sama commented Jan 19, 2025 •

edited

Loading

Rosalie241 commented Jan 19, 2025

Rosalie241 commented Jan 20, 2025

lestcape commented Jan 20, 2025

fix and improve AppImage #324

fix and improve AppImage #324

Conversation

Samueru-sama commented Jan 19, 2025 • edited Loading

Rosalie241 commented Jan 19, 2025 • edited Loading

Samueru-sama commented Jan 19, 2025

Rosalie241 commented Jan 19, 2025

Samueru-sama commented Jan 19, 2025 • edited Loading

Rosalie241 commented Jan 19, 2025

Samueru-sama commented Jan 19, 2025 • edited Loading

Rosalie241 commented Jan 19, 2025 • edited Loading

Samueru-sama commented Jan 19, 2025 • edited Loading

Samueru-sama commented Jan 19, 2025

lestcape commented Jan 19, 2025 • edited Loading

Samueru-sama commented Jan 19, 2025

Dangonizer commented Jan 19, 2025

Rosalie241 commented Jan 19, 2025

Samueru-sama commented Jan 19, 2025 • edited Loading

Rosalie241 commented Jan 19, 2025

Rosalie241 commented Jan 20, 2025

lestcape commented Jan 20, 2025

Samueru-sama commented Jan 19, 2025 •

edited

Loading

Rosalie241 commented Jan 19, 2025 •

edited

Loading

Samueru-sama commented Jan 19, 2025 •

edited

Loading

Samueru-sama commented Jan 19, 2025 •

edited

Loading

Rosalie241 commented Jan 19, 2025 •

edited

Loading

Samueru-sama commented Jan 19, 2025 •

edited

Loading

lestcape commented Jan 19, 2025 •

edited

Loading

Samueru-sama commented Jan 19, 2025 •

edited

Loading