incus: update to 6.5

[dkwo]

• I tested the changes in this PR: yes
• I built this PR locally for my native architecture, (x86_64-glibc)

[classabbyamp]

could we list the optdeps in readme.voidlinux? #49267 (comment)

[duskmoss]

Opt deps I know about that would be nice to have documented in the readme, I'm not sure if I'm missing one for vm support or if something else is going on.

OCI support confirmed working

• spokeo umoci

VM support not working yet, but incus does recognize qemu support and tries to launch

• qemu edk2-ovmf
• Add export INCUS_EDK2_PATH=/usr/share/edk2/x64/ to a conf file in the incus service directory

[duskmoss]

Edit: spoke too soon

The remaining issue is no "OVMF_CODE.secboot.4m.fd" looks like we don't package this in edk2-ovmf @classabbyamp is there a specific reason not to?

But setting "security.secureboot" to false on a vm instance or on a profile they're attached to allows vms to be started.

edit: They're started but they're not booting up properly still. apparently hanging on loading inital ramdisk another round of debugging.

[classabbyamp]

they're called something else:

/usr/share/edk2/ia32/OVMF_CODE.secure.4m.fd
/usr/share/edk2/ia32/OVMF_CODE.secure.fd
/usr/share/edk2/x64/OVMF_CODE.secure.4m.fd
/usr/share/edk2/x64/OVMF_CODE.secure.fd

[duskmoss]

they're called something else:

/usr/share/edk2/ia32/OVMF_CODE.secure.4m.fd
/usr/share/edk2/ia32/OVMF_CODE.secure.fd
/usr/share/edk2/x64/OVMF_CODE.secure.4m.fd
/usr/share/edk2/x64/OVMF_CODE.secure.fd

thanks.! I'll look into if theres a way of configuring incus to find them.

symlinking them allows incus to start vm instances with secury.secureboot=true , but same hang on ramdisk I mentioned in my edit above.

Should I start a different thread to track progress here? Maybe letting the version bump go through and adding optional deps to the readme in a new pull request once they're more identified.

[dkwo]

@rekh127 Can you try adding virtiofsd? I haven't tested vm or oci myself yet, but https://linuxcontainers.org/incus/docs/main/installing/#install-incus-from-source mentions that for alpine.

So it seems we could use a conf file for the service, or at least export some variables in the service.

[duskmoss]

@rekh127 Can you try adding virtiofsd? I haven't tested vm or oci myself yet, but https://linuxcontainers.org/incus/docs/main/installing/#install-incus-from-source mentions that for alpine.

No change. but maybe later I'll try and go through the alpine list of dependencies for their incus-vm package and see if I can figure anything out

So it seems we could use a conf file for the service, or at least export some variables in the service.

Yes I think at minimum we should add the EDK2 path.

https://github.com/zabbly/incus/blob/daily/systemd/incusd

These are the ones the maintainer uses in his packaged version, we could consider some. I've added the UI path for mine to play with the web ui (grabbed from his deb package). Incus-agent is probably useful as well, but we will need to package incus-agent for it (it's like qemu guest agent, but for incus specifically).

I can start working on a new pull request to land after this one for conf file and documenting opt deps. I also intend to make a PR (could combine it) to move the current incus-user/check to incus since it actually checks if incusd is up since it runs as root.

[duskmoss]

In the mean time I'll test this PR locally as a sanity check :)

[duskmoss]

Re ovmf secure vs secboot, looks like theres no mechanism for changing that, its a list of files in incus code. https://github.com/lxc/incus/blob/0bf83c3be3f3c1d883ba40b27c29874d7fcee1cd/internal/server/instance/drivers/edk2/driver_edk2.go#L95

looks like arch fedora and debian all call the files 'secboot' too
@classabbyamp should we consider matching?

We could also maybe install a symlink?

[classabbyamp]

just patch incus to use what we call them

[duskmoss]

Isn't that going to be a hassle to maintain, and probably end up needing to patch other vm tools down the road?

[dkwo]

The author says here lxc/incus#475 (comment) that those dependencies are enough, if you do the 4 things mentioned there. Doesn't that work for you?

in particular, the part

add INCUS_OVMF_PATH=/usr/share/edk2/x64/ to incusd environment, Make sure to disable security.secureboot on VMs

[duskmoss]

The author says here lxc/incus#475 (comment) that those dependencies are enough, if you do the 4 things mentioned there. Doesn't that work for you?

in particular, the part

add INCUS_OVMF_PATH=/usr/share/edk2/x64/ to incusd environment, Make sure to disable security.secureboot on VMs

summarizing earlier comments
#51907 (comment)
#51907 (comment)
#51907 (comment)

I have installed those packages and am exporting INCUS_EDK2_PATH=/usr/share/edk2/x64/ in incus service's conf.

Those plus disabling secureboot or symlinking "secure" files to "secboot" (since the void package has the name different from other distros) gets VM's launching.

However they aren't booting properly, they get stuck with QEMU using 100% of one core, and the last message on the console is "loading init ramdisK".

Not sure whether this is an issue that Stéphane missed, thinking when he got launch succeeded that was the end of issues, like I did at first, or if it's a issue he didn't run into.

As I said before, I don't think this should block updating to 6.4. And I would like to start a pull request to go in after this one for the CONF file, and documenting opt deps if you're okay with that.

[classabbyamp]

pretty sure there's another distro using secure in the filename, i consulted several other distro packages when writing the edk2-ovmf package

[dkwo]

Well, in his case he could exec bash (you see he gets a shell), while now for me incus exec v1 bash errors with "VM agent isn't currently running", even though status is running, and core also at 100%.

As for conf file, the service already sources conf file, so isn't it up to the user to edit conf?

[duskmoss]

Well, in his case he could exec bash (you see he gets a shell), while now for me incus exec v1 bash errors with "VM agent isn't currently running", even though status is running, and core also at 100%.

oops yep I missed that.

As for conf file, the service already sources conf file, so isn't it up to the user to edit conf?

I would suggest we either document it with the opt deps in the readme, or add the . Adding the default makes sense to me, since it makes it discoverable, and it's also the one that will need to be set for all users using ovmf from packages.

[dkwo]

I can document everything in the readme, but I'd like to figure out why vm gets stuck first..

[dkwo]

Ok, so if I downgrade edk2-ovmf to 202311 then the alpine/edge vm starts just fine.

[dkwo]

The 202402 still works, but 202405 breaks vm.
@classabbyamp Is it possble to downgrade it, if people care about vm in incus?
@rekh127 I'll take your word for oci, have not tested it myself.
For now, I can put in the readme a note about disabling secureboot, and we can patch incus later.

[duskmoss]

Do you not want to add something about linking the secure boot files as I have in my PR?

[dkwo]

I don't like to edit stuff in /usr/share by hand: we should either fix incus or edk2 imo.

[classabbyamp]

does #52057 work? what architecture (i.e., could this be an aarch64 cross issue)?

changelog for 202405 is not huge: https://github.com/tianocore/edk2/releases/tag/edk2-stable202405

[dkwo]

The first broken commit is 0b8d631 the update to 202405, i.e. it works until 0cfc368 included. arch=x86_64-glibc. I can test the new PR.

[dkwo]

The new update (202408) also works for me.

[classabbyamp]

everything to patch for edk2 looks to be here, it's not a hard patch :

https://github.com/lxc/incus/blob/665db311c9b5d266ac231687e5de81a687492297/internal/server/instance/drivers/edk2/driver_edk2.go#L36

if we make sure the default filenames and paths match void's packaging, no need for the export or secureboot disablement instructions

[classabbyamp]

or, better yet, this looks like fallback paths. maybe upstream would take a patch adding our filenames to those arrays

[dkwo]

Yes, actually upstream suggested the same, I was just lazy, but let me do it.

[dkwo]

I can now do without exporting variables, but secure boot does not yet work.. lxc/incus#1170

[duskmoss]

Thanks for working on the patch, and for including my change to the check file. @dkwo

Do we need to wait for upstream to merge before we update to 6.4? Given that it's not a regression -VM support is already shakey is 6.3?

[dkwo]

it seems like the simple patch is not enough, as qemu is still using the first available path, unless i remove them by hand.
done a bunch of other small changes.

[dkwo]

One patch has been merged upstream. The other (removing all other paths not in Void) is still necessary, as qemu conf generator does not play well with fallback paths, see bug report lxc/incus#1179
Can either merge as is for now, or wait for proper upstream fix.

[dkwo]

Other issue fixed as well.

[dkwo]

I believe this is now working properly. All patches are upstream. Probably v6.5 is also coming soon..

[dkwo]

v6.5 out and tested.