nixos: Support systemd-gpt-auto-root #282022

ElvishJerricco · 2024-01-19T10:42:41Z

Description of changes

Rather than removing the broken generator in #277703, why not just support it?

Note: You must add your desired file system type to boot.initrd.supportedFilesystems.

Things done

I had to run `sudo sgdisk --typecode=2:4F68BCE3-E8CD-4DB1-96E7-FBCAF984B709 /dev/nvme3n1` in order to cause the right partition type code to be applied to the correct partition, so that `systemd-gpt-auto-generator` would detect it at boot time.

philiptaron · 2024-01-19T20:21:01Z

philiptaron/flock.nix@58dfcb6 is me testing this out on my real machine; it works great.

I had to do sgdisk --typecode=2:4F68BCE3-E8CD-4DB1-96E7-FBCAF984B709 /dev/nvme3n1 as the type of the partition wasn't right at my initial format time.

philiptaron

I really like having systemd-gpt-auto-generator (and the other generators) find and detect partitions.

nixos/modules/system/boot/stage-1.nix

nixos/modules/system/boot/systemd/initrd.nix

nixos/modules/tasks/filesystems.nix

philiptaron · 2024-02-01T18:29:09Z

I'd love to see this committed -- what's outstanding, do you think, @ElvishJerricco?

ElvishJerricco · 2024-02-02T01:30:47Z

The only thing that gives me pause is the handling of root=, and how it relates to #285254. I think we might instead want something like

options.boot.initrd.rootFS = mkOption {
  type = types.enum [ "fstab" "gpt-auto" ];
};

so that it can be extended with the other options that systemd supports in the future, rather than having this boot.initrd.gptAutoRoot.enable option.

philiptaron · 2024-02-03T19:06:36Z

The only thing that gives me pause is the handling of root=, and how it relates to #285254. I think we might instead want something like
options.boot.initrd.rootFS = mkOption {
  type = types.enum [ "fstab" "gpt-auto" ];
};
so that it can be extended with the other options that systemd supports in the future, rather than having this boot.initrd.gptAutoRoot.enable option.

That makes sense. It would default to fstab, you think? Or to something else?

msanft · 2024-02-09T09:13:47Z

The only thing that gives me pause is the handling of root=, and how it relates to #285254. I think we might instead want something like
options.boot.initrd.rootFS = mkOption {
  type = types.enum [ "fstab" "gpt-auto" ];
};
so that it can be extended with the other options that systemd supports in the future, rather than having this boot.initrd.gptAutoRoot.enable option.

I agree here. There are various methods of how systemd can setup the mounts, and I think rather than having one or another (i.e. fileSystems, systemd-veritysetup, or systemd-gpt-auto-root), this might be a more future-proof solution.

msanft · 2024-02-09T09:17:46Z

nixos/modules/tasks/filesystems.nix

+      {
+        assertion = config.boot.initrd.gptAutoRoot.enable -> config.boot.loader.systemd-boot.enable;
+        message = ''
+          'boot.initrd.gptAutoRoot.enable' requires systemd-boot.
+        '';
+      }


This is not the case if booting from a UKI with systemd-stub, where it will take care of setting up the EFI variables for the root disk: https://www.freedesktop.org/software/systemd/man/latest/systemd-stub.html#LoaderDevicePartUUID

Well, we don't include ways of using systemd-stub at this time.

Well, we have a builder for ukify? We maybe can't properly check if a UKI is used, but then this assertion should be removed

Yeah I agree with @msanft we should not assert on that. Booting without enabling sd-boot via the module system is an actual use-case we have for appliance images.

nixos/modules/tasks/filesystems.nix

arianvp · 2024-02-09T13:25:01Z

options.boot.initrd.rootFS = mkOption {
type = types.enum [ "fstab" "gpt-auto" ];
};

This doesn't work. root= parameter supports way more values! like root=/dev/vda2 root=dhcpv6 root=nfs etc.

the root= parameter furthermore is also parsed by the kernel and NFS daemons.

I personally use root=tmpfs in my own setup.

ElvishJerricco · 2024-02-09T14:35:20Z

@arianvp But we don't support other things in root= right now. Currently, we require the root fs in SYSTEMD_SYSROOT_FSTAB, and having that along with root= on the cmdline leads to failed boots.

nikstur

This is really cool! The assertion in filesystems.nix on whether the user explicitly includes the rootfs has been bugging me for a while now!

nikstur · 2024-02-17T20:02:52Z

nixos/modules/tasks/filesystems.nix

+      {
+        assertion = config.boot.initrd.gptAutoRoot.enable -> config.boot.loader.systemd-boot.enable;
+        message = ''
+          'boot.initrd.gptAutoRoot.enable' requires systemd-boot.
+        '';
+      }


Yeah I agree with @msanft we should not assert on that. Booting without enabling sd-boot via the module system is an actual use-case we have for appliance images.

nixos/modules/tasks/filesystems.nix

philiptaron

A couple notes.

If boot.initrd.supportedFilesystems = ["ext4"]; (for example) is not set, systemd-gpt-auto-generator will fail to find the root file system. That's a sharp edge, and one that almost made me believe that the thing didn't work.
I had to edit the disk earlier in order to set the correct GUID (sgdisk --typecode=2:4F68BCE3-E8CD-4DB1-96E7-FBCAF984B709 /dev/nvme3n1) as the type of the partition wasn't right at my initial format time. That's a sharp edge.
I note this in the logs from this boot:

Mar 03 14:46:35 zebul systemd-gpt-auto-generator[577]: Failed to chase block device, ignoring: No such file or directory
Mar 03 14:46:35 zebul systemd-gpt-auto-generator[577]: Probed fstype 'ext4' on partition /dev/disk/by-diskseq/1-part2.
Mar 03 14:46:35 zebul systemd-gpt-auto-generator[577]: Reading EFI variable /sys/firmware/efi/efivars/LoaderDevicePartUUID-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f.
Mar 03 14:46:35 zebul systemd-gpt-auto-generator[577]: Root partition not marked for growing the file system in the GPT partition table, not generating drop-in for systemd-growfs-root.service.

I don't think any of those logs are a problem, just noting them as "expected output" from the tool.

philiptaron · 2024-03-03T22:21:51Z

nixos/modules/system/boot/stage-1.nix

@@ -688,7 +688,7 @@ in

  config = mkIf config.boot.initrd.enable {
    assertions = [
-      { assertion = any (fs: fs.mountPoint == "/") fileSystems;
+      { assertion = !config.boot.initrd.systemd.enable -> any (fs: fs.mountPoint == "/") fileSystems;


If config.boot.initrd.systemd.enable is set, there is no assertion. If config.boot.initrd.systemd.enable is not set, there is an assertion when no "/" mountpoint is set.

philiptaron · 2024-03-03T22:27:05Z

nixos/modules/system/boot/systemd/initrd.nix

+        assertion = cfg.root == "fstab" -> any (fs: fs.mountPoint == "/") (builtins.attrValues config.fileSystems);
+        message = "The ‘fileSystems’ option does not specify your root file system.";


Ah, and here's the other assertion.

msanft

Thanks! I think this is a good solution that can also serve as a ground for future additions, e.g. using systemd-veritysetup-generator.

I actually think this should work with all other systemd generators that are capable of setting up the root mounts, as even if some have other options (e.g. roothash for systemd-veritysetup-generator), all that I have looked at seem to be fine with additionally having root.

msanft · 2024-03-04T08:21:38Z

2. I had to edit the disk earlier in order to set the correct GUID (sgdisk --typecode=2:4F68BCE3-E8CD-4DB1-96E7-FBCAF984B709 /dev/nvme3n1) as the type of the partition wasn't right at my initial format time. That's a sharp edge.

Yes, might not hurt to additionally document the requirement for DDI-conformance when using this, either in the release notes or the module documentation.

nikstur

Let's fix the merge conflict and this is good to go on my end.

ElvishJerricco · 2024-03-15T23:34:32Z

This also happens to fix #276374. It's easy to modify the hibernate.nix test to reflect this (two line diff). Should that be included in this PR?

So that resumeDevice isn't needed to make systemd stage 1 work.

ElvishJerricco · 2024-03-16T19:12:58Z

@ofborg test hibernate hibernate-systemd-stage-1

r-vdp

I tested this on my systems that are all running impermanence with a tmpfs as root to make sure that that use case still works, and I haven't encountered any issues.

philiptaron

Thanks for getting this in!

lopsided98 · 2024-03-18T23:54:05Z

nixos/modules/system/boot/systemd/initrd.nix

@@ -371,7 +389,12 @@ in {
      "autofs"
      # systemd-cryptenroll
    ] ++ lib.optional cfg.enableTpm2 "tpm-tis"
-    ++ lib.optional (cfg.enableTpm2 && !(pkgs.stdenv.hostPlatform.isRiscV64 || pkgs.stdenv.hostPlatform.isArmv7)) "tpm-crb";
+    ++ lib.optional (cfg.enableTpm2 && !(pkgs.stdenv.hostPlatform.isRiscV64 || pkgs.stdenv.hostPlatform.isArmv7)) "tpm-crb"
+    ++ lib.optional cfg.package.withEfi "efivarfs";


Shouldn't this be conditional on cfg.root == "gpt-auto"? I don't see why this feature should add kernel modules for users who don't enable it. Adding it is problematic for me because it doesn't exist on armv6l.

@lopsided98 efivarfs is useful for a few features, notably including hibernation. I'd rather disable it on platforms that don't support it than require it to be explicitly enabled.

If efivarfs doesn't exist on a platform, then I'm surprised we build systemd with withEfi = true. Maybe that's the change we should make?

withEfi defaults to stdenv.hostPlatform.isEfi. In practice, isEfi means "platforms where systemd EFI support can be built". There are unlikely to be any armv6l systems that use EFI, so it would be fine to remove it from isEfi, but I'm still not super happy with tying efivarfs in the initrd to systemd EFI support.

There is no way to override boot.initrd.availableKernelModules to remove an item, so we should be careful about what is added unconditionally. If someone wants to disable EFI support in their kernel for whatever reason, they also have to override systemd and rebuild the world. If hibernation needs efivarsfs, it should be automatically enabled in that module as well. I actually have a similar objection to autofs, but it has never caused enough problems for me to try fixing it.

@lopsided98

You don't have to rebuild the world to override systemd. You can just set systemd.package = pkgs.systemd.override { withEfi = false; };

Hibernation isn't really something we have a module for. It's just something systemd is able to do. So simply having systemctl means a user is able to attempt to hibernate.

We could have an option like boot.initrd.systemd.efiSupport with a default of cfg.package.withEfi so that it can be reasonably overridden. I still think this is wrong, however. Ideally, a system that doesn't have EFI should have systemd.package set to a systemd override with EFI disabled.

Alternatively, we could have a new option like boot.initrd.optionalKernelModules which lists modules that should be included if the kernel build has them. This would be helpful in other situations as well. Like the all-hardware.nix profile breaks when you build with the RPi kernel, since it disables many of the modules that all-hardware.nix has in availableKernelModules.

I actually have a similar objection to autofs, but it has never caused enough problems for me to try fixing it.

autofs is required for systemd to boot as far as I am aware. as systemd ships with some required .automount units

I actually have a similar objection to autofs, but it has never caused enough problems for me to try fixing it.

autofs is required for systemd to boot as far as I am aware. as systemd ships with some required .automount units

From the docs on CONFIG_AUTOFS_FS:

The automounter is a tool to automatically mount remote file systems on demand. This implementation is partially kernel-based to reduce overhead in the already-mounted case; this is unlike the BSD automounter (amd), which is a pure user space daemon.

To use the automounter you need the user-space tools from https://www.kernel.org/pub/linux/daemons/autofs/; you also want to answer Y to "NFS file system support", below.

To compile this support as a module, choose M here: the module will be called autofs.

If you are not a part of a fairly large, distributed network or don't have a laptop which needs to dynamically reconfigure to the local network, you probably do not need an automounter, and can say N here.

I only see one .automount unit that ships with systemd and it looks trivially replaceable.

…forseeable.

…for the forseeable." This reverts commit 4b2c89e. Testing to see if it's what's causing my activation woes.

philiptaron · 2024-03-22T22:03:39Z

It brings me no pleasure to report that setting gpt-auto broke activation. When I booted my system that has the root detected through systemd-gpt-auto-root, it no longer updates /etc to reflect what's in the system closure.

$ journalctl -b -10 -u initrd-nixos-activation.service
Mar 22 09:37:40 localhost systemd[1]: initrd-nixos-activation.service: starting held back, waiting for: initrd-fs.target
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: AssertPathExists=/etc/initrd-release succeeded.
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Will spawn child (service_enter_start): /nix/store/57zdzvsd1wcsfxbdrkpdz7hx35jb0ds9-unit-script-initrd-nixos-activation-start/bin/initrd-nixos-activation-start
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Passing 0 fds to service
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: About to execute: /nix/store/57zdzvsd1wcsfxbdrkpdz7hx35jb0ds9-unit-script-initrd-nixos-activation-start/bin/initrd-nixos-activation-start
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Forked /nix/store/57zdzvsd1wcsfxbdrkpdz7hx35jb0ds9-unit-script-initrd-nixos-activation-start/bin/initrd-nixos-activation-start as 495
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Changed dead -> start
Mar 22 09:37:43 localhost systemd[1]: Starting NixOS Activation...
Mar 22 09:37:43 localhost (on-start)[495]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
Mar 22 09:37:43 localhost (on-start)[495]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
Mar 22 09:37:43 localhost initrd-nixos-activation-start[495]: booting system configuration /nix/store/bnrs9syvy2gz7pmy1r5dd4jip672srhh-nixos-system-zebul-24.05.20240321.b0c93fc
Mar 22 09:37:43 localhost initrd-nixos-activation-start[495]: running activation script...
Mar 22 09:37:43 localhost initrd-nixos-activation-start[526]: chmod: changing permissions of '/bin': Read-only file system
Mar 22 09:37:43 localhost initrd-nixos-activation-start[527]: ln: failed to create symbolic link '/bin/.sh.tmp': Read-only file system
Mar 22 09:37:43 localhost initrd-nixos-activation-start[528]: mv: cannot stat '/bin/.sh.tmp': No such file or directory
Mar 22 09:37:43 localhost initrd-nixos-activation-start[524]: Activation script snippet 'binsh' failed (1)
Mar 22 09:37:43 localhost initrd-nixos-activation-start[531]: write_file '/var/lib/nixos/.tempqKJuf' - sysopen: Read-only file system at /nix/store/38hbyd1gwyszqslh3wycv5ly03dr75gj-update-users-groups.pl line 23.
Mar 22 09:37:43 localhost initrd-nixos-activation-start[524]: Activation script snippet 'users' failed (30)
Mar 22 09:37:43 localhost initrd-nixos-activation-start[524]: setting up /etc...
Mar 22 09:37:43 localhost initrd-nixos-activation-start[534]: Died at /nix/store/rg5rf512szdxmnj9qal3wfdnpfsx38qi-setup-etc.pl line 27.
Mar 22 09:37:43 localhost initrd-nixos-activation-start[524]: Activation script snippet 'etc' failed (30)
Mar 22 09:37:43 localhost initrd-nixos-activation-start[562]: chmod: changing permissions of '/usr/bin': Read-only file system
Mar 22 09:37:43 localhost initrd-nixos-activation-start[563]: ln: failed to create symbolic link '/usr/bin/.env.tmp': Read-only file system
Mar 22 09:37:43 localhost initrd-nixos-activation-start[564]: mv: cannot stat '/usr/bin/.env.tmp': No such file or directory
Mar 22 09:37:43 localhost initrd-nixos-activation-start[524]: Activation script snippet 'usrbinenv' failed (1)
Mar 22 09:37:43 localhost initrd-nixos-activation-start[495]: /nix/store/bnrs9syvy2gz7pmy1r5dd4jip672srhh-nixos-system-zebul-24.05.20240321.b0c93fc/prepare-root: line 133: /etc/machine-id: Read-only file system
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Child 495 belongs to initrd-nixos-activation.service.
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Main process exited, code=exited, status=0/SUCCESS (success)
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Deactivated successfully.
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Service will not restart (restart setting)
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Changed start -> dead
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Job 2 initrd-nixos-activation.service/start finished, result=done
Mar 22 09:37:43 localhost systemd[1]: Finished NixOS Activation.
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Consumed 119ms CPU time.
Mar 22 09:37:43 localhost systemd[1]: initrd-nixos-activation.service: Releasing resources...
Mar 22 09:37:48 zebul systemd[1]: initrd-nixos-activation.service: Collecting.

… for the forseeable." This reverts commit d3bb46c. Per Matrix discussion, we're going to add in `rw` to the kernel arguments.

ElvishJerricco requested a review from RaitoBezarius January 19, 2024 10:42

ElvishJerricco requested review from a team and dasJ as code owners January 19, 2024 10:42

github-actions bot added 6.topic: nixos 8.has: module (update) 6.topic: systemd labels Jan 19, 2024

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Jan 19, 2024

philiptaron approved these changes Jan 20, 2024

View reviewed changes

nixos/modules/system/boot/stage-1.nix Outdated Show resolved Hide resolved

nixos/modules/system/boot/systemd/initrd.nix Show resolved Hide resolved

nixos/modules/tasks/filesystems.nix Outdated Show resolved Hide resolved

delroth added the 12.approvals: 1 label Jan 20, 2024

This was referenced Jan 27, 2024

hibernation not being automatic using a swap file #276374

Closed

/ does not need to be mounted if roothash is specified in the kernel command line #285254

Open

msanft reviewed Feb 9, 2024

View reviewed changes

nixos/modules/tasks/filesystems.nix Outdated Show resolved Hide resolved

philiptaron mentioned this pull request Feb 9, 2024

systemd-stage-1: Remove systemd-gpt-auto-generator #277703

Closed

13 tasks

nikstur reviewed Feb 17, 2024

View reviewed changes

ElvishJerricco force-pushed the gpt-auto-root branch from b972fc6 to b50070e Compare March 3, 2024 07:51

ElvishJerricco requested review from nikstur, msanft, philiptaron and arianvp March 3, 2024 07:51

philiptaron added a commit to philiptaron/flock.nix that referenced this pull request Mar 3, 2024

Test out NixOS/nixpkgs#282022 again

81f6853

philiptaron approved these changes Mar 3, 2024

View reviewed changes

msanft approved these changes Mar 4, 2024

View reviewed changes

ElvishJerricco mentioned this pull request Mar 8, 2024

Timed out waiting for device /dev/gpt-auto-root. on aarch64-linux VM (systemd stage 1 + zfs + remote unlocking) #293586

Closed

wegank added 12.approvals: 2 and removed 12.approvals: 1 labels Mar 8, 2024

nikstur approved these changes Mar 10, 2024

View reviewed changes

nixos: Support systemd-gpt-auto-root

f1731f2

ElvishJerricco force-pushed the gpt-auto-root branch from b50070e to f1731f2 Compare March 15, 2024 23:32

philiptaron approved these changes Mar 16, 2024

View reviewed changes

nixos/tests/hibernate: Use EFI

3758681

So that resumeDevice isn't needed to make systemd stage 1 work.

ElvishJerricco requested review from nikstur and philiptaron March 16, 2024 19:16

r-vdp approved these changes Mar 17, 2024

View reviewed changes

nikstur approved these changes Mar 18, 2024

View reviewed changes

nikstur merged commit a1c4f0a into NixOS:master Mar 18, 2024
24 checks passed

philiptaron reviewed Mar 18, 2024

View reviewed changes

lopsided98 reviewed Mar 18, 2024

View reviewed changes

philiptaron added a commit to philiptaron/flock.nix that referenced this pull request Mar 21, 2024

Now that NixOS/nixpkgs#282022 has landed, switch to gpt-auto for the …

4b2c89e

…forseeable.

philiptaron mentioned this pull request Mar 22, 2024

linuxPackages.nvidiaPackages.production: 535.154.05 -> 550.67 #297237

Merged

13 tasks

philiptaron added a commit to philiptaron/flock.nix that referenced this pull request Mar 22, 2024

Revert "Now that NixOS/nixpkgs#282022 has landed, switch to gpt-auto …

d3bb46c

…for the forseeable." This reverts commit 4b2c89e. Testing to see if it's what's causing my activation woes.

philiptaron mentioned this pull request Mar 22, 2024

nixos/systemd/initrd: make systemd mount root as rw when using gpt-auto to find it #298201

Merged

8 tasks

KiaraGrouwstra mentioned this pull request Apr 24, 2024

error 'Timed out waiting for device' on boot using boot.initrd.systemd.enable = true #306645

Closed

philiptaron mentioned this pull request Sep 3, 2024

nixos: support dm-verity #336983

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos: Support systemd-gpt-auto-root #282022

nixos: Support systemd-gpt-auto-root #282022

ElvishJerricco commented Jan 19, 2024 •

edited

Loading

philiptaron commented Jan 19, 2024

philiptaron left a comment

philiptaron commented Feb 1, 2024

ElvishJerricco commented Feb 2, 2024

philiptaron commented Feb 3, 2024

msanft commented Feb 9, 2024

msanft Feb 9, 2024

ElvishJerricco Feb 9, 2024

msanft Feb 9, 2024

nikstur Feb 17, 2024

arianvp commented Feb 9, 2024 •

edited

Loading

ElvishJerricco commented Feb 9, 2024

nikstur left a comment

nikstur Feb 17, 2024

philiptaron left a comment

philiptaron Mar 3, 2024

philiptaron Mar 3, 2024

msanft left a comment

msanft commented Mar 4, 2024

nikstur left a comment

ElvishJerricco commented Mar 15, 2024

ElvishJerricco commented Mar 16, 2024

r-vdp left a comment

philiptaron left a comment

lopsided98 Mar 18, 2024

ElvishJerricco Mar 19, 2024

lopsided98 Mar 19, 2024 •

edited

Loading

ElvishJerricco Mar 19, 2024

arianvp Mar 19, 2024 •

edited

Loading

philiptaron Mar 19, 2024

philiptaron commented Mar 22, 2024 •

edited

Loading

		assertion = cfg.root == "fstab" -> any (fs: fs.mountPoint == "/") (builtins.attrValues config.fileSystems);
		message = "The ‘fileSystems’ option does not specify your root file system.";

nixos: Support systemd-gpt-auto-root #282022

nixos: Support systemd-gpt-auto-root #282022

Conversation

ElvishJerricco commented Jan 19, 2024 • edited Loading

Description of changes

Things done

philiptaron commented Jan 19, 2024

philiptaron left a comment

Choose a reason for hiding this comment

philiptaron commented Feb 1, 2024

ElvishJerricco commented Feb 2, 2024

philiptaron commented Feb 3, 2024

msanft commented Feb 9, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arianvp commented Feb 9, 2024 • edited Loading

ElvishJerricco commented Feb 9, 2024

nikstur left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

philiptaron left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msanft left a comment

Choose a reason for hiding this comment

msanft commented Mar 4, 2024

nikstur left a comment

Choose a reason for hiding this comment

ElvishJerricco commented Mar 15, 2024

ElvishJerricco commented Mar 16, 2024

r-vdp left a comment

Choose a reason for hiding this comment

philiptaron left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lopsided98 Mar 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arianvp Mar 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

philiptaron commented Mar 22, 2024 • edited Loading

ElvishJerricco commented Jan 19, 2024 •

edited

Loading

arianvp commented Feb 9, 2024 •

edited

Loading

lopsided98 Mar 19, 2024 •

edited

Loading

arianvp Mar 19, 2024 •

edited

Loading

philiptaron commented Mar 22, 2024 •

edited

Loading