Rework profile management #1434

svartkanin · 2022-08-27T09:06:18Z

Reworking the current profile mangement

With the current version there are some major challenges. It is tightly coupled with archinstall.storage which means that any selection in the menu will modify the storage. This makes it very difficult to reset settings and/or override them without forgetting to cleanup the ones not set.

These major changes aim to make the profile handling more flexible and dynamic and give users the ability to programmatically generate profiles. Unfortunately, this is a breaking change and the old version of profile definitions will not work anymore.

@Torxed I've transferred over all existing profiles in the repository, but I tried my best to support an automatic transfer for profiles that have been defined by users. Unfortunately, that wasn't feasible without having to cover dozens of edge cases, so this will inevitably a breaking change :(

Related issues that should be fixed with this upgrade

New features

Profiles as classes

All profiles are now defined as a class. This allows that profiles can be inherited from each other, and therefore users can build
profiles on top of other profiles. This will be essential when creating custom profiles :)

Programmatically create profiles

User will now have the ability to programmatically create profiles and inject them, as an example

class TestProfileV2(ProfileV2):
	def __init__(self):
		super().__init__('TestMe', ProfileType.CustomType)

       @property
	def packages(self) -> List[str]:
		return ['my_fav_packages']

	def install(self, install_session: 'Installer'):
		install_session.add_additional_packages(self.packages())

handler = ProfileHandler()
handler.add_custom_profile(TestProfileV2())

This also has the major benefit that no separate files have to be create anymore for profiles.

Multiple profiles per file

Currently there's a single profile definition per file. One can now create profiles programmatically as shown above, or specify them
as in the current way in a separate file.
However, now one can also specify multiple profiles per file, since the actual class definitions will be recognized as the profile.

my_profiles.py

class ProfileOne(ProfileV2):
    ...

class ProfileTwo(ProfileV2):
    ...

Ability to install multiple desktop profiles

There is now the option to install multiple desktop profiles, similar to multiple servers that already exists

Display the packages to be installed on the desktop profiles

Select which greeter should be installed

Each desktop profile can define a default greeter. When selecting one or multiple desktop profiles, the user will get the option
to chose which greeter should be installed with these profiles. As the default there will be a recommendation of the
default greeter (if only one or if all have the same default).

Create custom profiles in the menu

In the profile menu, there's also the option to create custom profiles

They will also be exported as configuration settings. If they have been created and set to disabled, and another profile (e.g. Desktop) is set then they will still be exported, this case any definitions won't be lost.

New option in the `Menu` for "Back"

In some Menus it makes sense to display a visual "Back" option. This option will behave the same way as ESC essentially does but it is more intuitive.

Breaking changes

Old profiles will not work anymore so they have to moved over to the new class format manually (all existing profiles in the repository have been moved over already)

Schema changes

The schema for the profile configuration has changed. Also the gfx driver configuration has been moved inside the profile configuration since it is never used as a standalone but only in combination with the respective profiles selected

        "profile": {
            "main":  {
                "description": "Main top level profile ",
                "type": "string",
                "enum": [
                    "desktop",
                    "minimal",
                    "server",
                    "xorg",
                ]
            },
            "details": {
                "description": "Specific profile to be installed based on the 'main' selection; these profiles are present in profiles_v2/, use the name of a profile to install it (case insensitive)",
                "type": "string",
                "enum": [
                    "awesome",
                    "bspwm",
                    "budgie",
                    "cinnamon",
                    "cutefish",
                    "deepin",
                    "desktop",
                    "enlightenment",
                    "gnome",
                    "i3-wm",
                    "i3-gasp",
                    "kde",
                    "lxqt",
                    "mate",
                    "sway",
                    "xfce4",
                    "qtile",
                    "cockpit",
                    "docker",
                    "httpd",
                    "lighttpd",
                    "mariadb",
                    "nginx",
                    "postgresql",
                    "sshd",
                    "tomcat"
                ]
            },
            "custom": {
                "description": "Specific profile definitions for custom setup profiles)",
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "packages": "string",
                        "services": "string",
                        "enabled": "boolean"
                    }
                }
            },
            "gfx_driver": {
                "description": "Graphics Drivers to install if a desktop profile is used, ignored otherwise.",
                "type": "string",
                "enum": [
                    "VMware / VirtualBox (open-source)",
                    "Intel (open-source)",
                    "AMD / ATI (open-source)",
                    "All open-source (default)",
                    "Nvidia (open kernel module for newer GPUs, Turing+)",
                    "Nvidia (open-source nouveau driver)",
                    "Nvidia (proprietary)"
                ]
            }
        },

Torxed · 2022-08-28T22:26:30Z

This would be huge! Just the multi-select on it's own would be awesome hehe.
I gotta test this thurrowly once I'm done releasing v2.5.1!

svartkanin · 2022-08-29T01:10:13Z

I'll let you know when it's ready ;)

@Torxed but as a question which I'm currently unsure about is how should we handle which greeter to install? Desktop profiles define lightdm and ssdm at the moment. Any suggestions here?

svartkanin · 2022-09-04T02:19:38Z

@Torxed I think this is ready for a review now
I have tested it to a degree but it would be better to get someone new to jump on it :)

Torxed · 2022-09-04T08:11:27Z

I'll give it some testing during the week!

svartkanin · 2022-10-01T09:12:52Z

@Torxed now that the new release is out, any chance to start merging the open PRs :) ?

Torxed · 2022-10-03T14:44:11Z

I'm full speed ahead! Feels like a classic but it's been a lot lately. But I'm in merge-mode and I'm planning on getting this in within an hour or so. Testing it now :)

Torxed · 2022-10-03T14:51:40Z

Any reason why we removed /examples/?
I get that they moved into scripts, but the main reason for the /examples/ folder was to served as coding examples not just script entry-points. Could we move them back or would that be a horrific amount of change?

svartkanin · 2022-10-03T20:20:48Z

@Torxed the main reason I moved them under the scripts namespace was that they are used with the --script argument and therefore I think it makes sense having them located there.

I saw that the symlinked examples folder had to be explicitly handled for various operations (setup, pkgbuild...), and IDEs were quite confused with it (I'm entirely sure I could invest some time in setting it up properly)

I understand that they can also be seen as standalone examples, since they are essentially standalone.
I'm happy to create the examples symlink again, but as an alternative I might also suggest to maybe just refer to the scripts as the examples in the Readme or create separate explicit examples since way the scripts are setup they are not covering some actual use cases :)

nyabinary · 2022-10-07T13:44:08Z

Tested it, seems to not like updating the arch keyring as it spits out an error.

svartkanin · 2022-10-07T21:50:08Z

@VexfulNiko interesting, would you mind sharing the steps and possibly error?

@Torxed any more thoughts on this PR otherwise or can we aim to get it into the next release 😀

nyabinary · 2022-10-09T18:20:41Z

@svartkanin Just downloaded the Arch ISO and ran the Arch install, and it created errors saying it couldn't update the arch keyring. I tried to ping to ensure I had internet, and I did.

svartkanin · 2022-10-09T19:52:39Z

This PR isn't even merged so I don't think it has anything to do with the problem you encountered

nyabinary · 2022-10-09T22:45:30Z

An testing Archiso was created with this pr request if you go into actions

svartkanin · 2022-10-10T01:18:29Z

Ahh I see, regardless I don't think it has anything to do with the changes in this PR.

Have you tried running pacman -Sy archlinux-keyring?

nyabinary · 2022-10-13T15:45:39Z

I did! Said it was corrupted lol.

nyabinary · 2022-10-26T18:07:47Z

Any update for merging this?

andrewverify · 2022-11-18T18:03:41Z

Ahh I see, regardless I don't think it has anything to do with the changes in this PR.

Have you tried running pacman -Sy archlinux-keyring?

The latest archiso was built with this bug. For now doing rm -rf /etc/pacman.d/gnupg then pacman-keyring init pacman-keyring --populate archlinux fixes it. Also lsblk gives an error.

I am sorry but you fucked up the installer. It's not a shame to make a mistake but it's a shame to deny it. Please fix it.

svartkanin · 2022-11-18T20:08:57Z

@atillabirer How would this PR have "fucked up the installer" if the PR isn't even merged yet?

svartkanin · 2023-03-13T08:22:21Z

Has been merged into #1604 and will be added with it

svartkanin added 24 commits June 20, 2022 20:26

Invert mypy files

9c735d9

Update

08975dc

Update

2be9904

Update

1f99d97

Update

03bd196

Update

98a0cea

Update

f060a05

Add optional pre-commit hooks

de4b20b

Update

958ff03

Update

ee5e955

New profile structure

8ba5c46

Update

20b0a41

Merge remote-tracking branch 'origin/master' into update-mypy

8d29a18

Update

f1106bd

Merge branch 'update-mypy' into rework-profiles

5742c1f

Update

a8489c8

Update

0f42d1b

Update

5362132

Update

9107266

Serialize profiles

c366331

Merge remote-tracking branch 'origin/master' into rework-profiles

71a80f4

Update

fc42648

Update

fa25e5b

Update

3280bcf

svartkanin requested a review from Torxed as a code owner August 27, 2022 09:06

svartkanin added 3 commits August 27, 2022 19:20

Use profile instead of classmethod

8a546fd

Custom profile setup

db00b92

Separator between back

627ab97

svartkanin added 2 commits September 2, 2022 22:50

Cleanup

7f31cc2

Update

bdf992a

svartkanin changed the title ~~[WIP] Rework profile management~~ Rework profile management Sep 2, 2022

Update greeter setting

c55e6da

svartkanin added 2 commits September 4, 2022 12:27

Update schema

689fa59

Revert toml changes

f37c7df

svartkanin mentioned this pull request Sep 21, 2022

Support external dependencies #1478

Closed

Merge remote-tracking branch 'origin/master' into rework-profiles

7234794

Merge remote-tracking branch 'origin/master' into rework-profiles

75b18c8

Merge remote-tracking branch 'origin/master' into rework-profiles

7a9e75d

colossalchicken mentioned this pull request Nov 18, 2022

Failed to read disk "/dev/sdaX" with lsblk #1557

Closed

svartkanin mentioned this pull request Dec 21, 2022

PyParted and a large rewrite of the underlying partitioning #1604

Merged

svartkanin closed this Mar 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rework profile management #1434

Rework profile management #1434

svartkanin commented Aug 27, 2022 •

edited

Loading

Torxed commented Aug 28, 2022

svartkanin commented Aug 29, 2022

svartkanin commented Sep 4, 2022

Torxed commented Sep 4, 2022

svartkanin commented Oct 1, 2022

Torxed commented Oct 3, 2022

Torxed commented Oct 3, 2022

svartkanin commented Oct 3, 2022

nyabinary commented Oct 7, 2022

svartkanin commented Oct 7, 2022

nyabinary commented Oct 9, 2022

svartkanin commented Oct 9, 2022

nyabinary commented Oct 9, 2022

svartkanin commented Oct 10, 2022

nyabinary commented Oct 13, 2022

nyabinary commented Oct 26, 2022

andrewverify commented Nov 18, 2022

svartkanin commented Nov 18, 2022

svartkanin commented Mar 13, 2023

Rework profile management #1434

Rework profile management #1434

Conversation

svartkanin commented Aug 27, 2022 • edited Loading

Reworking the current profile mangement

Related issues that should be fixed with this upgrade

New features

Profiles as classes

Programmatically create profiles

Multiple profiles per file

Ability to install multiple desktop profiles

Display the packages to be installed on the desktop profiles

Select which greeter should be installed

Create custom profiles in the menu

New option in the Menu for "Back"

Breaking changes

Schema changes

Torxed commented Aug 28, 2022

svartkanin commented Aug 29, 2022

svartkanin commented Sep 4, 2022

Torxed commented Sep 4, 2022

svartkanin commented Oct 1, 2022

Torxed commented Oct 3, 2022

Torxed commented Oct 3, 2022

svartkanin commented Oct 3, 2022

nyabinary commented Oct 7, 2022

svartkanin commented Oct 7, 2022

nyabinary commented Oct 9, 2022

svartkanin commented Oct 9, 2022

nyabinary commented Oct 9, 2022

svartkanin commented Oct 10, 2022

nyabinary commented Oct 13, 2022

nyabinary commented Oct 26, 2022

andrewverify commented Nov 18, 2022

svartkanin commented Nov 18, 2022

svartkanin commented Mar 13, 2023

svartkanin commented Aug 27, 2022 •

edited

Loading

New option in the `Menu` for "Back"