Migrate from php8.0 to php8.1

[kiekerjan]

This pull request proposes the upgrade from php8.0 to php8.1. This would remove the dependency on an external ppa.
The logic works by introducing on a number of places a variable containing the php version. This makes the code independent of the used php version.
The upgrade logic is embedded in the nextcloud installation logic. If a version of nextcloud that does not support php8.1 is to be installed, the code will also install php8.0. Once a new enough version of nextcloud is detected, php8.0 is removed.

Installation has been tested via vagrant. Todo: test upgrade from v60 to this, and test upgrade from v57 to this.

[jvolkenant]

I'm not understanding why we would stop using

mailinabox/setup/functions.sh

Line 7 in e419b62

PHP_VER=8.0

I understood why PHP still had 'php8.0' as it didn't have access to that variable, but with get_php_version() we shouldn't need to go back to having 8.0 or 8.1 everywhere

[kiekerjan]

Sorry, I'm not sure what you mean.
This pull request will remove the hardcoded uses of php8.0 or 8.1, and replace them by the mentioned get_php_version(). We can no longer depend on the hardcoded PHP_VER variable as the generic code needs to be able to handle either php8.0 or php8.1. The only place where the version is mentioned is on the places where it is installed by apt-get, and to set the default used php.
Or do you mean something else?

Anyway, also note that @binarykitchen proposed to split this pull request up into a version handling bit (see #2308, derived from this work) and a php upgrade bit.

[binarykitchen]

Right @jvolkenant

MiaB must depend on a clearly defined PHP version, not on whatever is installed by random on the server.

That's why my low-risk PR doesn't do that, nor is upgrading anything. All my PR does it to keep its version consistent across all the code base without having it hardcoded too much.

Secondly, there is no need to upgrade PHP to v8.1 right now.

Although, I'd like to give credit to @kiekerjan for some inspiration in my other PR.

[matidau]

TLDR; Nextcloud's strict requirements on maximum PHP version has made our Nextcloud upgrades tricky. I don't have an answer, just stating some possibilities.

I've had a few thoughts around our potential PHP8.1 and Nextcloud conundrum. This might be a bit long winded so please bear with me. I'm also likely to repeat a lot of what was raised in the other issues and PRs related to this.

Nextcloud v27 minimum non-deprecated version of PHP is 8.1. and presumably in a year it will be 8.2 for whatever version is released. This is Nextcloud keeping in step with PHP release cycles (aggressively so?).

As we upgrade each previous Nextcloud version through all the current ones, it might be reasoned that users will need to have PHP8.0 installed to upgrade up to Version24 (only applicable if coming from a MIAB version prior to v60 and the current version is on PHP8.1)

Options
These are the different options that we could consider as I see them.

I understood @kiekerjan 's function was to help resolve if we have multiple versions of PHP needed during the installation process so that Nextcloud could be upgraded no matter what version of PHP or Nextcloud database there is or is needed. Possibly the idea being able to determine what version of PHP was installed so the correct commands and sequence get called.

Another similar option, having looked at the Nextcloud repo it seems that the version check for the command line is only for versions below 8.0 for v27 and the current master. Perhaps we could use PHP8.0 to do all the upgrades and then finally upgrade to PHP 8.1 (or PHP8.x in the future). This is in a way similar to the function approach but we would hardcode installing, utilising and removing PHP8.0 just for the steps that are needed.

Alternatively we could enforce that users have to upgrade to a specific version of MIAB if coming from previous nextcloud ones. I have more to say on this one. Either at the start of the setup scripts or during Nextcloud's one.

Or last find an alternative solution to Nextcloud so we aren't tied to their strict PHP versions. I don't think anyone has time to work on an alternative and the scripted migration process needed. If they had it would have probably been done by now :)

PHP Versions
Just a couple of notes
Both web.sh and webmail.sh, which run before nextcloud.sh, already install the current version of PHP as listed in $PHP_VER, so 8.1 would be installed with the current arrangement of setup files.
It might be possible to have 2 versions of PHP (php-cli) installed during the nextcloud setup for upgrade purposes, then remove php8.0 after the upgrade to v24 or v27 where it is no longer needed. This would need to be tested.
php8.0 packages might need to be removed so that only 8.1 is left.

different classes of users

1. those that stay up to date
2. those that fall a few versions behind
3. those that haven't updated since v60
4. those that are still on versions lower than v60

Hopefully most are in 1 and no one is in 4 but we still need to (should?) allow for these upgrade paths.
Those in 1 aren't a problem as the upgrade will generally work no matter what solution we go for.
Those in 2 won't be for PHP8.1, but when PHP8.2 is required they might be caught out, depending on how far behind they are.
3 won't be a problem for PHP8.1 but will be if they are still on this version when PHP8.2 is required.
4 will be a problem as they can't get to V24 without php8.0 installed.

Bit more on each option

Use PHP8.0 and 8.1 during nextcloud upgrade
I think this might be the trickiest to get right but would likely offer the most pleasant upgrade experience for those on older versions of MIAB. we could be on v69 with some later Nextcloud and users in group 3 and 4 moving from v57a or upgrading from v60 would be clueless to all the effort going on to enable this. Using a function to determine the version, already having 8.1 installed from earlier scripts, having PHP8.0 installed alongside PHP8.1, or removing PHP8.1 installing PHP8.0 and then back again all seem risky to me. For users that are staying up to date there is no upside for them on this one just risk. These are my opinions, not fact, but others may share them.

Force users to update to a certain version
This is a bit different from how MIAB would normally do it. moving from 18.04 to 22.04 they will need to specify the MIAB version using the TAG environment variable to a PHP8.0 version, say TAG=v64, and do the migration with that. We would also need to include a warning message in the Nextcloud setup for people moving as they might have already installed the latest version post 8.0 before trying to restore their 18.04 backup.

Updates to the upgrade doco on the website and probably a pinned post on the forum would probably also need to be done for the late comers as that is likely the first places they will look.

When we move to 8.2 we may need to include something in the bootstrap to check the version and update it to the latest supported range of pre8.2.

This obviously has a fair few drawbacks, and will confuse some people when it comes to updating.

Summary
I don't see a super easy path forward when we hit v27 and PHP8.1 or future versions.

Hopefully be corrected on the above and an easy solution is put forward.

Also sorry for the long post and thank you for reading.

[kiekerjan]

Thank you for that elaborate post. I don't have a detailed reply on all your arguments right now, but I wanted to add some insight into this pull request.

Use PHP8.0 and 8.1 during nextcloud upgrade

I think this is what this pu request proposes. By default, php8.1 will be installed from the ubuntu repositories. This works for all software installed.
Only on upgrades of nextcloud will php8.0 be temporarily installed, and only for nexrcloud versions that need it.

I like the statement on different classes of users

those that stay up to date

Should have no issues, on update php8.1 is installed, and php8.0 removed

those that fall a few versions behind

Similare to the previous class. On an update, php8.1 is installed. Php8.0 is already installed, might be used to accomodate nextcloud upgrades, then is removed

those that haven't updated since v60

Same as previous class

those that are still on versions lower than v60

On upgrade, php8.1 is installed. During upgrade of nextcloud, php8.0 is temporarily installed to accomodate upgrades.

[downtownallday]

One more option that I had proposed during the discussion on upgrading to Jammy is to use docker and the Nextcloud images on docker hub to upgrade the MIAB Nextcloud database. This eliminates all OS/Nextcloud/PHP version issues because the docker image contains everything.

The purpose would be to get you to a Nextcloud version that works with the desired PHP installed on the system.

The steps are:

o install docker, if not already installed
o pull the nextcloud image for the next major version (eg. docker pull nextcloud:21 if you're at 20)
o start the image with STORAGE_ROOT/owncloud mounted in the image at the same location (docker run -d --network=none -p 127.0.0.1:8080:80 -v $STORAGE_ROOT/owncloud:$STORAGE_ROOT/owncloud nextcloud:21)
o link the image's config.php to ours (docker exec $CONTAINER_ID /bin/ln -sf $STORAGE_ROOT/owncloud/config.php /var/www/html/config/config.php)
o copy calendar, contacts and user_external to the image (docker cp /usr/local/lib/owncloud/apps/calendar $CONTAINER_ID:/var/www/html/apps/, etc)
o perform the upgrade (docker exec --user www-data $CONTAINER_ID php occ upgrade)
o destroy the image

At this point owncloud.db and config.php are upgraded to (in this example) Nextcloud 21. Repeat the steps until the database and config.php are upgraded to the desired Nextcloud version.

Finally, copy the Nextcloud installation files onto the production system (download and untar), and you're good to go.

The whole upgrade process doesn't even have to happen in situ. Copy the database and config to a local machine, run through the docker sequence there, then copy them back.

[downtownallday]

Regarding this PR...

Why not use PHP_VER for setup\*.sh and /usr/bin/php --version for management?

That would avoid the hard coded php version numbers in setup.

It might be clearer that way.

[JoshData]

and /usr/bin/php --version for management?

I would prefer to see an explicit version number everywhere in case there are multiple versions of PHP installed.

[matidau]

And just to add another option to the mix:
main...matidau:mailinabox:test-php81

Modifying nextcloud's versioncheck.php so as to remove the restriction on PHP8.1 for older nextcloud versions used for the upgrade.

Tested this with a test v57a upgrade to v64.

Pros:
Keeps the code changes to a minimum.
Easy to add an additional line for when we come to needing PHP 8.2.

Cons:
Modifying the Nextcloud package directly
While there isn't (shouldn't be?) any breaking changes between the PHP 8 versions, this might not be able to be widely tested enough for our risk appetite.

I haven't done a PR for this as the branch isn't complete by itself, More wanted to demonstrate this as an idea. Happy for someone else to pick it up if it provides inspiration.

[binarykitchen]

@matidau Thanks for giving fresh ideas and listing the Pros + Cons.

Regarding "risk appetite", MiaB has always been and must be well-tested for any version bumps of any dependencies. MiaB is about reliable services without excuses. If one email doesn't arrive, goodbye.

Our release frequency is low and everyone is busy, so cannot afford regressions. We need more unit tests, automated tests and the missing gaps must be tested by humans.

Just adding my two cents. Low risks, please.

[kiekerjan]

I would prefer to see an explicit version number everywhere in case there are multiple versions of PHP installed.

@JoshData I see that you have a different opinion on how multiple versions of php should be handled. I'm willing to put in some more time to realize this, but I would like to ask you for your input beforehand. Are there conditions you want to see before you would consider merging? What kind of things would you reject?
I will not ask you for an exhaustive list, just want to know your thoughts on this matter.

[binarykitchen]

Hello all - what's happening with this PR? Any problems? We need PHP v8.1 soon ...

[binarykitchen]

Ping?

[binarykitchen]

@kiekerjan I see you pushed and rebased. Ready for review and merge?