mariadb: Increase timeout for MariaDB shutdown

[agners]

Currently, when MariaDB takes longer than 20s to shutdown, Docker will simply kill it. This can be problematic on upgrade, as the new MariaDB version cannot handle a not cleanly shutdown MariaDB (see #3566).

Increase the timeout of the container shutdown to 60s. Also define a per-service kill timeout for mariadb-core. This is largely optional, and essentially leads to the same outcome if MariaDB takes even longer (it will get killed, one way or another). But by defining this as part of the s6 service makes sure that the add-on log itself shows that MariaDB had to be killed, and makes sure the rest of the s6 service tree gets a chance to shutdown gracefully still:

[09:29:37] INFO: Service mariadb exited with code 256 (by signal 9) s6-rc: info: service mariadb-core successfully stopped s6-rc: info: service mariadb-pre: stopping
...

The 18s timeout of S6_SERVICES_GRACETIME seems to apply for legacy services only which are no longer used in this add-on. So drop this definition.

Fixes #3566

[lmagyar]

@lmagyar review welcome 😄

Maybe increase it even further? 300 and 295000???

[lmagyar]

Maybe add a note to the beginning of the changelog, to "restart the addon before upgrade if the current version is lower than 2.7.1", and delete it a few month version later.

[agners]

I was actually considering a bit higher value too 🤔 . I guess we should not make too much assumption of how long this takes. It also doesn't hurt much... If it takes long... well that is what it takes 😅

Increased to 5 minutes.

[agners]

Maybe add a note to the beginning of the changelog, to "restart the addon before upgrade if the current version is lower than 2.7.1", and delete it a few month version later.

I've added a comment. However, I only mention "lower than 2.7.0 only": We don't do a major bump, so even if 2.7.0 fails to terminate gracefully, the problem won't appear as 2.7.1 will be able to recover just like 2.7.0 will be.

[lmagyar]

Yes, you are right, "version is lower than 2.7.0" should be fine:

• even if 2.7.0 shutdown fails, 2.7.1 has the same MariaDB version, so no problem to recover, no need to prior restart
• 2.7.0 had only a few days to run, it has a good chance to shutdown correctly

[mdegat01]

I wonder if there's some way to get s6 to run a script when it does do kill. It would be nice if it could make a sentry issue when that happens so we can get a sense of whether we picked the right value or not

[agners]

I wonder if there's some way to get s6 to run a script when it does do kill. It would be nice if it could make a sentry issue when that happens so we can get a sense of whether we picked the right value or not

The finish script has the exit code, we can do things based on that (see PR description, that INFO line is printed by the finish script). The exit code of the container should be 137 for this case, as we use the regular "exit code when killed by signal" logic (128 + signal).