Logging loop after SIGHUP when a StreamHandler is configured

[richvdh]

If you use the default logging config, which has a StreamHandler pointing at sys.stderr, and do not set --no-redirect-stdio, then when you send synapse a SIGHUP, it will stack overflow and abort.

The problem appears to be that when you SIGHUP it, the streamhandler gets repointed at the redirected stderr, which goes back into a logger: so as soon as anything is logged, we get an infinite loop.

[richvdh]

I thought this used to work, but I just tried the same thing on v0.20.0 (the version that introduced reloading the log config on SIGHUP) and it did the same.

[richvdh]

This seems like yet another reason that redirecting stdio is a terrible idea, and we should implement #1539 (comment).

[richvdh]

(the default logging config no longer has a StreamHandler pointing at sys.stderr, but plenty of peoples' logging configs do have that setup)

[richvdh]

as of #8268, you no longer get a stack overflow, but you do get logging that looks like this:

logging during logging: 2021-06-10T14:34:07+0100 [stderr#error] 2021-06-10 14:34:07,796 - twisted - 258 - ERROR - sentinel - 2021-06-10 14:34:07,796 - synapse.metrics - 596 - DEBUG - sentinel - Collecting gc 0

It's worth noting that this (logs-to-stderr plus redirect-stdio-to-logs) is the default configuration for the docker image.

[richvdh]

(a StreamHandler is no longer the default for the non-docker case as of #8040)