Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Errors during init get lost in the logs #4641

Open
richvdh opened this issue Feb 14, 2019 · 1 comment
Open

Errors during init get lost in the logs #4641

richvdh opened this issue Feb 14, 2019 · 1 comment
Labels
A-Logging Synapse's logs (structured or otherwise). Not metrics. O-Uncommon Most users are unlikely to come across this or unexpected workflow S-Minor Blocks non-critical functionality, workarounds exist. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. z-p2 (Deprecated Label)

Comments

@richvdh
Copy link
Member

richvdh commented Feb 14, 2019

When started with -D, Synapse daemonizes itself before it binds any TCP/SSL listeners, which means that any errors that occur during that process (such as permission errors or "Address already in use", but also any of a number of potential configuration errors) get lost in the logs rather than being visible in stderr, and the process manager (synctl or systemd) can't tell that there was an error.

@richvdh
Copy link
Member Author

richvdh commented Feb 14, 2019

I can see a few potential solutions to this:

  1. Use a temporary reactor for the ACME init, so that once it finishes, we can stop the reactor again, bind the sockets, daemonize, and then start the main reactor.

  2. Bind the TCP sockets before daemonizing and starting the reactor, but don't start listening on them until later, when we have the certs (I don't think twisted's Port implementation allows this currently though). Also try to do more validation of config before forking.

  3. Reimplement daemonization so that the original process does not exit until there is some signal that the forked process has started happily. This probably involves throwing away daemonize which is a thing I want to do anyway as per stderr goes to /dev/null when we run with --daemonize #1539 (comment).

  4. Decide that daemonization isn't our job and we should leave it up to systemd or some other process manager.

Personally I like option 3.

@neilisfragile neilisfragile added A-Logging Synapse's logs (structured or otherwise). Not metrics. z-p2 (Deprecated Label) labels Feb 21, 2019
@reivilibre reivilibre added S-Minor Blocks non-critical functionality, workarounds exist. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. O-Uncommon Most users are unlikely to come across this or unexpected workflow labels May 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A-Logging Synapse's logs (structured or otherwise). Not metrics. O-Uncommon Most users are unlikely to come across this or unexpected workflow S-Minor Blocks non-critical functionality, workarounds exist. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. z-p2 (Deprecated Label)
Projects
None yet
Development

No branches or pull requests

3 participants