If the TLS private key isn't readable, synapse just exits with code 0

[xuhdev]

Description

When the private key specified by tls_private_key_path is not readable, synapse simply exits with code 0.

Steps to reproduce

• In homeserver.yaml, have no_tls set to False and tls_private_key_path set to a private key (such as /etc/letsencrypt/live/my.matrix.server/privkey.pem)

• Ensure that user matrix-synapse does not have the permission to read the private key.

• Run

  sudo -u matrix-synapse /opt/venvs/matrix-synapse/bin/python -m synapse.app.homeserver --config-path=/etc/matrix-synapse/homeserver.yaml --config-path=/etc/matrix-synapse/conf.d/
  

The process simply exits silently without any error code.

Version information

Output of

$ curl http://localhost:8008/_synapse/admin/v1/server_version

is

{
    "python_version": "3.8.2",
    "server_version": "1.20.1"
}

• Install method:

Ubuntu 20.04, from the synapse official repo via apt.

[richvdh]

Can you share your logging configuration?

[xuhdev]

# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema

version: 1

formatters:
    precise:
        format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'

handlers:
    file:
        class: logging.handlers.TimedRotatingFileHandler
        formatter: precise
        filename: /var/log/matrix-synapse/homeserver.log
        when: midnight
        backupCount: 3  # Does not include the current log file.
        encoding: utf8

    # Default to buffering writes to log file for efficiency. This means that
    # will be a delay for INFO/DEBUG logs to get written, but WARNING/ERROR
    # logs will still be flushed immediately.
    buffer:
        class: logging.handlers.MemoryHandler
        target: file
        # The capacity is the number of log lines that are buffered before
        # being written to disk. Increasing this will lead to better
        # performance, at the expensive of it taking longer for log lines to
        # be written to disk.
        capacity: 10
        flushLevel: 30  # Flush for WARNING logs as well

    # A handler that writes logs to stderr. Unused by default, but can be used
    # instead of "buffer" and "file" in the logger handlers.
    console:
        class: logging.StreamHandler
        formatter: precise

loggers:
    synapse.storage.SQL:
        # beware: increasing this to DEBUG will make synapse log sensitive
        # information such as access tokens.
        level: INFO

    twisted:
        # We send the twisted logging directly to the file handler,
        # to work around https://github.com/matrix-org/synapse/issues/3471
        # when using "buffer" logger. Use "console" to log to stderr instead.
        handlers: [file]
        propagate: false

root:
    level: INFO

    # Write logs to the `buffer` handler, which will buffer them together in memory,
    # then write them to a file.
    #
    # Replace "buffer" with "console" to log to stderr instead. (Note that you'll
    # also need to update the configuation for the `twisted` logger above, in
    # this case.)
    #
    handlers: [buffer]

disable_existing_loggers: false

[clokep]

Looks like that's the standard logging configuration.

I tested this a little bit and see an error in the logs: PermissionError: [Errno 13] Permission denied: '<path to key>'. (Note that I tested using a key file, not a pem, not sure if that makes a difference though.)

It seems the exit code is 0 though. The end of the logs are:

Traceback (most recent call last):
  File "venv/synapse/app/_base.py", line 262, in start
    refresh_certificate(hs)
  File "venv/synapse/app/_base.py", line 205, in refresh_certificate
    hs.config.read_certificate_from_disk(require_cert_and_key=True)
  File "venv/synapse/config/tls.py", line 238, in read_certificate_from_disk
    self.tls_private_key = self.read_tls_private_key()
  File "venv/synapse/config/tls.py", line 517, in read_tls_private_key
    private_key_pem = self.read_file(private_key_path, "tls_private_key_path")
  File "venv/synapse/config/_base.py", line 193, in read_file
    with open(file_path) as file_stream:
PermissionError: [Errno 13] Permission denied: 'venv/demo/etc/localhost:8480.tls.key'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/clokep/.pyenv/versions/synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "venv/synapse/app/homeserver.py", line 443, in start
    _base.start(hs, config.listeners)
  File "venv/synapse/app/_base.py", line 290, in start
    sys.exit(1)
SystemExit: 1
2020-10-06 07:56:01,034 - twisted - 231 - INFO -  - Main loop terminated.
2020-10-06 07:56:01,011 - synapse.config.tls - 516 - INFO -  - Loading TLS key from venv/demo/etc/localhost:8480.tls.key

[xuhdev]

I was using a pem file. I'm pretty sure the exit code was zero. Perhaps the key difference is whether it's a pem file. (Are you also testing on 1.20.1?)

Do you have a development guide that generates a minimal isolated environment, so I can do more tests? Or, could you perhaps test on a pem file if it is easier?

[richvdh]

the incorrect exit code is covered by #4640.

I think this particular bug is a duplicate of #4641.

[clokep]

I was using a pem file. I'm pretty sure the exit code was zero. Perhaps the key difference is whether it's a pem file. (Are you also testing on 1.20.1?)

I believe I saw the same behavior as you.

I agree that the two issue above seems to cover this. 👍