Docker container custom labels

[lpulley]

Hi all,

I'm currently using Traefik for my reverse proxy. Traefik grabs configuration data for containers from labels on running Docker containers, so I'm wondering if there is a way for me to assign labels to the Matrix containers from the main.yml. If not, is this something that could be easily added? I'm not familiar with how the containers are started in this playbook, but it shouldn't be hard to "inject" labels at runtime with docker run -l or something like that.

Doing things this way also requires the containers to be on the same Docker network; can I use matrix_docker_network to change the Docker network of the Matrix containers to match the network of my Traefik container?

Thanks!

[spantaleev]

Hi!

There's a dedicated variable for each service which lets you inject additional docker run arguments.
For the Synapse service, it's called matrix_synapse_container_extra_arguments. You can use it like this:

matrix_synapse_container_extra_arguments: ['--label key=value', '--label another=value']

You can use matrix_docker_network to change the Docker network to something else (defaults to 'matrix' for most services).

[lpulley]

Oh, sweet! Somehow I missed that. Thanks so much for your detailed reply! I really appreciate this project, thanks for helping so much!

[spantaleev]

You're welcome! Happy to hear that!

[lpulley]

@spantaleev Finally getting around to this. I can't get the Nginx proxy container to not expose 80 and 443, so it won't start. (I need it to expose nothing, or unused ports, because I am using 80 and 443 for Traefik.)

Any idea on how I can use the built in Nginx container without any port mappings?

[spantaleev]

@lpulley, as of right now, matrix-nginx-proxy.service always exposes ports 80 and 443 with no way to disable that. We can make it configurable though.

Until we do, I suggest that you modify /etc/systemd/system/matrix-nginx-proxy.service manually, to remove the port-mapping lines. Then do systemctl daemon-reload and restart it (systemctl restart matrix-nginx-proxy).

See if removing these port mappings and doing all your other tweaks leads you to a nice and working setup. If it does, we should definitely investigate how to remove the port mappings by tweaking playbook configuration.

[lpulley]

Thanks!!
I think I'm getting close. I can successfully prevent Nginx from trying to bind to 80 and 443. Now the error stopping the Nginx container from starting is the fact that it's looking for SSL certificates for Dimension despite me having disabled SSL certificates:

Dec 04 11:13:18 raven matrix-nginx-proxy[1787]: nginx: [emerg] cannot load certificate "/matrix/ssl/config/live/dimension.example.com/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/matrix/ssl/config/live/dimension.example.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

The relevant lines of my vars.yml:

matrix_dimension_enabled: true
matrix_ssl_retrieval_method: none

Do I need some line in my vars.yml to prevent Nginx from trying to SSL Dimension?

[spantaleev]

Our nginx reverse proxy requires that certificates are available one way or another.

An https-enabled vhost is always part of the nginx configuration and it attempts to look for certificates.
It's not just for Dimension, but for all other services as well.

You may wish to use matrix_ssl_retrieval_method: self-signed, which will generate certificates for all services and nginx would be able to start.

How do you think of plugging traefik into this whole thing? Making it talk to nginx? Or to each service container?

[lpulley]

Looks like self-signed helped. It starts just fine now. However it doesn't look like it accepts non-SSL connections on the container port 8080, which I suppose makes sense when running standalone. With the reverse proxy, however, the proxy should terminate the SSL connection and pass through without SSL to the service.

My plan with Traefik has been to attach labels to the Nginx proxy container and set all containers to be part of the Traefik network, using the following configuration:

matrix_ssl_retrieval_method: self-signed
matrix_docker_network: 'traefik'
matrix_nginx_proxy_container_extra_arguments:
  - '--label "traefik.enable=true"'
  - '--label "traefik.http.routers.matrix-nginx-proxy.rule=Host(`matrix.example.com`)||Host(`riot.example.com`)||Host(`dimension.example.com`)"'
  - '--label "traefik.http.routers.matrix-nginx-proxy.entrypoints=web-secure"'
  - '--label "traefik.http.routers.matrix-nginx-proxy.tls.certResolver=default"'
  - '--label "traefik.http.services.matrix-nginx-proxy.loadbalancer.server.port=8080"'

Ideally, the container will generate self-signed certificates to make itself happy, and never use them. Traefik will generate the certificates for the three subdomains and direct all matrix.example.com, riot.example.com, and dimension.example.com traffic to the Nginx proxy container port 8080 without SSL, and the proxy container will then direct to the various containers as it sees fit.

Essentially what I'm looking for is a way to politely ask Nginx to not do strict SSL, and to let me handle it in the reverse proxy, which is typical as far as I am aware.

[spantaleev]

Problem is, nginx only serves meaningful stuff on the HTTPS-enabled vhost.
The plain-HTTP vhost only ever does a redirect to the HTTPS-enabled one.

Can you make traefik talk to the HTTPS-enabled vhost and not choke on its self-signed certificates?

Although.. I think I've had some trouble with self-signed certificates and nginx.. It would get confused for some reason and drive all traffic to the same vhost, regardless of the host I'm trying to talk to.

[lpulley]

I can try routing it like that, but that would be using Traefik's TCP routing, not HTTP, which isn't ideal but might work for me for now.

I think it would be worth adding a config item to use Nginx with only plain HTTP, for other people running behind a reverse proxy besides Nginx.

[spantaleev]

@lpulley, good news!

I have added support for nginx to work in plain-HTTP-only mode. See the documentation here: https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/docs/configuring-playbook-own-webserver.md#method-2-fronting-the-integrated-nginx-reverse-proxy-webserver-with-another-reverse-proxy

Let me know if anything else is needed to get Traefik working! Also, documenting your Traefik setup somewhere once you get it working would be nice!

[lpulley]

You're a saint!!

I have everything working (with the exception of Coturn, but if there's a way to fix that I'd be willing to help). The configuration I'm using differs a bit from the example you put in the docs, but I believe it's the ideal way for Traefik to interact with Synapse and Nginx:

inventory/host_vars/matrix.example.com/vars.yml

...

matrix_ssl_retrieval_method: none                          # Don't generate or retrieve any SSL certs
matrix_nginx_proxy_https_enabled: false                    # Don't configure Nginx to use SSL
matrix_nginx_proxy_container_http_host_bind_port: ''       # Don't bind any HTTP port to the OS
matrix_nginx_proxy_container_federation_host_bind_port: '' # Don't bind any federation port to the OS
matrix_coturn_enabled: false                               # Disable Coturn because it needs SSL certs
matrix_docker_network: 'traefik'                           # All containers need to be on the same Docker network as Traefik

matrix_nginx_proxy_container_extra_arguments:
  - '--label "traefik.enable=true"'                        # May enable automatically depending on Traefik config
  - '--label "traefik.http.routers.matrix-nginx-proxy.rule=Host(`matrix.example.com`,`riot.example.com`,`dimension.example.com`)"'
  - '--label "traefik.http.routers.matrix-nginx-proxy.entrypoints=web-secure"' # Listens on 443 with SSL
  - '--label "traefik.http.routers.matrix-nginx-proxy.tls.certResolver=default"'
  - '--label "traefik.http.services.matrix-nginx-proxy.loadbalancer.server.port=8080"'
                                                           # Traefik maps to internal port of Nginx container
matrix_synapse_container_extra_arguments:
  - '--label "traefik.enable=true"'                        # May enable automatically depending on Traefik config
  - '--label "traefik.http.routers.matrix-synapse.rule=Host(`matrix.example.com`)"'
  - '--label "traefik.http.routers.matrix-synapse.entrypoints=synapse"' # Listens on 8448 with SSL
  - '--label "traefik.http.routers.matrix-synapse.tls.certResolver=default"'
  - '--label "traefik.http.services.matrix-synapse.loadbalancer.server.port=8048"'
                                                           # Traefik maps to internal port of Synapse container

The main difference from your example is that I decided to eliminate the HTTP and federation port binds entirely and have Traefik route directly to the Nginx (on 443) and Synapse (on 8448) containers, as Traefik typically does things with the Docker provider. This does mean that port 8448 on the Nginx container goes unused but that shouldn't be a problem, since it just redirects to the Synapse container on 8048 and I have Traefik doing that instead.

This now has Matrix working for me with no modifications between setup-all and start. self-check also finishes successfully. Riot and other clients work well.

This is fantastic because it means I can treat this playbook as I do my other Docker services: all of the app configuration is in the same file as the reverse proxy options, and when the containers are down Traefik recognizes it and removes the routing rules.

I'll go ahead and edit the doc you linked above to match this, unless you think it should have both. I'm by no means a Traefik expert, but from everything I've read it seems like the method above is the "right" way.

[spantaleev]

Great to hear that! Thank you for contributing your documentation improvements in #325.