Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete support for dynamic container names #231

Merged
merged 6 commits into from
Jul 13, 2017

Conversation

helderco
Copy link

@helderco helderco commented Jul 8, 2017

This is my attempt to solve the same #205 issue.

In Swarm Mode and Docker Cloud, container names are dynamic. #181 solved this partially with a label on jwilder/nginx-proxy but not if they're separate containers. #126 tries a different approach, but I think using a label is a better one.

This PR simply adds a com.github.jrcs.letsencrypt_nginx_proxy_companion.docker_gen=true label to the docker-gen container.

It also moves the label related detection to runtime (functions.sh: nginx-reload) instead of startup time (entrypoint.sh), because the nginx and docker-gen containers could be redeployed during the lifecycle of the letsencrypt companion, and thus, getting different ids.

There's also a new assumption in that I think it's simpler to just recommend using the label instead of volumes-from to detect the nginx-proxy container. The reason is that in some environments (like mine), you can't use volumes-from, but I think you can always use labels.

Note: If you're wandering why I've chosen not to set the variables NGINX_DOCKER_GEN_CONTAINER and NGINX_PROXY_CONTAINER, that's because we can use them to make sure an explicitly set container name takes precedence over a label.

Now I'm running a successful separate containers deployment in Swarm Mode, here's my stack:

version: '3.2'
services:
  nginx:
    image: nginx:alpine
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./nginx/conf.d:/etc/nginx/conf.d
      - ./nginx/vhost.d:/etc/nginx/vhost.d
      - ./nginx/certs:/etc/nginx/certs
      - ./nginx/letsencrypt:/usr/share/nginx/html
    labels:
      - com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy
    deploy:
      placement:
        constraints:
          - node.role == manager

  dockergen:
    image: helder/docker-gen
    command: -notify "docker-label-sighup com.docker.swarm.service.name=frontend_nginx" -watch /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.conf
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - ./nginx.tmpl:/etc/docker-gen/templates/nginx.tmpl
      - ./nginx/conf.d:/etc/nginx/conf.d
      - ./nginx/vhost.d:/etc/nginx/vhost.d
      - ./nginx/certs:/etc/nginx/certs
    labels:
      - com.github.jrcs.letsencrypt_nginx_proxy_companion.docker_gen
    deploy:
      placement:
        constraints:
          - node.role == manager

  letsencrypt:
    image: helder/letsencrypt-nginx-proxy-companion
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./nginx/conf.d:/etc/nginx/conf.d
      - ./nginx/vhost.d:/etc/nginx/vhost.d
      - ./nginx/certs:/etc/nginx/certs
      - ./nginx/letsencrypt:/usr/share/nginx/html
    deploy:
      placement:
        constraints:
          - node.role == manager

As you can see, I'm also using a custom helder/docker-gen image which simply adds a docker-label-sighup script that allows me to reload the nginx container from a label, because... well, dynamic names. jwilder/docker-gen doesn't support reloading containers with dynamic names (yet).

@buchdag
Copy link
Member

buchdag commented Jul 8, 2017

Maybe the commits should be squashed to have a cleaner commit history and no doing-undoing commits ?

Otherwise nice PR, I hope it'll be merged. Even outside of swarm and docker cloud it makes more sense this way than having a label on one container and env variables on the other.

@helderco
Copy link
Author

helderco commented Jul 8, 2017

Agreed, but that's done when merging. Afterwards, my fork will be deleted. 😺

README.md Outdated
@@ -63,7 +63,7 @@ To run nginx proxy as a separate container you'll need:
curl https://raw.githubusercontent.com/jwilder/nginx-proxy/master/nginx.tmpl > /path/to/nginx.tmpl
```

2) Set the `NGINX_DOCKER_GEN_CONTAINER` environment variable to the name or id of the docker-gen container.
2) Use the `com.github.jrcs.letsencrypt_nginx_proxy_companion.docker_gen=true` label on the docker-gen container, or explicitly set the `NGINX_DOCKER_GEN_CONTAINER` environment variable to the name or id of that container.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no need for =true. Just a plain label is enough. It doesn't have to be a key-value pair.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually there is, because it defaults to an empty string and I kept the logic from #181 where it checks for the value "true": jq -r '.[] | select( .Labels["'$1'"] == "true")|.Id'

We would have to change this query, I'm not opposed to it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think it's better to disregard the value, since it doesn't matter: jq -r '.[] | select( .Labels["'$1'"])|.Id' 👍

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it's incredibly inefficient to fetch the list of all containers. The Docker Engine API allows filtering by labels.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll look into that!

Copy link

@teohhanhui teohhanhui Jul 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better for performance but also makes it easier to filter by simply a label or label=value.

I improved my docker-label-sighup, based on this.

It's cooler now, because it allows me to do this: -notify "docker-label-sighup com.docker.swarm.service.name=frontend_nginx"

So I can reuse the labels that already exist.

The mere existence of the label is enough.
@nbejansen
Copy link

Works great, thanks! When will this be merged?

@JrCs JrCs merged commit 27d433c into nginx-proxy:master Jul 13, 2017
@helderco helderco deleted the docker-gen branch July 13, 2017 16:17
@jasonchi38
Copy link

jasonchi38 commented Sep 19, 2017

The communication is not working if you stack deploy a container on another node within the swarm. Has this problem been resolved?
For example:
2 worker node, and 1 management node.

manage node is running nginx-letsencrypt server using deploy with contraints to this manage node.

  • If I deploy an application contraints to the management node, nginx-letsencrypt log show that it detects this application and function properly.
  • If I deploy an application contraints to workers node, nginx-letsencrypt doesn't detect the container has been started.

If I create two docker-gen service in the same deployment, one for manager and worker. this acknowledges container start detection in workder node, but the process returns error shown below on worker node:
"Error running notify command: docker-label-sighup com.docker.swarm.service.name=frontend_nginx, exit status 1". sounds like somehow "frontend_nginx" is not valid from the worker node.

so basically, running everything in the same node works fine. but if proxy and application container are in different node within Swarm it doesn work.

Here is the frontend yml

version: '3.2'
services:
  nginx: #manger node- only one instance
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"   
    #volumes are pointed to nfs, all node has access to these files.
    volumes:
      - ./nginx-conf/etc/conf.d:/etc/nginx/conf.d
      - ./nginx-conf/etc/vhost.d:/etc/nginx/vhost.d
      - ./nginx-conf/usr/share/nginx/html:/usr/share/nginx/html
      - ./nginx-conf/var/letsencrypt/certs:/etc/nginx/certs:ro
    labels:
      - com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy
    deploy:
      placement:
        constraints: [node.id == rxyh541dpy2wnucsvif1cvcn6]       
  dockergen: #manager node
    image: helder/docker-gen
    command: -notify "docker-label-sighup com.docker.swarm.service.name=frontend_nginx" -watch /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.conf  
    volumes:
       - ./nginx-conf/etc/conf.d:/etc/nginx/conf.d
       - ./nginx-conf/etc/vhost.d:/etc/nginx/vhost.d
       - ./nginx-conf/var/docker-gen/templates/nginx.tmpl:/etc/docker-gen/templates/nginx.tmpl
       - ./nginx-conf/var/letsencrypt/certs:/etc/nginx/certs
       - /var/run/docker.sock:/tmp/docker.sock:ro
    deploy:
      placement:
        constraints: [node.id == rxyh541dpy2wnucsvif1cvcn6]    
  dockergen-rm1: #worker node
    image: helder/docker-gen
    command: -notify "docker-label-sighup com.docker.swarm.service.name=frontend_nginx" -watch /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.conf
    volumes:
       - ./nginx-conf/etc/conf.d:/etc/nginx/conf.d
       - ./nginx-conf/etc/vhost.d:/etc/nginx/vhost.d
       - ./nginx-conf/var/docker-gen/templates/nginx.tmpl:/etc/docker-gen/templates/nginx.tmpl
       - ./nginx-conf/var/letsencrypt/certs:/etc/nginx/certs
       - /var/run/docker.sock:/tmp/docker.sock:ro   
    labels:
      - com.github.jrcs.letsencrypt_nginx_proxy_companion.docker_gen
    deploy:
      placement:
        constraints: [node.id == rbom24xvsp57b0dwe3064xekm]            
  letsencrypt: #manager node
    image: jrcs/letsencrypt-nginx-proxy-companion
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      #- /var/run/docker.sock:/var/run/docker.sock:ro
      - ./nginx-conf/etc/conf.d:/etc/nginx/conf.d
      - ./nginx-conf/etc/vhost.d:/etc/nginx/vhost.d
      - ./nginx-conf/var/letsencrypt/certs:/etc/nginx/certs
      - ./nginx-conf/usr/share/nginx/htmlt:/usr/share/nginx/html
    deploy:
      placement:
        constraints: [node.id == rxyh541dpy2wnucsvif1cvcn6]          
  letsencrypt-rm1: #worker node
    image: jrcs/letsencrypt-nginx-proxy-companion
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./nginx-conf/etc/conf.d:/etc/nginx/conf.d
      - ./nginx-conf/etc/vhost.d:/etc/nginx/vhost.d
      - ./nginx-conf/var/letsencrypt/certs:/etc/nginx/certs
      - ./nginx-conf/usr/share/nginx/htmlt:/usr/share/nginx/html
    deploy:
      placement:
        constraints: [node.id == rbom24xvsp57b0dwe3064xekm]           
    environment:
      NGINX_DOCKER_GEN_CONTAINER: "dockergen"
      NGINX_PROXY_CONTAINER: "nginx"               
networks:
  default:
    external:
      name: proxy

Here is the application yml.

version: '3'

volumes:
  nextcloud:
  db:

services:
     
  db:
    image: mariadb
    restart: always
    volumes:
      - db:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=secret_root_pass
      - MYSQL_PASSWORD=secret_user_pass
      - MYSQL_DATABASE=nextcloud 
      - MYSQL_USER=nextcloud
    deploy:
      placement:
        constraints: [node.id == rbom24xvsp57b0dwe3064xekm]     #worker node     
  app:
    image: nextcloud
    links:
      - db
    volumes:
      - nextcloud:/var/www/html
    environment:
      - VIRTUAL_HOST=www.doamin.com
      - LETSENCRYPT_HOST=www.domain.com
      - LETSENCRYPT_EMAIL=name@email.com
    restart: always
    deploy:
      placement:
        constraints: [node.id == rbom24xvsp57b0dwe3064xekm]    #worker node    
networks:
  default:
    external:
      name: proxy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants