WIP: bind mount for acme.sh is missing #236

rwese · 2020-12-09T17:20:57Z

This is a suggestion, and WILL BREAK EXISTING INSTALLATIONS since it mounts a new directory into the /etc/acme.sh please do not blindly apply this

Having no fixed mount for acme.sh can cause a loss of your private key and all your signing keys for your domains, which hapend to me yesterday.

I noticed this when I got greeted with lots of:

too many certificates already issued for exact set of domains

A restart of the containers confirmed this suspicion:

nginx-letsencrypt    | [Wed Dec  9 16:48:07 UTC 2020] Create account key ok.
nginx-letsencrypt    | [Wed Dec  9 16:48:07 UTC 2020] Registering account: https://acme-v02.api.letsencrypt.org/directory
nginx-letsencrypt    | [Wed Dec  9 16:48:08 UTC 2020] Registered

Now having this bind-mount for the /etc/acme.d will put them where they should be.

Reference: Readme: nginx-proxy/docker-letsencrypt-nginx-proxy-companion

Having no fixed mount for acme.sh can cause a loss of your private key and all your signing keys for your domains, which hapend to me yesterday. I noticed this when I got greeted with lots of: `too many certificates already issued for exact set of domains` A restart of the containers confirmed this suspicion: ``` nginx-letsencrypt | [Wed Dec 9 16:48:07 UTC 2020] Create account key ok. nginx-letsencrypt | [Wed Dec 9 16:48:07 UTC 2020] Registering account: https://acme-v02.api.letsencrypt.org/directory nginx-letsencrypt | [Wed Dec 9 16:48:08 UTC 2020] Registered ``` Now having this bind-mount for the `/etc/acme.d` will put them where they should be. Reference: [Readme: nginx-proxy/docker-letsencrypt-nginx-proxy-companion](https://github.com/nginx-proxy/docker-letsencrypt-nginx-proxy-companion#step-2---letsencrypt-nginx-proxy-companion)

evertramos · 2020-12-10T21:59:56Z

Wow @rwese, thank you very much for you PR, that's was definitely missing in our repo.

I will check the breaking changes as you suggested, but it seems I can have a work around script for that...

rwese · 2020-12-11T08:27:42Z

@evertramos
I was bondering a fix, and fixing this without breaking would be adding an init-script which copies existing keys to the new mounted location and symlink the mounted-dir to /etc/acme.sh. This would be hands-off for the users.

Or modify the nginx-letsencrypt container init to use a different /etc/acme.sh location and migrate existing scripts with a init script.

Anyway it is not easy to make this fix non-breaking.

evertramos · 2020-12-17T12:21:14Z

@rwese

This changes is due to an update on the companion image...

https://github.com/nginx-proxy/docker-letsencrypt-nginx-proxy-companion#step-2---letsencrypt-nginx-proxy-companion

Which is not using simp.le anymore...

We were expecting an update on simp.le but never happened.

It will be necessary some extra updates in this repo in order to fix it... I am working in a side project that will have a bunch of automation including this repo as well, but it will take me a coupld weeks to finish it. But for now it still works for the newer users.

I will update this to the new companion way of doing things... as acme.sh is a good way of generating the certificates.

Catch you up later on this.

MarioProjects · 2021-03-02T10:39:52Z

Same problem here after few functional tests

Generating new certificate private key
ACME server returned an error: urn:ietf:params:acme:error:rateLimited :: There were too many requests of a given type :: Error creating new order :: too many certificates already issued for exact set of domains: maparla.es: see https://letsencrypt.org/docs/rate-limits/

Finally you solve that? Thanks in advance

evertramos · 2021-03-02T14:05:22Z

@MarioProjects can you checkout v2 branch and test the docker-compose file in your environment as well? Also if you do so, could you post here which environment you have, linux version, distro, docker version, docker-compose version etc. thanks!

MarioProjects · 2021-03-02T14:28:49Z

@evertramos this branch solves this certificate thing problem? I tried, I accessed to the branch and downloaded using git clone -b v2 https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion.git proxy but when calling `bin/start.sh

ERROR: The Compose file './../docker-compose.yml' is invalid because:
services.nginx-proxy-automation-web.ports is invalid: Invalid port "::/0:443:443", should be [[remote_ip:]remote_port[-remote_port]:]port[/protocol]
services.nginx-proxy-automation-web.ports is invalid: Invalid port "::/0:80:80", should be [[remote_ip:]remote_port[-remote_port]:]port[/protocol]

I did a fresh installation. Should I had the files and containers running from this branch first and then call that start.sh? In the README references fresh_start but it does not exist.

EDIT: After install first master and without removing anything calling start on v2, it prompts same error

EDIT2: I think I misunderstood you. I installed master, changed only docker-compose and same error :(

MarioProjects · 2021-03-03T08:55:27Z

@evertramos For example, If I run a container with flask and I want to update the code... this will call lets encrypt again and again, and will cause the same errors? How should we deploy the container and reuse the cert?

evertramos · 2021-03-03T14:26:59Z

you shouldn't use the start script... just docker-compose

$ docker-compose up -d

and the rest about the same to test the certificate, and yes it solves it. I am preparing a whole new script and automation for that which will help other I think.

about the environment you need to start from scratch remove all old files please... and try this new set up.

MarioProjects · 2021-03-03T14:45:41Z

docker-compose up -d

ERROR: The Compose file './docker-compose.yml' is invalid because:
services.nginx-proxy-automation-web.ports is invalid: Invalid port "::/0:443:443", should be [[remote_ip:]remote_port[-remote_port]:]port[/protocol]

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
docker-compose version 1.25.0, build unknown

evertramos · 2021-03-03T14:55:06Z

comment the ipv6 and check if you have anything running in port 80 or 443 stop everything and try again please

MarioProjects · 2021-03-03T15:17:54Z

What I did:

git clone -b v2 https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion.git proxy
cd proxy
docker network create proxy  # As your docker-compose requires 'proxy' network
# comment ipv6 lines in docker-compose.yml
docker-compose up -d
# 3 containers running:  nginx-proxy-automation-web / nginx-proxy-automation-letsencrypt / nginx-proxy-automation-gen

Now create a simple site with Dockerfile

FROM nginx:alpine
COPY . /usr/share/nginx/html

Then, create container and attach with your config:

docker build -t personal-site .

docker run -d -e VIRTUAL_HOST=maparla.es \
              -e LETSENCRYPT_HOST=maparla.es \
              -e LETSENCRYPT_EMAIL=maparla@prhlt.upv.es \
              --network=proxy \
              --name nginx-personal-site \
              personal-site

LetsEncrypt container logs:

[Wed Mar  3 15:10:29 UTC 2021] Create account key ok.
[Wed Mar  3 15:10:29 UTC 2021] Registering account: https://acme-v02.api.letsencrypt.org/directory
[Wed Mar  3 15:10:30 UTC 2021] Registered
[Wed Mar  3 15:10:30 UTC 2021] ACCOUNT_THUMBPRINT='YWBvw20BMukszEukj3GDEIq3zsSVple0M4CHcnRGB4I'
Reloading nginx docker-gen (using separate container nginx-proxy-automation-gen)...
Reloading nginx (using separate container 490d233c8554b03c902578b8f30fe6debaae666b6fbc0a30c4ac32f489cd8cb3)...
Creating/renewal maparla.es certificates... (maparla.es)
[Wed Mar  3 15:10:31 UTC 2021] Using CA: https://acme-v02.api.letsencrypt.org/directory
[Wed Mar  3 15:10:31 UTC 2021] Creating domain key
[Wed Mar  3 15:10:32 UTC 2021] The domain key is here: /etc/acme.sh/maparla@prhlt.upv.es/maparla.es/maparla.es.key
[Wed Mar  3 15:10:33 UTC 2021] Single domain='maparla.es'
[Wed Mar  3 15:10:33 UTC 2021] Getting domain auth token for each domain
[Wed Mar  3 15:10:34 UTC 2021] Create new order error. Le_OrderFinalize not found. {
  "type": "urn:ietf:params:acme:error:rateLimited",
  "detail": "Error creating new order :: too many certificates already issued for exact set of domains: maparla.es: see https://letsencrypt.org/docs/rate-limits/",
  "status": 429
}
[Wed Mar  3 15:10:34 UTC 2021] Please check log file for more details: /dev/null
Sleep for 3600s

After that I tried with one subdomain from maparla.es -> info.maparla.es.

docker run -d -e VIRTUAL_HOST=info.maparla.es \
              -e LETSENCRYPT_HOST=info.maparla.es \
              -e LETSENCRYPT_EMAIL=maparla@prhlt.upv.es \
              --network=proxy \
              --name nginx-personal-site \
              personal-site

Here the logs:

2021/03/03 15:12:40 Debounce minTimer fired
2021/03/03 15:12:40 Generated '/app/letsencrypt_service_data' from 5 containers
2021/03/03 15:12:40 Running '/app/signal_le_service'
Creating/renewal info.maparla.es certificates... (info.maparla.es)
[Wed Mar  3 15:12:41 UTC 2021] Using CA: https://acme-v02.api.letsencrypt.org/directory
[Wed Mar  3 15:12:41 UTC 2021] Creating domain key
[Wed Mar  3 15:12:42 UTC 2021] The domain key is here: /etc/acme.sh/maparla@prhlt.upv.es/info.maparla.es/info.maparla.es.key
[Wed Mar  3 15:12:42 UTC 2021] Single domain='info.maparla.es'
[Wed Mar  3 15:12:42 UTC 2021] Getting domain auth token for each domain
[Wed Mar  3 15:12:44 UTC 2021] Getting webroot for domain='info.maparla.es'
[Wed Mar  3 15:12:44 UTC 2021] Verifying: info.maparla.es
[Wed Mar  3 15:12:48 UTC 2021] Success
[Wed Mar  3 15:12:48 UTC 2021] Verify finished, start to sign.
[Wed Mar  3 15:12:48 UTC 2021] Lets finalize the order.
[Wed Mar  3 15:12:48 UTC 2021] Le_OrderFinalize='https://acme-v02.api.letsencrypt.org/acme/finalize/114524160/8232878452'
[Wed Mar  3 15:12:49 UTC 2021] Downloading cert.
[Wed Mar  3 15:12:49 UTC 2021] Le_LinkCert='https://acme-v02.api.letsencrypt.org/acme/cert/04be838c10624a41f2cb447cae28047a8259'
[Wed Mar  3 15:12:50 UTC 2021] Cert success.
-----BEGIN CERTIFICATE-----
MIIGJDCCBQygAwIBAgISBL6DjBBiSkHyy0R8rigEeoJZMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTAzMDMxNDEyNDhaFw0yMTA2MDExNDEyNDhaMBoxGDAWBgNVBAMT
D2luZm8ubWFwYXJsYS5lczCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIB
ANGTULroG/iVqkJsb6AuNVfG9K9vpsXhBvq8xXzmBmHPRBhfGpKAvDa+VMWsX+Kk
P5wVrrZW0xSIEUNsgy1oIJnBRvdHR+ZS1XceHCPYScz7rm4tVduMo6nCoY5Maqyd
f9QdQc0Eu1TdvvF78FSETeE69L8ZElRzh0EZBAihu3aj91VUS3c5R6YJ1djxrlAC
XOjLXuYPsbwgAmtwBbrZQ1ob8jACDOE/5ILXSmDKf7R71qiN7+ZJlgQ27fTQaIXu
pfPzMVyyqW9E+T1u0bHgjXVlVuV0AfmY/IsoiUaDd+MR3vch3ZqUbbckS3DwtN3Y
K/pQCctCRLEsSCpFE6ARQg0aMujU+WYvW69K8RHKoMCstYR6AZUCfUIKDabbcmJc
Pu9+pK5UnGx8hwaVqDhNY9JvoSflJ4OZ8GkQQxl4oA9jCsLTXHw58pU6JKCgb6er
0IMjse5xLd7qMKZPpjcBvJq7sR19JrqRLq/r2QDg8gOpIwPrKMJ+wUkNmYaK3Xp6
HLwLiizY+iM3j/RF6MRwyxJfgupyqYFqgFCziIFh9v86Dvj52UW+oqqQBfVjC2P9
YP1c1TBQFEmw1mYgGTLKPwDXE8jEH4hJ7IuqPSDoXEN6V9lluVMduJxOzG++PD56
+yDhGjlcCCiQfaA66Jf9ZVPpLiMfM38Avzc00PRZ4bwvAgMBAAGjggJKMIICRjAO
BgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwG
A1UdEwEB/wQCMAAwHQYDVR0OBBYEFCbBZL9YGDC4NGOMvC2LpzTUPSGOMB8GA1Ud
IwQYMBaAFBQusxe3WFbLrlAJQOYfr52LFMLGMFUGCCsGAQUFBwEBBEkwRzAhBggr
BgEFBQcwAYYVaHR0cDovL3IzLm8ubGVuY3Iub3JnMCIGCCsGAQUFBzAChhZodHRw
Oi8vcjMuaS5sZW5jci5vcmcvMBoGA1UdEQQTMBGCD2luZm8ubWFwYXJsYS5lczBM
BgNVHSAERTBDMAgGBmeBDAECATA3BgsrBgEEAYLfEwEBATAoMCYGCCsGAQUFBwIB
FhpodHRwOi8vY3BzLmxldHNlbmNyeXB0Lm9yZzCCAQQGCisGAQQB1nkCBAIEgfUE
gfIA8AB2AJQgvB6O1Y1siHMfgosiLA3R2k1ebE+UPWHbTi9YTaLCAAABd/imEZ8A
AAQDAEcwRQIhAKfVfGqrXBk/x+Wq2M1gyMI9jOgdaykqdbqgOiRPqCfsAiBbwRBe
oX9fbQT6PCxMDPsJ1/7dU0IztT4jI7oyH6FOiwB2AH0+8viP/4hVaCTCwMqeUol5
K8UOeAl/LmqXaJl+IvDXAAABd/imEcoAAAQDAEcwRQIhAMXvCjBCF4RjNI7ZNYHh
hCb1613OdQDvH4xeBBj9zyF3AiB49luGVh8tycmmRBKPR5HdsxRL0dzMyl0COo5k
bW661DANBgkqhkiG9w0BAQsFAAOCAQEARgY4eA166oHnAoNPziQwGbZ0hoByfCB6
YhvTPVPiSUBuMzgUsxUqlZGct28y9ZjO66BN+3QSkGmpwpeD8kH9f+Tvon/sEj4o
zeETWcx+s+zi6Grga1JF87uSxlRWL1I92/8cHCs69tIMY/O75cub1NSQS8fzrH4b
78b1BLydKv0T0+2SnXweFeQs01i+Lzd2TnWWOILrStWUn6H/XTNh8xGjeEqOmEJN
Iz5rwOZCbuZM80FTKJsyBy9xjRVoAuGilrIipO204I7Yb6bBSn7Lj/qvzDnf22q2
/J7lVyt4gTtcxoD8ilJ/XBdwFT9+yW/jSvSnWC1M/pwo/WFWn3ylDQ==
-----END CERTIFICATE-----
[Wed Mar  3 15:12:50 UTC 2021] Your cert is in  /etc/acme.sh/maparla@prhlt.upv.es/info.maparla.es/info.maparla.es.cer 
[Wed Mar  3 15:12:50 UTC 2021] Your cert key is in  /etc/acme.sh/maparla@prhlt.upv.es/info.maparla.es/info.maparla.es.key 
[Wed Mar  3 15:12:50 UTC 2021] The intermediate CA cert is in  /etc/acme.sh/maparla@prhlt.upv.es/info.maparla.es/ca.cer 
[Wed Mar  3 15:12:50 UTC 2021] And the full chain certs is there:  /etc/acme.sh/maparla@prhlt.upv.es/info.maparla.es/fullchain.cer 
[Wed Mar  3 15:12:50 UTC 2021] Installing cert to:/etc/nginx/certs/info.maparla.es/cert.pem
[Wed Mar  3 15:12:50 UTC 2021] Installing CA to:/etc/nginx/certs/info.maparla.es/chain.pem
[Wed Mar  3 15:12:50 UTC 2021] Installing key to:/etc/nginx/certs/info.maparla.es/key.pem
[Wed Mar  3 15:12:50 UTC 2021] Installing full chain to:/etc/nginx/certs/info.maparla.es/fullchain.pem
Creating/renewal maparla.es certificates... (maparla.es)
[Wed Mar  3 15:12:51 UTC 2021] Using CA: https://acme-v02.api.letsencrypt.org/directory
[Wed Mar  3 15:12:51 UTC 2021] Creating domain key
[Wed Mar  3 15:12:52 UTC 2021] The domain key is here: /etc/acme.sh/maparla@prhlt.upv.es/maparla.es/maparla.es.key
[Wed Mar  3 15:12:52 UTC 2021] Single domain='maparla.es'
[Wed Mar  3 15:12:52 UTC 2021] Getting domain auth token for each domain
[Wed Mar  3 15:12:54 UTC 2021] Create new order error. Le_OrderFinalize not found. {
  "type": "urn:ietf:params:acme:error:rateLimited",
  "detail": "Error creating new order :: too many certificates already issued for exact set of domains: maparla.es: see https://letsencrypt.org/docs/rate-limits/",
  "status": 429
}
[Wed Mar  3 15:12:54 UTC 2021] Please check log file for more details: /dev/null
Reloading nginx docker-gen (using separate container nginx-proxy-automation-gen)...
Reloading nginx (using separate container 490d233c8554b03c902578b8f30fe6debaae666b6fbc0a30c4ac32f489cd8cb3)...
Sleep for 3600s

info.maparla.es is working with HTTPS :) The error to maparla.es I think is not solvable, I have to wait one week I read in the letsencrypt forums, but the subdomain worked. What do you think?

If I have to update the code of the personal-site container... How should I create and update the container? Thanks in advance and for your work!

evertramos · 2021-03-03T17:01:07Z

thanks you very much for the tests! I will publish everything next thursday!

evertramos · 2021-03-10T17:22:08Z

@rwese how are you doing?

Could you test the script automation I have done in order to start the nginx proxy in more automated way?

I would really appreciate it if you do.

Thanks!!

v2

in bin folder you just run

$ ./fresh-start.sh

That should do it.

Please let me know what you think check the helper $ ./fresh-start.sh -h as well.

I added you to my private basescript repo, which I plan to opensoure soon.

You need to do run the git options below in order to get the submodule:

$ git checkout v2
$ git pull
$ cd basescript
$ git submodule init
$ git submodule update

Thanks!

evertramos · 2021-03-10T23:47:53Z

I will close ok? Thanks for your help!

rwese · 2021-03-11T07:39:20Z

I haven't yet had time, but will try it later this week and notify you if I find anything.

evertramos added the bug label Dec 10, 2020

evertramos added enhancement and removed bug labels Dec 17, 2020

evertramos closed this Mar 10, 2021

rwese deleted the patch-1 branch March 11, 2021 07:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: bind mount for acme.sh is missing #236

WIP: bind mount for acme.sh is missing #236

rwese commented Dec 9, 2020

evertramos commented Dec 10, 2020

rwese commented Dec 11, 2020

evertramos commented Dec 17, 2020

MarioProjects commented Mar 2, 2021

evertramos commented Mar 2, 2021

MarioProjects commented Mar 2, 2021 •

edited

Loading

MarioProjects commented Mar 3, 2021

evertramos commented Mar 3, 2021 •

edited

Loading

MarioProjects commented Mar 3, 2021 •

edited

Loading

evertramos commented Mar 3, 2021

MarioProjects commented Mar 3, 2021

evertramos commented Mar 3, 2021

evertramos commented Mar 10, 2021 •

edited

Loading

evertramos commented Mar 10, 2021

rwese commented Mar 11, 2021

WIP: bind mount for acme.sh is missing #236

WIP: bind mount for acme.sh is missing #236

Conversation

rwese commented Dec 9, 2020

evertramos commented Dec 10, 2020

rwese commented Dec 11, 2020

evertramos commented Dec 17, 2020

MarioProjects commented Mar 2, 2021

evertramos commented Mar 2, 2021

MarioProjects commented Mar 2, 2021 • edited Loading

MarioProjects commented Mar 3, 2021

evertramos commented Mar 3, 2021 • edited Loading

MarioProjects commented Mar 3, 2021 • edited Loading

evertramos commented Mar 3, 2021

MarioProjects commented Mar 3, 2021

evertramos commented Mar 3, 2021

evertramos commented Mar 10, 2021 • edited Loading

evertramos commented Mar 10, 2021

rwese commented Mar 11, 2021

MarioProjects commented Mar 2, 2021 •

edited

Loading

evertramos commented Mar 3, 2021 •

edited

Loading

MarioProjects commented Mar 3, 2021 •

edited

Loading

evertramos commented Mar 10, 2021 •

edited

Loading