pki: Embedded ACME server

[mholt]

This PR implements an embedded ACME server, powered by Smallstep libraries.

See #3021

It's really cool! You can use it simply by giving your own ACME directory endpoint in your other Caddy configs.

🚫 Not ready to merge. ✅ Ready to merge after 2.0

Still some cleanup to do, and some more features to come as well!

Below is a config that demonstrates this feature. To use it, simply add smallstep.rocks to your hosts file so that it points to 127.0.0.1.

When you start Caddy, you should see that it will try to obtain a certificate for smallstep.rocks from the ACME CA at https://localhost/acme/local/directory.

{
	"apps": {
		"http": {
			"servers": {
				"dev": {
					"listen": [":443"],
					"routes": [
						{
							"match": [
								{"host": ["localhost"]}
							],
							"handle": [
								{"handler": "acme_server"},
								{"handler": "static_response", "body": "Hello! Not an ACME request."}
							]
						}
					]
				}
			}
		},
		"tls": {
			"certificates": {
				"automate": [
					"smallstep.rocks"
				]
			},
			"automation": {
				"policies": [
					{
						"issuer": {
							"module": "acme",
							"ca": "https://localhost/acme/local/directory"
						}
					}
				]
			}
		}
	}
}

My logs:

$ go run main.go run --config acme.json
2020/03/27 16:52:57.456 INFO    using provided configuration    {"config_file": "acme.json", "config_adapter": ""}
2020/03/27 16:52:57.503 INFO    admin   admin endpoint started  {"address": "localhost:2019", "enforce_origin": false, "origins": ["localhost:2019"]}
2020/03/27 10:52:57 [INFO][cache:0xc000878910] Started certificate maintenance routine
2020/03/27 16:52:57.504 INFO    http    server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "dev", "https_port": 443}
2020/03/27 16:52:57.504 INFO    http    enabling automatic HTTP->HTTPS redirects        {"server_name": "dev"}
2020/03/27 16:52:57.511 INFO    tls     setting internal issuer for automation policy that has only internal subjects but no issuer configured  {"subjects": ["localhost"]}
2020/03/27 10:52:57 [INFO][smallstep.rocks] Obtain certificate; acquiring lock...
2020/03/27 10:52:57 [INFO][smallstep.rocks] Obtain: Lock acquired; proceeding...
2020/03/27 16:52:57.545 INFO    tls     cleaned up storage units
2020/03/27 10:52:57 [ERROR] Making new ACME client: get directory at 'https://localhost/acme/local/directory': Get "https://localhost/acme/local/directory": dial tcp [::1]:443: connect: connection refused (attempt 1/2)
2020/03/27 16:52:57.654 INFO    pki.ca.local    root certificate is already trusted by system   {"path": "storage:pki/authorities/local/root.crt"}
2020/03/27 16:52:57.654 INFO    http    enabling automatic TLS certificate management   {"domains": ["localhost"]}
2020/03/27 10:52:57 [WARNING] Stapling OCSP: no OCSP stapling for [localhost]: no OCSP server specified in certificate
2020/03/27 16:52:57.655 INFO    autosaved config        {"file": "/Users/matt/Library/Application Support/Caddy/autosave.json"}
2020/03/27 16:52:57.655 INFO    serving initial configuration
2020/03/27 10:52:58 {"newNonce":"https://localhost/acme/local/new-nonce","newAccount":"https://localhost/acme/local/new-account","newOrder":"https://localhost/acme/local/new-order","revokeCert":"https://localhost/acme/local/revoke-cert","keyChange":"https://localhost/acme/local/key-change"}
2020/03/27 10:52:58 {"status":"valid","orders":"https://localhost/acme/local/account/nkTzog8cB7UrAnIYlR0Kf13XYSg6NwJS/orders"}
2020/03/27 10:52:58 [INFO][smallstep.rocks] Waiting on rate limiter...
2020/03/27 10:52:58 [INFO][smallstep.rocks] Done waiting
2020/03/27 10:52:58 [INFO] [smallstep.rocks] acme: Obtaining bundled SAN certificate given a CSR
2020/03/27 10:52:58 {"status":"pending","expires":"2020-03-28T16:52:59Z","identifiers":[{"type":"dns","value":"smallstep.rocks"}],"notBefore":"0001-01-01T00:00:00Z","notAfter":"0001-01-01T00:00:00Z","authorizations":["https://localhost/acme/local/authz/KyGbdYrA9xVLmJK6UI3ExW376hsiG1s9"],"finalize":"https://localhost/acme/local/order/7qcFUSq12gynXfIQlMsCFdRXlLmhkyUQ/finalize"}
2020/03/27 10:52:58 {"identifier":{"type":"dns","value":"smallstep.rocks"},"status":"pending","expires":"2020-03-28T16:52:59Z","challenges":[{"type":"http-01","status":"pending","token":"GIqfrUkkVThJa2zA3kglUMrkCBzhTIYh","url":"https://localhost/acme/local/challenge/XNdyhajxbVHFve42rJOvaoK0YxlbtDt8"},{"type":"tls-alpn-01","status":"pending","token":"rp6Lmvugg1dT75qxQlaBSLPFuTtCzIAj","url":"https://localhost/acme/local/challenge/SNoUMHk1eLeb6bk5eUUjQXGfRx3OUCKv"},{"type":"dns-01","status":"pending","token":"gflNMacyMR6bPxbf0kybd74MfdnpywyK","url":"https://localhost/acme/local/challenge/RYSY2RFDZSTr77VB3A4NFGteSQrBqutp"}],"wildcard":false}
2020/03/27 10:52:58 [INFO] [smallstep.rocks] AuthURL: https://localhost/acme/local/authz/KyGbdYrA9xVLmJK6UI3ExW376hsiG1s9
2020/03/27 10:52:58 [INFO] [smallstep.rocks] acme: Could not find solver for: tls-alpn-01
2020/03/27 10:52:58 [INFO] [smallstep.rocks] acme: use http-01 solver
2020/03/27 10:52:58 [INFO] [smallstep.rocks] acme: Trying to solve HTTP-01
2020/03/27 10:52:58 [INFO][smallstep.rocks] Served key authentication (HTTP challenge)
2020/03/27 10:52:58 {"type":"http-01","status":"valid","token":"GIqfrUkkVThJa2zA3kglUMrkCBzhTIYh","validated":"2020-03-27T16:52:59Z","url":"https://localhost/acme/local/challenge/XNdyhajxbVHFve42rJOvaoK0YxlbtDt8"}
2020/03/27 10:52:58 [INFO] [smallstep.rocks] The server validated our request
2020/03/27 10:52:58 [INFO] [smallstep.rocks] acme: Validations succeeded; requesting certificates
2020/03/27 10:52:58 {"status":"valid","expires":"2020-03-28T16:52:59Z","identifiers":[{"type":"dns","value":"smallstep.rocks"}],"notBefore":"0001-01-01T00:00:00Z","notAfter":"0001-01-01T00:00:00Z","authorizations":["https://localhost/acme/local/authz/KyGbdYrA9xVLmJK6UI3ExW376hsiG1s9"],"finalize":"https://localhost/acme/local/order/7qcFUSq12gynXfIQlMsCFdRXlLmhkyUQ/finalize","certificate":"https://localhost/acme/local/certificate/IGVyxAJ5P0OsGGgC5j4E7GIX3Sp437bJ"}
2020/03/27 10:52:58 [INFO] [smallstep.rocks] Server responded with a certificate.
2020/03/27 10:52:58 [INFO][smallstep.rocks] Certificate obtained successfully
2020/03/27 10:52:58 [INFO][smallstep.rocks] Obtain: Releasing lock

You'll notice that the ACME server is in fact just an HTTP handler module, which is pretty neat.

The above config also uses all default settings, but some things can be customized.

/cc @mmalone @maraino @dopey - at some point, it would be good to do a review together!

Update/bonus: TLS client auth certs can now be automated as well!