Tang unlocking with HTTPS

[koenvanhauwe]

Can anyone confirm if clevis is able to unlock when the tang server is behind an HTTPS proxy?
My tang server is public so HTTPS is required. It succesfully binds when running clevis luks bind, but during boot I get 'Error communicating with the server!'.

Thanks

[jmarcelletti]

Probably not the answer you are looking for (because I can't 100% answer your question but..).. Having tang public seems to defeat the purpose? Further, https doesn't provide you anything. The McCallum-Relyea is designed to work over HTTP. If you are concerned about somebody snooping the HTTP connection, and then doing the brute force on the communication to determine anything.. remember they'd still need access to your machine to make use of any of that. Which.. if they had access to your machine with a public tang server, HTTP or HTTPS, they'd have all your data anyway.

[npmccallum]

While there is no HTTPS requirement, there is also nothing preventing clevis from talking to tang over an HTTPS proxy. The only problem is that we haven't tested it. If you want to test it and fix any issues that arise, we're glad to merge the patches.

[npmccallum]

@koenvanhauwe FYI, the relevant HTTP request is here: https://github.com/latchset/clevis/blob/master/src/pins/tang/clevis-decrypt-tang#L78

It is just curl, which does HTTPS just fine. So making it work should be trivial.

[koenvanhauwe]

Probably not the answer you are looking for (because I can't 100% answer your question but..).. Having tang public seems to defeat the purpose? Further, https doesn't provide you anything. The McCallum-Relyea is designed to work over HTTP. If you are concerned about somebody snooping the HTTP connection, and then doing the brute force on the communication to determine anything.. remember they'd still need access to your machine to make use of any of that. Which.. if they had access to your machine with a public tang server, HTTP or HTTPS, they'd have all your data anyway.

My setup is kinda different than most people. I have a small device (RPi) I'm placing in a strange network. If the tang server has to be in the same network I would have to place another device running the tang server, which is not what I want.
What I'm doing instead is running a firewall on my public tang server which only allows requests from the public IP where the encrypted device is situated. When the device gets stolen it wont be making requests from the same IP anymore and wont unlock.

I've seen it uses curl, and I see several requests incoming on my tang server when booting. But something goes wrong somewhere (error communicating with the server). I'll have to do some further testing.

[npmccallum]

@koenvanhauwe Another possible feature you might want to think about is client certificate support. That way you don't have to track the IP. The client gets a certificate and does TLS client auth to the server.

[jmarcelletti]

@npmccallum Wouldn't that defeat the point to some extent? If the machine gets stolen it would have the client cert and therefore auth and decrypt itself?

[Tetha]

I just stumbled across this ticket. I'd like to share my knowledge on this, because I've been stuck on this for a day or so until the lights turned on and they are way too bright. I don't think clevis is able to unlock the root partition of a system in a default initramfs with an https-enabled tang server, at least that's what my current status tells me, unless extra steps are taken.

After a bunch of back and forth on my way to a server provisioning and build queue, I eventually ended up with a containerized tang-server, which was loadbalanced as https://build-only-insecure-tang-server.example.com. This setup mostly exists so the initial provisioning can reboot the system multiple times and it would decrypt it's rootfs automatically during the build process. One of the last build steps removes the insecure tangserver and only a bunch of tang-servers with private IPs are able to decrypt the rootfs.

Given that, some build script would call https://build-only-insecure-tang-server.example.com/adv and push the received advertisement into a kickstart file as a clevis luks bind and go from there. This way, the kickstarted server doesn't have to communicate with the tang server or check trust with it upon build.

However, upon reboot, this doesn't work. The system only logs "Unable to communicate with server!". Once dracut times out and dumps me into an emergency shell, I can execute the curl-call on it's own:

# curl https://build-only-insecure-tang-server.example.com
curl: (77) error setting certificate verify locations:
    CAfile: /etc/pki/tls/certs/ca-bundle.crt
    CApath: none

In other words, the HTTPS-request fails, because the root certificates aren't in an expected place. Checking around, the initramfs (in my case) doesn't contain any ca-bundle. The only way to get this request to work is by telling curl to skip the SSL verification via -k. This all in all makes sense because the initramfs build by dracut and such is indeed very minimal and usually shouldn't need HTTPs connections.

No such argument is set in the call linked by @npmccallum , and as such, with an https enabled tang server (or ssl terminating proxy in front of it), the tang unlocking will fail even with layer 3 connectivity in place (another fun thing on my end atm) due to a cert validation error.

This should however work if someone manages to fenagle the ca-bundle.crt into the initramfs.

As a check, I've switched the loadbalancer to allow the tang-server to be accessed via http, and the boot works as expected.

[francsw]

A workaround for this problem is to add --insecure to the curl command here https://github.com/latchset/clevis/blob/master/src/pins/tang/clevis-decrypt-tang#L78
if ! rep="$(curl --insecure -sfg -X POST -H "$ct" --data-binary @- "$url" <<< "$xfr")"; then

I'm working on getting certificates to work and will post here if I get it working. See comment below

[francsw]

If someone ends up here trying to get Clevis to use HTTPS, here are some steps and scripts to help you along.
https://github.com/francsw/clevis-HTTPS

[bdelwood]

A possible solution is to add the url-lib module to dracut, as it includes SSL certs.

[keitharogers]

I've been banging my head against a brick wall trying to use Clevis with a Tang server behind a load balancer using 'https' as well. Eventually I updated the clevis initramfs hook script with the following line on my Ubuntu 22.04 install:

copy_file data /etc/ssl/certs/ca-certificates.crt || die 2 "Unable to copy certificate bundle to initrd image"

This fixed the issue after running update-initramfs -u -k all.

After doing numerous packet captures, I came to the conclusion the TLS handshake wasn't happening and then found that someone raised a PR very similar to what I have above: 47fd9d5 but it appears to have been ignored for 2 years. Incidentally, there PR doesn't actually work for me, so I may raise a new one, but is there any point?

I really feel like this should be possible out of the box without additional steps.

[MaxiStarling56]

Hi, after some testing i was able to use a tang server behind https, without modifying the source code of clevis, when building with dracut add the following: --include /etc/ssl /etc/ssl --force, @latchset i think it should be added in the docs.

[deamen]

In RHEL9, only the ca-bundle.crt is needed:

# cat /etc/dracut.conf.d/clevis.conf

hostonly_cmdline=yes
kernel_cmdline+=" rd.neednet=1 "
install_items+=/etc/pki/tls/certs/ca-bundle.crt

Then re-generate the initrd:

dracut -fv --regenerate-all

This should keep the auditor happy.

[nemihome]

I'm currently struggeling with the same topic with initfamfs-tools (Ubuntu 22.04) without dracut (with dracut the system does not even boot in my case and dependencies drop dropbear before installtion of dracut). Dropbear does use SSL, Clevis does not.

I can bind the keys in combination with https connection to luks device with booted system without problems (using HAProxy infront of tang but seems to be clevis-initramfs does not unclude SSL). Without SSL it's working with SSL I see the clevis message at early boot and nothing happens.

Update: Did not see the post above francis, I found it via Google. That's working, perfect.