Check for the signature algorithms supported by the server

[nabla-c0d3]

Bindings for SSL_set1_sigalgs() are now available in nassl (nabla-c0d3/nassl#124). They could probably be used to check which signature algorithms are supported by the server.

Some documentation regarding the signature algorithms extension:

• https://timtaubert.de/blog/2016/07/the-evolution-of-signatures-in-tls/
• https://security.stackexchange.com/questions/233658/how-to-interpret-security-impact-of-server-signature-algorithms-sslscan

[mxsasha]

The check I wrote to be included in internet.nl with nabla-c0d3/nassl#118 does something similar, but not precisely. It may be useful to consider.

The standard we follow is the Dutch NCSC guidelines for TLS which requires SHA2 support for signatures in key exchange. Notably different from many other requirements in this standard and common practice: it does not require exclusively SHA2 support. So it is sufficient if SHA2 is supported, even if other less secure hashes are also supported. This standard is currently being updated, so our requirements may very well change this year.

My check now calls set_sigalgs() with all digest+pubkey tuples with SHA2 and attempts to connect. If this fails, we presume SHA2 is not supported. Then, I call get_peer_signature_nid() to check the used hash. That second step would seem superfluous, but I have seen one case where a server chose SHA1 despite the client saying it only supported SHA2, and the connection working because SHA1 was actually supported by the openssl client. However, we should count the server as failing SHA2 support.