SslStreamCertificateContext forced validation, X509Chain platform differences

[dbeinder]

Description

I was looking forward to the resolution of #35844 and being able to explicitly set the certificate chain for a SslStream. In testing the current (5.0 preview8) implementation I found a few pain points:

• There is no control over whether to send the Root certificate, this is hard-coded, per-platform
  • Especially on low-throughput / IoT applications, being able to reduce the handshake size would be nice
  • Incidentally, this line in SslStreamCertificateContext seems to do the opposite of what the bool implies
• There is no Create overload to create a context without chain validation
  • This would be useful to create tests with malformed / malicious certificate chains
  • Trying to create a context with an invalid chain does not work on Linux, this is due to an inconsistency in X509Chain
  • I believe this has been noticed before (X509Chain behaviour inconsistent on Windows and Linux #29164), and the result was that nothing could be done about it - in that case I'd think that makes the case stronger for a Create overload without verification
  • The problem on Linux is only caused by the static Create function, it works when creating SslStreamCertificateContext by invoking the private constructor using reflection
  • If creating a context with a prevalidated chain is really not possible, there should at least be a way to tell, that parts of the chain were discarded and won't be sent out
• In general, it would be great to have a flag to tell SslStream to only use the chain exactly as it was given / without trying to work some back-channel magic, user-dependent certificate stores, automatic chain building, etc.

My test code: in this gist
It shows the platform-dependent behavior of X509Chain and then starts a test server, so the advertised chain can be checked using openssl s_client -connect localhost:8077

Configuration

• .NET 5.0 preview 8, x64, Windows 10 v2004 / Debian 10.0 WSL

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[scalablecory]

Thanks for trying out the preview. Do you have some specific APIs you can outline here for us to discuss?

CC @wfurt @bartonjs

[dbeinder]

Currently, this factory will silently discard intermediate certificates if the X509Chain verification fails, depending on the platform:

SslStreamCertificateContext.Create(X509Certificate2 target, X509Certificate2Collection? additionalCertificates, bool offline)

I would like a flag or overload to avoid this behavior and have it just use the provided chain as is, including the root, without creating a dependency on the presence / absence of objects in the system certificate store. I believe that was also the wish of #35844 and earlier requests.

Something like this:
SslStreamCertificateContext.CreateUnchecked(X509Certificate2 target, IEnumerable<X509Certificate2>? additionalCertificates = null)
The user would be responsible to pass a valid chain, the order of the elements should be reflected in the TLS Hello message.

[wfurt]

The root should never go out IMHO. If the peer does not have it (and trust it), the handshake will fail anyway.
The possibility of accepting exactly what user provided was discussed - and rejected.
That would not be generally feasible - at least not on Windows.
And the goal was not to send invalid data for testing.

[wfurt]

the condition was indeed wrong @dbeinder. fixed with #46904 for 5.0

Biggest technical problem with taking just the list and sending ASIS its fact that it is not doable on Windows AFAIK. The option to give the power to the caller was discussed. While I agree it may be useful for testing, it was rejected by quorum.

Aside from testing corner cases, is there some particular problem you are trying to solve @dbeinder? There may be some more work in this area for 6.0 and I would like to understand the use case

[dbeinder]

Thanks @wfurt, I must have misread the older discussions.

I was mostly hoping for the option to lose the dependency on the Windows certificate store, with all its permissions and behind-the-scenes magic. While it is great to have the option to integrate X509 operations with the full framework that extends into smartcards / HSM hardware, etc - it would be really nice to be able have a fully self contained app, that just does TLS with the Root-CA and keys built into my app without going through the full cert store.

The exceptions from the SChannel/SSPI backend are usually rather cryptic and generic and I've just wasted a lot of time debugging exceptions that came down to the certificate being stuck in the store but secured and the executing user not having the permissions - when all I'm trying to do is run a client app that has the CA embedded and the client private key in a config file.

I understand SslStream is more geared towards the web with the full multi-RootCA infrastructure, but when I'm using it for securing a custom protocol with our own PKI, the cert store gets more in the way than it helps. Because we also deploy on Linux, where the certificate store is just emulated, I'd rather not use it at all. If we just used Windows, there could be an argument to take advantage of the deep integration into system users / permissions, but for our use case having a more platform-independent, self-contained TLS wrapper in .NET would be most useful.

[wfurt]

extends into smartcards / HSM hardware, etc - it would be really nice to be able have a fully self contained app, that just does TLS with the Root-CA and keys built into my app without going through the full cert store.

That is not possible on Windows. Schannel can only use certificates via store (and capi2) We wanted to take X509Certificate2Collection as custom trust for #45456 but it may not be possible because of Windows limitations.

I think one way you can try to approach is to create and use custom store. It is still a store, but application it self can be responsible for maintaining the content. That is still not quite what you want but I'm not sure that is feasible on all platforms at the moment.

That still leaves you with Schannel errors. I no way out of it at the moment. Perhaps you can do some chain and trust validation upfront e.g. use X509Chain before attempt of TLS handshake to see if the the chain can be constructed. That would at least show any problems with permissions or store content.

[wfurt]

I don't think we can implement it on Windows. In general there is no way how to pass specific certificates to SSPI and schannel to use them exactly.
This can be revisited if Windows add some way how to do it or if somebody can make it work with current versions.
Feel free to open new issue @dbeinder if I missed anything.