select the stream muxers by intersecting the lists of supported muxers

[marten-seemann]

To select the stream muxer, in #227, the server sends a list of supported stream muxers and lets the client pick one:

client                                                server
                                                       <- Offer [ A, B, C ]
*picks B* -> Use [ B ]

This can be done in parallel to the cryptographic handshake because we use Early Data, right after receiving the ClientHello. The list is encrypted with 1-RTT keys, but at this moment the server hasn't yet verified the client's identity. A similar reasoning applies to Noise handshakes.

@raulk remarked in his review of #227 that there's a different option (which we had discussed before, but which I hadn't included in the final proposal). The motivation is that #227 relies on assumptions about the cryptographic handshake (it's the server that's able to send Early Data first), which hold true for TLS 1.3 and Noise, but might not hold for every cryptographic handshake protocol.
Instead of specifying which endpoint offers the stream muxers, and which one picks, we could instead have both endpoints send a list of muxers at the earliest time possible (as soon as the handshake protocols allows sending of encrypted application data). The stream multiplexer is then determined by an algorithm that picks a mutually supported stream multiplexer. If this algorithm is deterministic, both endpoints will select the same stream multiplexer, and it doesn't matter which endpoint sent the list first:

client                                              server
-> OfferMuxer [ B, C, D ]                           <- OfferMuxer [ A, B, C ]
*=> use B*                                             * => use B*

The downside of this proposal is that we need a third protobuf message. I think we can live with that.

[yusefnapora]

Nice, I like this approach.

I think I also like changing the Offer message to offer a single protocol, since it's simpler. But it seems like we're losing the ability to offer multiple versions of the same protocol, where we want the remote peer to choose at most one of them. For example I could send /dht/2.0 and /dht/1.0 and you can pick whichever you support.

I suppose I could just send multiple Offers in parallel though, and close whichever streams I don't want if the responding peer accepts more than one, but that seems like it could be tricky to manage and possibly a bit racy, since the streams will be opened asynchronously.

@raulk I seem to remember you saying the speculative "one of" protocol selection was an important requirement, but I can't remember exactly where you said it 😄 Do you mind weighing in here?

[raulk]

@yusefnapora one simple example is multiple versions of the same protocol. If the initiator is able to speak /dht/v1 and /dht/v2, it can pack both request messages in the same packet and let the responder decide which protocol it chooses.

This scheme has the downside of being subject to downgrade attacks, but so is anything p2p where negotiation occurs.

Even if we did a serial negotiation, where we offer v2 and the peer does not know that we also offer v1, an attacker could stall that stream awaiting our response from the Identify protocol which enumerates the concrete protocols we support, to then subsequently reject v2 to force us to fall back to v1.

[raulk]

@marten-seemann haven't read the diff or thread in detail, but would you mind summarising how the proposed protocol enables one-of XOR protocol selection? e.g. using XML:

<one-of>
  <choice id="1">
    <id>/dht/v1</id>
    <message>[[ message ]]</message>
  </choice>
  <choice id="2">
    <id>/dht/v2</id>
    <message>[[ message ]]</message>
  </choice>
  <choice id="3">
    <id>/dht/v3</id>
    <message>[[ message ]]</message>
  </choice>
</one-of>

[marten-seemann]

I realize I should have kept things separate here. Removing the option to propose multiple protocols in Offer makes selecting protocol versions more difficult, as @yusefnapora noted. I've reverted this commit.

[raulk]

Copying over a 3-way convo elsewhere, slightly modified, to keep the discussion in a single thread.

@marten-seemann:
I remember that @Stebalien once objected to treating stream muxer selection any different than any other protocol negotiation. It seems though that there are actual benefits of acknowledging that stream muxer selection is a special step in the connection bootstrapping.

@Stebalien:
There's no reason stream-muxer selection needs to be tied to performance. The goal behind not making it special was to allow, e.g., layering compression below it and/or simply not negotiating a stream muxer in certain cases.

@raulk:
Is the compression / alternate stream transcoders feature captured in the ms2.0 design doc? If this was discussed in the original 100+-comment PR, I’m afraid it was not elicited elsewhere, and it can easily fall off the radar.

@raulk:
What you are pointing to is “connection capabilities”. We don’t need to build a recursive protocol for that IMO.

@raulk:
Instead of communicating stream multiplexers, we can just communicate connection capabilities, and have the initiator decide which ones it wants to use. On the surface this would seem like a 1.5 RTT interaction, but in practice it’s 1 RTT because the initiator would pipeline the “connection capability selection” with their first application-level message.

[marten-seemann]

@Stebalien Not negotiating a stream muxer is possible with the current proposal (#227) by offering the /monoplex stream "multiplexer".

@raulk @Stebalien Compression / alternate stream transcoder selection is not in the design doc. Adding additional requirements that late is a bit difficult, but I hope we can still make it work.

Does it make sense to first apply encryption, and then apply a stream muxer? It seems to me that applying encryption above the stream muxer makes a lot more sense (and potentially makes cross-protocol attacks harder).

Advertising capabilities instead of muxers could be done in two ways:

1. We could either concatenate them, so an endpoint would send: /yamux, /yamux/gzip, /yamux/brotli, /spdy, /spdy/gzip, /yamux/brotli, or
2. we could introduce a new field on the OfferMultiplexer protobuf, e.g. change it to

message OfferMultiplexer {
    repeated string mutltipexers = 1;
    repeated string compressors = 2;
}

I think I'd prefer option 2 here, since it doesn't blow up the list we're sending to N*M entries. We'd probably then need to define an entry /nocompress as well, which is sent when an endpoint doesn't want to use compression at all.

I'd like to point out that the protobuf format will allow us to specify further capabilities in a backwards-compatible way in the future: We will define a new field on the OfferMultiplexer protobuf, which updated implementations will know how to send and how to interpret. Old implementations will not send this field, and will skip it when parsing.

[marten-seemann]

Merging this PR. I think this is good enough, let's discuss everything combined in #227.