Traffic Dump: dump server-side protocol stack

[bneradt]

This also adds some TLS information in the protocol stack, such as the
verify_mode applied to the SSL session and whether the server requested
a client certificate.

With this change, there are now two kinds of protocol stacks that are
dumped. There is a session-level one that describes the client-side
protocol stack. It can look something like this:

    "sessions": [
        {
            "protocol": [
                {
                    "name": "http",
                    "version": "2"
                },
                {
                    "name": "tls",
                    "version": "TLSv1.2",
                    "sni": "www.tls.com",
                    "proxy-verify-mode": 0,
                    "proxy-provided-cert": true
                },
                {
                    "name": "tcp"
                },
                {
                    "name": "ip",
                    "version": "4"
                }
            ],

In addition, there is now a server-side protocol stack that will be printed in
every proxy-request node. It can look something like this:

                    "proxy-request": {
                        "protocol": [
                            {
                                "name": "http",
                                "version": "1.1"
                            },
                            {
                                "name": "tls",
                                "version": "TLSv1.2",
                                "sni": "www.tls.com",
                                "proxy-verify-mode": 1,
                                "proxy-provided-cert": false
                            },
                            {
                                "name": "tcp"
                            },
                            {
                                "name": "ip",
                                "version": "4"
                            }
                        ],

Note that the dump provides TLS information from the perspective of the proxy,
therefore the TLS nodes are prefixed by "proxy-". The schema is also updated to
anticipate verifier directives that dictate client and server TLS behavior
("verify-mode" and "request-certificate").

[maskit]

You say protocol stack but you don't care the order?
What's the key for protocol? ALPN id? Protocol name? Protocol name with version?
I'd define the format like this:

"protocol": [
  {
    "name": "ip",
    "version" : "4"
  },
  {
    "name": "tcp"
  },
  {
    "name": "tls",
    "version" : "1.2",
    "alpn": "h2",
    "sni": "example.com"
  },
  {
    "name": "http",
    "version": "2"
  },
]

Also, I'd be curious what the format for QUIC would be like. It could be "udp", "quic" and "tls", but technically "tls" is not on top of "quic".

[bneradt]

[approve ci]

[bneradt]

You say protocol stack but you don't care the order?

Oh, good point. The JSON pretty printer I used changed the order around. I'll fix that.

That said, the way I wrote the plugin, the "tls" node is handled separately and is always placed at the end. I'll see whether I can fix that.

[bneradt]

You say protocol stack but you don't care the order?

Oh, good point. The JSON pretty printer I used changed the order around. I'll fix that.

That said, the way I wrote the plugin, the "tls" node is handled separately and is always placed at the end. I'll see whether I can fix that.

With the latest commit, the stack now looks like the following (note that the protocols are in order, from higher to lower layers):

    "sessions": [
        {
            "protocol": {
                "h2": {},
                "tls": {
                    "version": "TLSv1.2",
                    "sni": "www.tls.com",
                    "proxy-verify-mode": 0,
                    "proxy-provided-cert": true
                },
                "tcp": {},
                "ipv4": {}
            },

And the server side:

                    "proxy-request": {
                        "protocol": {
                            "http/1.1": {},
                            "tls": {
                                "version": "TLSv1.2",
                                "sni": "www.tls.com",
                                "proxy-verify-mode": 1,
                                "proxy-provided-cert": false
                            },
                            "tcp": {},
                            "ipv4": {}
                        },

[maskit]

Thanks for the update, but that's not my point unfortunately. I'm saying "protocol" should be an array.
https://stackoverflow.com/questions/7214293/is-the-order-of-elements-in-a-json-list-preserved

[bneradt]

Thanks for the update, but that's not my point unfortunately. I'm saying "protocol" should be an array.
https://stackoverflow.com/questions/7214293/is-the-order-of-elements-in-a-json-list-preserved

Ah, that makes sense. Sorry, I was thinking that you were pointing out that the map elements weren't ordered according to the protocol layers.

I think you're absolutely right. Making the protocol an array is the right thing to do. I've done that, using the scheme you described, in the latest commit.

Thanks for your feedback, I think this improves things a lot.

[bneradt]

Thanks for the update, but that's not my point unfortunately. I'm saying "protocol" should be an array.
https://stackoverflow.com/questions/7214293/is-the-order-of-elements-in-a-json-list-preserved

Ah, that makes sense. Sorry, I was thinking that you were pointing out that the map elements weren't ordered according to the protocol layers.

I think you're absolutely right. Making the protocol an array is the right thing to do. I've done that, using the scheme you described, in the latest commit.

Thanks for your feedback, I think this improves things a lot.

Note that I updated the PR comment with a description of what the protocol nodes look like now.

[maskit]

The output looks good to me now. Thank you for updating. I haven't looked into the detail yet but I left a few comments.

Since you're adding new TS APIs, please follow the API review process below.
https://cwiki.apache.org/confluence/display/TS/API+Review+Process

[bneradt]

The output looks good to me now. Thank you for updating. I haven't looked into the detail yet but I left a few comments.

Since you're adding new TS APIs, please follow the API review process below.
https://cwiki.apache.org/confluence/display/TS/API+Review+Process

Thank you Masakazu. I wasn't aware of this review process. I sent an email to the dev list concerning these functions.