Extra flags in TCP handshake will break JA4L calculation

[aminebenhariz]

During the testing of JA4+ in our labs at Venari Security, we noticed that the JA4L is not showing for some TCP sessions. After further investigation, we narrowed down this behaviour to TCP handshake flags.

TCP handshake usually follows the SYN, SYN-ACK, ACK sequence. But for some systems (specially with ECN enabled by default on Apple products, since iOS 11 and macOS Sierra). Also, we often encounter the usage of PSH flag in the TCP handshake in many systems.

Here is a example of a TLS session with ECN (pcap file: macos_tcp_flags.zip)

Notice how the extra flags ECN and CWR are used in the TCP handshake. Those extra flags will prevent the script from calculating the JA4L-C and JA4L-S because it's expecting a clean SYN, SYN-ACK, ACK sequence:

$ python3 python/ja4.py macos_tcp_flags.pcap
{'stream': 0, 'src': '172.16.5.16', 'dst': '172.67.24.71', 'srcport': '61311', 'dstport': '443', 'domain': 'venarisecurity.com', 'JA4': 't13d2613h2_2802a3db6c62_845d286b0d67', 'JA4S': 't130200_1301_234ea6891581'}

[john-althouse]

Thanks, good catch! We'll want to add this fix to the Rust and Wireshark versions as well. @igr001-galactica

[vvv]

@aminebenhariz @john-althouse So how should the corrected JA4L algorithm handle TCP flags?

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Source Port          |       Destination Port        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Sequence Number                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Acknowledgment Number                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Data |       |C|E|U|A|P|R|S|F|                               |
   | Offset| Rsrvd |W|C|R|C|S|S|Y|I|            Window             |
   |       |       |R|E|G|K|H|T|N|N|                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Checksum            |         Urgent Pointer        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           [Options]                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               :
   :                             Data                              :
   :                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Option 1

JA4L should look at SYN and ACK bits only.

'tcp[13] & 0x12 == 2'     # SYN
'tcp[13] & 0x12 == 0x12'  # SYN-ACK
'tcp[13] & 0x12 == 0x10'  # ACK

Option 2

JA4L should unset CWR, ECE, and PSH bits and compare the result with SYN, SYN-ACK, and ACK.

'tcp[13] & 0x37 == 2'     # SYN
'tcp[13] & 0x37 == 0x12'  # SYN-ACK
'tcp[13] & 0x37 == 0x10'  # ACK

Option 3

?

---

@aminebenhariz's patch (#23) uses Option 1. I'll apply it for the Rust implementation.