Quic ietf 4967 v8 #7130

catenacyber · 2022-03-10T12:48:48Z

Link to redmine ticket:
https://redmine.openinfosecfoundation.org/issues/4967
https://redmine.openinfosecfoundation.org/issues/5143
https://redmine.openinfosecfoundation.org/issues/5166

Describe changes:

Parses standardized QUIC IETF v1
bump up rust carte tls_parser already used by rep, so that it can be used by quic
Rustfmt previous code

suricata-verify-pr: 780
OISF/suricata-verify#780

Replaces #7122 with naming the keyword ja3s with an s if it is from the server

Still to do more generally about quic : see https://redmine.openinfosecfoundation.org/issues/4966 (frame support)

Ticket: 4967 The format of initial packet for quic ietf, ie quic v1, is described in rfc 9000, section 17.2.2

so that we can use new functions in quic parser

and logs interesting extensions from crypto frame

As it can be 4, but it can also be 1, based on the first decrypted byte

The way to determine if the payload is encrypted is by storing in the state if we have seen a crypto frame in both directions...

So as to keep parse not too big

for detection

Ticket: 5143

Ticket: 5166

As the ja3 standard names it so, with an s, when it comes from the server to the client.

codecov · 2022-03-10T15:20:04Z

Codecov Report

Merging #7130 (71bc1fd) into master (3a490fb) will increase coverage by 0.01%.
The diff coverage is 92.15%.

@@            Coverage Diff             @@
##           master    #7130      +/-   ##
==========================================
+ Coverage   78.06%   78.08%   +0.01%     
==========================================
  Files         628      629       +1     
  Lines      185266   185319      +53     
==========================================
+ Hits       144635   144702      +67     
+ Misses      40631    40617      -14

Flag	Coverage Δ
fuzzcorpus	`60.08% <52.94%> (+0.09%)`	⬆️
suricata-verify	`54.58% <88.88%> (-0.02%)`	⬇️
unittests	`63.12% <74.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

victorjulien · 2022-03-11T06:13:49Z

ja3s is still logged as ja3. For a comparison, see our tls-ja3s SV test:

{
  "timestamp": "2019-05-15T08:11:18.955582+0000",
  "flow_id": 143691514005172,
  "pcap_cnt": 11,
  "event_type": "tls",
  "src_ip": "2a03:b0c0:0002:00d0:0000:0000:0bd3:4001",
  "src_port": 48106,
  "dest_ip": "2606:2800:0220:0001:0248:1893:25c8:1946",
  "dest_port": 443,
  "proto": "TCP",
  "tls": {
    "subject": "C=US, ST=California, L=Los Angeles, O=Internet Corporation for Assigned Names and Numbers, OU=Technology, CN=www.example.org",
    "issuerdn": "C=US, O=DigiCert Inc, CN=DigiCert SHA2 Secure Server CA",
    "serial": "0F:D0:78:DD:48:F1:A2:BD:4D:0F:2B:A9:6B:60:38:FE",
    "fingerprint": "7b:b6:98:38:69:70:36:3d:29:19:cc:57:72:84:69:84:ff:d4:a8:89",
    "sni": "example.com",
    "version": "TLS 1.2",
    "notbefore": "2018-11-28T00:00:00",
    "notafter": "2020-12-02T12:00:00",
    "ja3": {
      "hash": "1fe4c7a3544eb27afec2adfb3a3dbf60",
      "string": "771,49196-49200-159-52393-52392-52394-49195-49199-158-49188-49192-107-49187-49191-103-49162-49172-57-49161-49171-51-157-156-61-60-53-47-255,0-11-10-13172-16-22-23-13,29-23-25-24,0-1-2"
    },
    "ja3s": {
      "hash": "5d79edf64e03689ff559a54e9d9487bc",
      "string": "771,49199,65281-0-11-16-23"
    }
  }
}

As this is a bidir tx it has both ja3 (client) and ja3s (server). In the quic case we just need to make sure the fields are named properly, so a tx will have either ja3 or ja3s.

Can we also log both the hash and the string link in tls?

victorjulien · 2022-03-11T06:18:31Z

Can you add quic-events.rules? I wanted to see why I have so many errors:

app_layer.flow.quic                           | Total                     | 102201
app_layer.tx.quic                             | Total                     | 145479
app_layer.error.quic.parser                   | Total                     | 5277807

victorjulien · 2022-03-11T06:23:27Z

Btw eve has no anomaly records for these errors either. So we may not be setting events?

suricata-qa · 2022-03-11T10:05:02Z

WARNING:

field	test	baseline	%
	tlpr1_stats_chk
.app_layer.flow.quic	33523	39556	84.75%
.app_layer.tx.quic	106109	4198	2527.61%
.app_layer.error.quic.parser	54065	43951	123.01%

Pipeline 6538

victorjulien · 2022-03-11T11:33:22Z

WARNING:

field test baseline %
tlpr1_stats_chk
.app_layer.flow.quic 33523 39556 84.75%
.app_layer.tx.quic 106109 4198 2527.61%
.app_layer.error.quic.parser 54065 43951 123.01%
Pipeline 6538

Fewer quic flows... is that expected?

jasonish · 2022-03-14T23:13:02Z

rust/src/quic/frames.rs

+fn quic_get_tls_extensions(
+    input: Option<&[u8]>, ja3: &mut String, client: bool,
+) -> Vec<QuicTlsExtension> {
+    let mut extv = Vec::new();


How often, if ever would we endup not adding anything to this vector?

when parse_tls_extensions fails, that is on malformed messages...
What are you thinking of ?

catenacyber · 2022-03-16T13:22:43Z

Can we also log both the hash and the string link in tls?

Done in next version

I did not do the detection part, as it is only applying a to_md5 transform...

catenacyber · 2022-03-16T13:46:58Z

Can you add quic-events.rules?

Done

So we may not be setting events?

There were no events so far, added with the rules

victorjulien · 2022-03-16T15:18:56Z

Can we also log both the hash and the string link in tls?

Done in next version

I did not do the detection part, as it is only applying a to_md5 transform...

I'd like to reconsider the quic.ja3 keyword completely, and consider adding support for the existing ja3* keywords. These are not limited to tls by their name, so doing something like alert quic ... ja3.string; content:"abc"; seems to make most sense to me.

catenacyber · 2022-03-16T20:03:05Z

Fewer quic flows... is that expected?

I think not, I made a fix against some .app_layer.error.quic.parser let's see if we get more flows then

catenacyber · 2022-03-16T20:20:31Z

I'd like to reconsider the quic.ja3 keyword completely, and consider adding support for the existing ja3* keywords. These are not limited to tls by their name, so doing something like alert quic ... ja3.string; content:"abc"; seems to make most sense to me.

Done in next version

catenacyber · 2022-03-16T20:22:41Z

Replaced by #7144

We currently pin "time" to 0.3.20, but this version no longer compiles on nightly Rust, and will likely not compile on the next stable version of Rust. During configure, set the version of "time" based on the version of rustc found, 0.3.20 for Rust < 1.67, and the current version for Rust 1.67 and newer, which also builds on Rust nightly. Issue: OISF#7130

Issue: OISF#7130

Fixes build on rustc 1.80. Bumps the MSRV to 1.67.1. Bug: OISF#7130.

catenacyber added 14 commits March 6, 2022 13:59

quic: rustfmt

8dbe67e

quic: complete parsing of initial for non gquic

6d6f589

Ticket: 4967 The format of initial packet for quic ietf, ie quic v1, is described in rfc 9000, section 17.2.2

rdp: bump up tls-parser crate version

9c5f217

so that we can use new functions in quic parser

quic: parse more frames

6feb585

and logs interesting extensions from crypto frame

quic: use packet num length when decrypting

38c90ee

As it can be 4, but it can also be 1, based on the first decrypted byte

quic: do not try to parse encrypted data

a6f8a2b

The way to determine if the payload is encrypted is by storing in the state if we have seen a crypto frame in both directions...

quic: decrypt in its own function

2d77782

So as to keep parse not too big

quic: use sni from crypto frame with tls

5babeaa

for detection

quic: ja3 computation and logging and detection

556a131

Ticket: 5143

quic: function split in 2 and other style improvements

2fcce92

quic: move earlier the check against header.length

48ed785

quic: use Suricata's own quic_tls_extension_name

9e0d20b

quic: parse gquic version Q039

6bd8d80

Ticket: 5166

quic: check against right remaining buffer

190d858

catenacyber requested review from victorjulien, jasonish and a team as code owners March 10, 2022 12:48

This was referenced Mar 10, 2022

Quic ietf 4967 v7 #7122

Closed

Quic ietf 4967 v5 #7106

Closed

quic: detect ja3 from server with keyword quic.ja3s

71bc1fd

As the ja3 standard names it so, with an s, when it comes from the server to the client.

catenacyber force-pushed the quic-ietf-4967-v8 branch from 65a0dc0 to 71bc1fd Compare March 10, 2022 14:37

jasonish reviewed Mar 14, 2022

View reviewed changes

catenacyber mentioned this pull request Mar 16, 2022

Quic ietf 4967 v9 #7144

Closed

catenacyber closed this Mar 16, 2022

jasonish added a commit to jasonish/suricata that referenced this pull request Jul 2, 2024

rust/time: use branch of time fixed to build on latest rust nightly

9114d8d

Issue: OISF#7130

jasonish added a commit to jasonish/suricata that referenced this pull request Jul 2, 2024

rust/time: use branch of time fixed to build on latest rust nightly

44b4db5

Issue: OISF#7130

jasonish added a commit to jasonish/suricata that referenced this pull request Jul 2, 2024

rust/time: use branch of time fixed to build on latest rust nightly

7271014

Issue: OISF#7130

victorjulien added a commit to victorjulien/suricata that referenced this pull request Jul 27, 2024

rust: bump time to most recent

37801b4

Fixes build on rustc 1.80. Bumps the MSRV to 1.67.1. Bug: OISF#7130.

victorjulien added a commit to victorjulien/suricata that referenced this pull request Jul 28, 2024

rust: bump time to most recent

f873172

Fixes build on rustc 1.80. Bumps the MSRV to 1.67.1. Bug: OISF#7130.

victorjulien added a commit to victorjulien/suricata that referenced this pull request Jul 28, 2024

rust: bump time to most recent

f9d379c

Fixes build on rustc 1.80. Bumps the MSRV to 1.67.1. Bug: OISF#7130.

victorjulien added a commit to victorjulien/suricata that referenced this pull request Jul 29, 2024

rust: bump time to most recent

da1645b

Fixes build on rustc 1.80. Bumps the MSRV to 1.67.1. Bug: OISF#7130.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Quic ietf 4967 v8 #7130

Quic ietf 4967 v8 #7130

catenacyber commented Mar 10, 2022

codecov bot commented Mar 10, 2022

victorjulien commented Mar 11, 2022

victorjulien commented Mar 11, 2022

victorjulien commented Mar 11, 2022

suricata-qa commented Mar 11, 2022

victorjulien commented Mar 11, 2022

jasonish Mar 14, 2022

catenacyber Mar 16, 2022

catenacyber commented Mar 16, 2022

catenacyber commented Mar 16, 2022

victorjulien commented Mar 16, 2022

catenacyber commented Mar 16, 2022

catenacyber commented Mar 16, 2022

catenacyber commented Mar 16, 2022

Quic ietf 4967 v8 #7130

Quic ietf 4967 v8 #7130

Conversation

catenacyber commented Mar 10, 2022

codecov bot commented Mar 10, 2022

Codecov Report

victorjulien commented Mar 11, 2022

victorjulien commented Mar 11, 2022

victorjulien commented Mar 11, 2022

suricata-qa commented Mar 11, 2022

victorjulien commented Mar 11, 2022

jasonish Mar 14, 2022

Choose a reason for hiding this comment

catenacyber Mar 16, 2022

Choose a reason for hiding this comment

catenacyber commented Mar 16, 2022

catenacyber commented Mar 16, 2022

victorjulien commented Mar 16, 2022

catenacyber commented Mar 16, 2022

catenacyber commented Mar 16, 2022

catenacyber commented Mar 16, 2022