Tls fingerprint and storage support #1

regit · 2012-08-21T07:13:14Z

This branch is a proposal for TLS fingerprint computation and keyword (#443) as well as TLS store keyword (#444)

Creation of the log-tlslog file in order to log tls message. Need to add some information into suricata.yaml to work. - tls-log: enabled: yes # Log TLS connections. filename: tls.log # File to store TLS logs.

Adding util-crypt containing cryptographic functions as SHA1 and Base64.

Adding a pointer in ssl_state struct and compute fingerprint during certificate decoding.

Improve TLS logging by adding the certificate fingerprint to TLS Log file. Add the extending option to the tls-log entry in suricata.yaml.

Add the support for tls.fingerprint keyword in rules.

This patch adds a TLS store option to save certificate in PEM format. Each time the store action is met, a file and a metafile are created. Reworked-by: Eric Leblond <eric@regit.org>

When multiple certificates forming a chain are sent. A pointer to the start of each certificate is kept. This will allow treatment on certificates chains.

When using the tls.store command, a dump of all certificates in the chain is now done on the disk.

inliniac · 2012-08-21T09:29:38Z

src/app-layer-tls-handshake.c

                        DerFree(cert);
                        return -1;
                    }
                }
+                ncert = (SSLCertsChain *)SCMalloc(sizeof(SSLCertsChain));


Missing NULL check.

CID 1428797 (#1 of 1): Unchecked return value (CHECKED_RETURN) check_return: Calling HashTableAdd without checking return value (as is done elsewhere 5 out of 6 times).

Loading rules with iprep keywords cause memory leaks due to missing frees. Direct leak of 8 byte(s) in 4 object(s) allocated from: #0 0x7f81c862bd28 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc1d28) #1 0x7f81c6afea69 in pcre_get_substring (/lib/x86_64-linux-gnu/libpcre.so.3+0x27a69) #2 0x43206f7420676e68 (<unknown module>) SUMMARY: AddressSanitizer: 8 byte(s) leaked in 4 allocation(s).

Loading rules with iprep keyword cause memory leaks due to missing frees. Direct leak of 8 byte(s) in 4 object(s) allocated from: #0 0x7f81c862bd28 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc1d28) #1 0x7f81c6afea69 in pcre_get_substring (/lib/x86_64-linux-gnu/libpcre.so.3+0x27a69) #2 0x43206f7420676e68 (<unknown module>) SUMMARY: AddressSanitizer: 8 byte(s) leaked in 4 allocation(s).

This fixes two memleaks found with ASAN. Direct leak of 96 byte(s) in 1 object(s) allocated from: #0 0x7f59cf4a4d28 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc1d28) #1 0xd7f92f in ReceiveNFLOGThreadInit /home/glongo/suricata/src/source-nflog.c:221 #2 0xe9c8eb in TmThreadsSlotPktAcqLoop /home/glongo/suricata/src/tm-threads.c:293 #3 0x7f59cd7aa4a3 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x74a3) Indirect leak of 70000 byte(s) in 1 object(s) allocated from: #0 0x7f59cf4a4d28 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc1d28) #1 0xd814ea in ReceiveNFLOGThreadInit /home/glongo/suricata/src/source-nflog.c:324 #2 0xe9c8eb in TmThreadsSlotPktAcqLoop /home/glongo/suricata/src/tm-threads.c:293 #3 0x7f59cd7aa4a3 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x74a3) SUMMARY: AddressSanitizer: 70096 byte(s) leaked in 2 allocation(s).

Updated buffers for suri5 - pass OISF#1

Fixes runs with --enable-debug-validation. The target did not init a packet pool, so for a tunnel packet would try to get a packet from an uninitialized pool. In non-debug mode, this silently works by falling back to a packet from alloc. (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007ffff35a6801 in __GI_abort () at abort.c:79 #2 0x00007ffff359639a in __assert_fail_base (fmt=0x7ffff371d7d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x555557fe7260 "!(pool->initialized == 0)", file=file@entry=0x555557fe7220 "tmqh-packetpool.c", line=line@entry=253, function=function@entry=0x555557fe7500 <__PRETTY_FUNCTION__.21181> "PacketPoolGetPacket") at assert.c:92 #3 0x00007ffff3596412 in __GI___assert_fail (assertion=0x555557fe7260 "!(pool->initialized == 0)", file=0x555557fe7220 "tmqh-packetpool.c", line=253, function=0x555557fe7500 <__PRETTY_FUNCTION__.21181> "PacketPoolGetPacket") at assert.c:101 #4 0x00005555577e24be in PacketPoolGetPacket () at tmqh-packetpool.c:253 #5 0x0000555556914ecd in PacketGetFromQueueOrAlloc () at decode.c:183 #6 0x00005555569161e1 in PacketTunnelPktSetup (tv=0x555559863980 <tv>, dtv=0x614000068e40, parent=0x61e0000fc080, pkt=0x61e0000fc470 "LL", len=72, proto=DECODE_TUNNEL_IPV4) at decode.c:286 #7 0x00005555569de694 in DecodeIPv4inIPv6 (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc470 "LL", plen=72) at decode-ipv6.c:59 #8 0x00005555569e60b5 in DecodeIPV6ExtHdrs (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc470 "LL", len=112) at decode-ipv6.c:522 #9 0x00005555569e846f in DecodeIPV6 (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc420 "cLL", len=255) at decode-ipv6.c:641 #10 0x0000555556a032f9 in DecodeRaw (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc420 "cLL", len=255) at decode-raw.c:70 #11 0x0000555557659ba8 in DecodePcapFile (tv=0x555559863980 <tv>, p=0x61e0000fc080, data=0x614000068e40) at source-pcap-file.c:412 #12 0x0000555556573401 in LLVMFuzzerTestOneInput (data=0x613000000047 "\241\262\315\064", size=339) at tests/fuzz/fuzz_sigpcap.c:158 #13 0x0000555557a4dc66 in main (argc=2, argv=0x7fffffffdfa8) at tests/fuzz/onefile.c:51 That line: BUG_ON(pool->initialized == 0);

In case of bad IPv4, TCP or UDP, the per packet ip4vars/tcpvars/udpvar structures would not be cleaned up because the cleanup depends on the 'header' pointer being set, but the error handling would unset that. This could mean these structures were already filled with values before the error was detected. As packets were recycled, the next packet decoding would use this unclean structure. To make things worse these structures are part of unions. IPv4/IPv6 and TCP/ICMPv4/ICMPv6 share the same memory location. LibFuzzer+UBSAN found this both locally and in Oss-Fuzz: decode-ipv6.c:654:9: runtime error: load of value 6, which is not a valid value for type 'bool' #0 0x6146f0 in DecodeIPV6 /src/suricata/src/decode-ipv6.c:654:9 #1 0x617e96 in DecodeNull /src/suricata/src/decode-null.c:70:13 #2 0x9dd8a4 in DecodePcapFile /src/suricata/src/source-pcap-file.c:412:9 #3 0x4c8ed2 in LLVMFuzzerTestOneInput /src/suricata/src/tests/fuzz/fuzz_sigpcap.c:158:25 #4 0x457e51 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:556:15 #5 0x457575 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:470:3 #6 0x459917 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:698:19 #7 0x45a6a5 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:830:5 #8 0x448728 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:824:6 #9 0x472552 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:19:10 #10 0x7ff0d097b82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f) #11 0x41bde8 in _start (/out/fuzz_sigpcap+0x41bde8) Bug: OISF#3610

In case of bad IPv4, TCP or UDP, the per packet ip4vars/tcpvars/udpvar structures would not be cleaned up because the cleanup depends on the 'header' pointer being set, but the error handling would unset that. This could mean these structures were already filled with values before the error was detected. As packets were recycled, the next packet decoding would use this unclean structure. To make things worse these structures are part of unions. IPv4/IPv6 and TCP/ICMPv4/ICMPv6 share the same memory location. LibFuzzer+UBSAN found this both locally and in Oss-Fuzz: decode-ipv6.c:654:9: runtime error: load of value 6, which is not a valid value for type 'bool' #0 0x6146f0 in DecodeIPV6 /src/suricata/src/decode-ipv6.c:654:9 #1 0x617e96 in DecodeNull /src/suricata/src/decode-null.c:70:13 #2 0x9dd8a4 in DecodePcapFile /src/suricata/src/source-pcap-file.c:412:9 #3 0x4c8ed2 in LLVMFuzzerTestOneInput /src/suricata/src/tests/fuzz/fuzz_sigpcap.c:158:25 #4 0x457e51 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:556:15 #5 0x457575 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:470:3 #6 0x459917 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:698:19 #7 0x45a6a5 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:830:5 #8 0x448728 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:824:6 #9 0x472552 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:19:10 #10 0x7ff0d097b82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f) #11 0x41bde8 in _start (/out/fuzz_sigpcap+0x41bde8) Bug: OISF#3496

Make sure to first close all ports before freeing device mempools. Thread 1 "Suricata-Main" received signal SIGSEGV, Segmentation fault. 0x00007ffff456a3fb in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so (gdb) bt #0 0x00007ffff456a3fb in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so #1 0x00007ffff469a948 in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so #2 0x00007ffff45606aa in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so #3 0x00007ffff6d4ed8d in rte_eth_dev_close () from /usr/lib/x86_64-linux-gnu/librte_ethdev.so.20.0 #4 0x000000000055fc4c in DPDKCloseDevice (ldev=ldev@entry=0xe3a400) at util-dpdk.c:53 #5 0x000000000055f4eb in LiveDeviceListClean () at util-device.c:331 #6 0x00000000005511c8 in GlobalsDestroy (suri=<optimized out>) at suricata.c:381 #7 0x0000000000550a76 in SuricataMain (argc=<optimized out>, argv=<optimized out>) at suricata.c:3059 #8 0x00007ffff6a24083 in __libc_start_main (main=0x54cca0 <main>, argc=8, argv=0x7fffffffe4c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe4b8) at ../csu/libc-start.c:308 #9 0x000000000054cbde in _start () Bug: #5619.

Make sure to first close all ports before freeing device mempools. Thread 1 "Suricata-Main" received signal SIGSEGV, Segmentation fault. 0x00007ffff456a3fb in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so (gdb) bt #0 0x00007ffff456a3fb in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so OISF#1 0x00007ffff469a948 in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so OISF#2 0x00007ffff45606aa in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so OISF#3 0x00007ffff6d4ed8d in rte_eth_dev_close () from /usr/lib/x86_64-linux-gnu/librte_ethdev.so.20.0 OISF#4 0x000000000055fc4c in DPDKCloseDevice (ldev=ldev@entry=0xe3a400) at util-dpdk.c:53 OISF#5 0x000000000055f4eb in LiveDeviceListClean () at util-device.c:331 OISF#6 0x00000000005511c8 in GlobalsDestroy (suri=<optimized out>) at suricata.c:381 OISF#7 0x0000000000550a76 in SuricataMain (argc=<optimized out>, argv=<optimized out>) at suricata.c:3059 OISF#8 0x00007ffff6a24083 in __libc_start_main (main=0x54cca0 <main>, argc=8, argv=0x7fffffffe4c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe4b8) at ../csu/libc-start.c:308 OISF#9 0x000000000054cbde in _start () Bug: OISF#5619.

Popof and others added 10 commits August 21, 2012 09:06

tls: adding TLS Log support

520daf1

Creation of the log-tlslog file in order to log tls message. Need to add some information into suricata.yaml to work. - tls-log: enabled: yes # Log TLS connections. filename: tls.log # File to store TLS logs.

tls: adding cryptographic functions.

0491266

Adding util-crypt containing cryptographic functions as SHA1 and Base64.

tls: add NSS version for SHA1 computing function.

e8d3e61

tls: adding fingerprint calculation.

e810a9e

Adding a pointer in ssl_state struct and compute fingerprint during certificate decoding.

tls: adding fingerprint to TLS Log information.

a6a4683

Improve TLS logging by adding the certificate fingerprint to TLS Log file. Add the extending option to the tls-log entry in suricata.yaml.

tls: adding support for fingerprint rule matching.

e97bc8d

Add the support for tls.fingerprint keyword in rules.

tls: adding store option for TLS

6077ff5

This patch adds a TLS store option to save certificate in PEM format. Each time the store action is met, a file and a metafile are created. Reworked-by: Eric Leblond <eric@regit.org>

tls: keep pointers to all certificates in chain

103d85b

When multiple certificates forming a chain are sent. A pointer to the start of each certificate is kept. This will allow treatment on certificates chains.

tls: store all the certificates chain in the written PEM file.

c02f7a3

When using the tls.store command, a dump of all certificates in the chain is now done on the disk.

tls: fix error message.

29b4268

inliniac reviewed Aug 21, 2012
View reviewed changes

victorjulien referenced this pull request in victorjulien/suricata Jul 18, 2018

mpm/hs: fix minor coverity warning

1863914

CID 1428797 (#1 of 1): Unchecked return value (CHECKED_RETURN) check_return: Calling HashTableAdd without checking return value (as is done elsewhere 5 out of 6 times).

This was referenced Aug 1, 2019

signature: avoids overflow from VariableNameHash #4083

Closed

signature: avoids overflow from VariableNameHash #4084

Closed

hadojae added a commit to hadojae/suricata that referenced this pull request Nov 25, 2019

Update http-keywords.rst

0176eec

Updated buffers for suri5 - pass OISF#1

pjyoung1 mentioned this pull request Jan 9, 2020

Napatech bypass v1.09 #4478

Closed

3 tasks

This was referenced Aug 13, 2020

runmodes: patch identified leak on runmode single #5293

Closed

runmodes: patch identified leak on runmode single #5294

Closed

runmodes: patch asan-identified leak on runmode single #5303

Closed

worldpeace365 mentioned this pull request Jul 27, 2021

Hyperscan MPM integration v6 #1965

Closed

nowaits mentioned this pull request May 14, 2022

stmp: missing node name for default scheme 'http' #7411

Closed

victorjulien mentioned this pull request Aug 3, 2023

dpdk/mlx5: fix shutdown crash in IPS mode #9341

Closed

jasonish mentioned this pull request Jan 24, 2024

detect/requires: reset sigerror flags for each rule #10237

Merged

catenacyber mentioned this pull request Aug 27, 2024

detect/dataset: delay set operation after signature full match #11662

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tls fingerprint and storage support #1

Tls fingerprint and storage support #1

regit commented Aug 21, 2012

inliniac Aug 21, 2012

regit Aug 21, 2012

Tls fingerprint and storage support #1

Tls fingerprint and storage support #1

Conversation

regit commented Aug 21, 2012

inliniac Aug 21, 2012

Choose a reason for hiding this comment

regit Aug 21, 2012

Choose a reason for hiding this comment