[ASM] Added 1 hour timer to vulnerability deduplication #4411

NachoEchevarria · 2023-07-18T08:24:02Z

Summary of changes

Vulnerability deduplication stores the detected vulnerabilities and reports them only once per execution. A 1 hour timer is required to be set in order to clear the deduplication cache periodically. That will mean that every hour, repeated vulnerabilities might be sent. A single timer will be used for the whole cache. The individual vulnerabilities have no timer associated.

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging · 2023-07-18T08:50:12Z

Datadog Report

Branch report: nacho/deduplicationTimeout
Commit report: 66b9961

✅ dd-trace-dotnet: 0 Failed, 0 New Flaky, 291699 Passed, 1091 Skipped, 27m 33.59s Wall Time

tracer/src/Datadog.Trace/Iast/HashBasedDeduplication.cs

daniel-romano-DD

LGTM with a NIT

Co-authored-by: Daniel Romano <108014683+daniel-romano-DD@users.noreply.github.com>

andrewlock · 2023-07-18T14:53:46Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4411) - mean (2,986ms)  : 2888, 3084
     .   : milestone, 2986,
    master - mean (2,998ms)  : 2926, 3070
     .   : milestone, 2998,

    section CallTarget+Inlining+NGEN
    This PR (4411) - mean (3,831ms)  : 3731, 3930
     .   : milestone, 3831,
    master - mean (3,817ms)  : 3735, 3899
     .   : milestone, 3817,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4411) - mean (3,113ms)  : 3044, 3181
     .   : milestone, 3113,
    master - mean (3,118ms)  : 3028, 3207
     .   : milestone, 3118,

    section CallTarget+Inlining+NGEN
    This PR (4411) - mean (3,617ms)  : 3552, 3681
     .   : milestone, 3617,
    master - mean (3,614ms)  : 3548, 3680
     .   : milestone, 3614,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4411) - mean (3,098ms)  : 3029, 3168
     .   : milestone, 3098,
    master - mean (3,113ms)  : 3027, 3199
     .   : milestone, 3113,

    section CallTarget+Inlining+NGEN
    This PR (4411) - mean (3,576ms)  : 3510, 3642
     .   : milestone, 3576,
    master - mean (3,571ms)  : 3515, 3627
     .   : milestone, 3571,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4411) - mean (192ms)  : 186, 197
     .   : milestone, 192,
    master - mean (191ms)  : 187, 195
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (4411) - mean (1,129ms)  : 1096, 1162
     .   : milestone, 1129,
    master - mean (1,123ms)  : 1089, 1157
     .   : milestone, 1123,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4411) - mean (375ms)  : 369, 382
     .   : milestone, 375,
    master - mean (375ms)  : 368, 381
     .   : milestone, 375,

    section CallTarget+Inlining+NGEN
    This PR (4411) - mean (1,179ms)  : 1158, 1199
     .   : milestone, 1179,
    master - mean (1,185ms)  : 1159, 1211
     .   : milestone, 1185,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4411) - mean (361ms)  : 356, 367
     .   : milestone, 361,
    master - mean (365ms)  : 357, 372
     .   : milestone, 365,

    section CallTarget+Inlining+NGEN
    This PR (4411) - mean (1,139ms)  : 1111, 1166
     .   : milestone, 1139,
    master - mean (1,146ms)  : 1117, 1175
     .   : milestone, 1146,

andrewlock · 2023-07-18T14:57:28Z

Benchmarks Report 🐌

Benchmarks for #4411 compared to master:

1 benchmarks are faster, with geometric mean 1.141
4 benchmarks are slower, with geometric mean 1.174
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	488μs	176ns	659ns	0	2.62 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	648μs	238ns	890ns	0	2.63 KB
master	`WriteAndFlushEnrichedTraces`	net472	792μs	224ns	836ns	0.393	3.22 KB
#4411	`WriteAndFlushEnrichedTraces`	net6.0	480μs	148ns	534ns	0	2.62 KB
#4411	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	640μs	400ns	1.55μs	0	2.63 KB
#4411	`WriteAndFlushEnrichedTraces`	net472	804μs	543ns	2.1μs	0.403	3.22 KB

Benchmarks.Trace.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4411

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net472	1.292	144.11	186.19

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	38.8μs	12.3ns	46.1ns	0.0197	0	1.65 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	40.7μs	36.3ns	136ns	0.0202	0	1.63 KB
master	`AllCycleSimpleBody`	net472	41.8μs	12.6ns	47.1ns	0.249	0	1.69 KB
master	`AllCycleMoreComplexBody`	net6.0	222μs	85.4ns	331ns	0.11	0	9.22 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	229μs	362ns	1.35μs	0.113	0	9.12 KB
master	`AllCycleMoreComplexBody`	net472	240μs	86.3ns	323ns	1.43	0	9.28 KB
master	`ObjectExtractorSimpleBody`	net6.0	121ns	0.0413ns	0.149ns	0.00393	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	187ns	0.0875ns	0.327ns	0.00374	0	272 B
master	`ObjectExtractorSimpleBody`	net472	144ns	0.137ns	0.53ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	2.97μs	1.21ns	4.38ns	0.054	0	3.88 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.03μs	1ns	3.75ns	0.0503	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	net472	4.07μs	2.22ns	8.61ns	0.618	0.0061	3.89 KB
#4411	`AllCycleSimpleBody`	net6.0	38.8μs	13.7ns	53ns	0.0193	0	1.65 KB
#4411	`AllCycleSimpleBody`	netcoreapp3.1	40.3μs	63.9ns	221ns	0.0203	0	1.63 KB
#4411	`AllCycleSimpleBody`	net472	41.5μs	85.4ns	331ns	0.25	0	1.69 KB
#4411	`AllCycleMoreComplexBody`	net6.0	219μs	45.9ns	172ns	0.11	0	9.22 KB
#4411	`AllCycleMoreComplexBody`	netcoreapp3.1	229μs	73.8ns	411ns	0.114	0	9.12 KB
#4411	`AllCycleMoreComplexBody`	net472	241μs	637ns	2.38μs	1.45	0	9.28 KB
#4411	`ObjectExtractorSimpleBody`	net6.0	116ns	0.0379ns	0.131ns	0.00396	0	280 B
#4411	`ObjectExtractorSimpleBody`	netcoreapp3.1	180ns	0.257ns	0.994ns	0.0037	0	272 B
#4411	`ObjectExtractorSimpleBody`	net472	185ns	0.817ns	3.16ns	0.0446	0	281 B
#4411	`ObjectExtractorMoreComplexBody`	net6.0	2.95μs	1.15ns	4.31ns	0.0547	0	3.88 KB
#4411	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.99μs	1.66ns	5.99ns	0.0501	0	3.78 KB
#4411	`ObjectExtractorMoreComplexBody`	net472	4.03μs	2.38ns	9.22ns	0.617	0.00605	3.89 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	166μs	311ns	1.2μs	0.251	18.08 KB
master	`SendRequest`	netcoreapp3.1	190μs	236ns	816ns	0.191	20.14 KB
master	`SendRequest`	net472	9.93E‑05ns	7.22E‑05ns	0.00026ns	0	0 b
#4411	`SendRequest`	net6.0	168μs	73ns	273ns	0.25	18.08 KB
#4411	`SendRequest`	netcoreapp3.1	189μs	190ns	735ns	0.187	20.15 KB
#4411	`SendRequest`	net472	0.00173ns	0.000494ns	0.00191ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #4411

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.58 KB	41.97 KB	397 B	0.95%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	513μs	472ns	1.83μs	0.508	0	0	41.58 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	628μs	885ns	3.43μs	0.316	0	0	41.85 KB
master	`WriteAndFlushEnrichedTraces`	net472	769μs	3.25μs	12.6μs	8.1	2.31	0.386	53.21 KB
#4411	`WriteAndFlushEnrichedTraces`	net6.0	533μs	638ns	2.47μs	0.532	0	0	41.97 KB
#4411	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	617μs	1.28μs	4.96μs	0.302	0	0	41.85 KB
#4411	`WriteAndFlushEnrichedTraces`	net472	777μs	2.91μs	11.3μs	8.1	2.31	0.386	53.2 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	901ns	0.265ns	1.03ns	0.0108	0	768 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.2μs	0.328ns	1.14ns	0.0103	0	768 B
master	`ExecuteNonQuery`	net472	1.5μs	0.611ns	2.37ns	0.116	0	730 B
#4411	`ExecuteNonQuery`	net6.0	919ns	0.249ns	0.93ns	0.0108	0	768 B
#4411	`ExecuteNonQuery`	netcoreapp3.1	1.11μs	0.599ns	2.24ns	0.0104	0	768 B
#4411	`ExecuteNonQuery`	net472	1.5μs	0.397ns	1.49ns	0.116	0.000753	730 B

Benchmarks.Trace.ElasticsearchBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4411

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑netcoreapp3.1	1.114	1,305.41	1,453.82

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`CallElasticsearch`	net6.0	1.11μs	0.445ns	1.67ns	0.014	0	992 B
master	`CallElasticsearch`	netcoreapp3.1	1.23μs	0.443ns	1.66ns	0.0136	0	992 B
master	`CallElasticsearch`	net472	2.16μs	0.644ns	2.5ns	0.159	0.00108	1 KB
master	`CallElasticsearchAsync`	net6.0	1.07μs	0.436ns	1.63ns	0.0134	0	968 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.31μs	0.384ns	1.44ns	0.0137	0	1.04 KB
master	`CallElasticsearchAsync`	net472	2.27μs	0.797ns	2.98ns	0.167	0.00113	1.06 KB
#4411	`CallElasticsearch`	net6.0	1.03μs	0.637ns	2.38ns	0.014	0	992 B
#4411	`CallElasticsearch`	netcoreapp3.1	1.3μs	3.1ns	12ns	0.0133	0	992 B
#4411	`CallElasticsearch`	net472	2.11μs	0.488ns	1.89ns	0.159	0	1 KB
#4411	`CallElasticsearchAsync`	net6.0	1.04μs	0.277ns	0.999ns	0.0135	0	968 B
#4411	`CallElasticsearchAsync`	netcoreapp3.1	1.45μs	1.45ns	5.61ns	0.0139	0	1.04 KB
#4411	`CallElasticsearchAsync`	net472	2.17μs	0.501ns	1.94ns	0.168	0.00107	1.06 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.07μs	0.407ns	1.47ns	0.0128	912 B
master	`ExecuteAsync`	netcoreapp3.1	1.28μs	0.568ns	2.12ns	0.0121	912 B
master	`ExecuteAsync`	net472	1.55μs	0.384ns	1.49ns	0.139	875 B
#4411	`ExecuteAsync`	net6.0	1.14μs	0.413ns	1.6ns	0.0126	912 B
#4411	`ExecuteAsync`	netcoreapp3.1	1.35μs	0.624ns	2.34ns	0.0122	912 B
#4411	`ExecuteAsync`	net472	1.56μs	1.23ns	4.59ns	0.139	875 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	3.54μs	1.05ns	4.07ns	0.0266	1.94 KB
master	`SendAsync`	netcoreapp3.1	4.34μs	1.38ns	4.98ns	0.0324	2.48 KB
master	`SendAsync`	net472	6.88μs	3.71ns	13.9ns	0.482	3.05 KB
#4411	`SendAsync`	net6.0	3.67μs	1.03ns	3.99ns	0.0258	1.94 KB
#4411	`SendAsync`	netcoreapp3.1	4.34μs	0.936ns	3.24ns	0.0325	2.48 KB
#4411	`SendAsync`	net472	6.86μs	2.2ns	8.51ns	0.484	3.05 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.27μs	0.512ns	1.92ns	0.0227	1.62 KB
master	`EnrichedLog`	netcoreapp3.1	2.01μs	1.78ns	6.9ns	0.0222	1.62 KB
master	`EnrichedLog`	net472	2.33μs	2.13ns	8.26ns	0.245	1.54 KB
#4411	`EnrichedLog`	net6.0	1.24μs	0.698ns	2.61ns	0.023	1.62 KB
#4411	`EnrichedLog`	netcoreapp3.1	2.04μs	2.37ns	8.86ns	0.0222	1.62 KB
#4411	`EnrichedLog`	net472	2.2μs	1.19ns	4.28ns	0.244	1.54 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	110μs	82.9ns	321ns	0.0554	0	4.21 KB
master	`EnrichedLog`	netcoreapp3.1	116μs	135ns	522ns	0.0583	0	4.21 KB
master	`EnrichedLog`	net472	145μs	146ns	566ns	0.651	0.217	4.38 KB
#4411	`EnrichedLog`	net6.0	110μs	69.2ns	259ns	0.0548	0	4.21 KB
#4411	`EnrichedLog`	netcoreapp3.1	116μs	131ns	508ns	0	0	4.21 KB
#4411	`EnrichedLog`	net472	144μs	45.2ns	163ns	0.649	0.216	4.38 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.85μs	1.42ns	5.52ns	0.0301	2.18 KB
master	`EnrichedLog`	netcoreapp3.1	3.98μs	2.06ns	7.73ns	0.0284	2.18 KB
master	`EnrichedLog`	net472	4.47μs	1.3ns	4.7ns	0.314	1.99 KB
#4411	`EnrichedLog`	net6.0	2.84μs	0.39ns	1.46ns	0.0295	2.18 KB
#4411	`EnrichedLog`	netcoreapp3.1	3.88μs	2.39ns	8.62ns	0.0295	2.18 KB
#4411	`EnrichedLog`	net472	4.39μs	1.95ns	7.56ns	0.316	1.99 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net6.0	1.17μs	0.872ns	3.38ns	0.0163	0	1.16 KB
master	`SendReceive`	netcoreapp3.1	1.44μs	0.517ns	1.79ns	0.0162	0	1.16 KB
master	`SendReceive`	net472	1.85μs	0.79ns	2.74ns	0.185	0	1.16 KB
#4411	`SendReceive`	net6.0	1.2μs	1.12ns	4.18ns	0.016	0	1.16 KB
#4411	`SendReceive`	netcoreapp3.1	1.43μs	0.558ns	2.09ns	0.0157	0	1.16 KB
#4411	`SendReceive`	net472	1.77μs	2ns	7.73ns	0.184	0.000895	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.55μs	1.09ns	4.07ns	0.0216	1.53 KB
master	`EnrichedLog`	netcoreapp3.1	3.67μs	2.97ns	11.5ns	0.0198	1.58 KB
master	`EnrichedLog`	net472	4.04μs	2.15ns	8.04ns	0.311	1.96 KB
#4411	`EnrichedLog`	net6.0	2.42μs	0.467ns	1.75ns	0.021	1.53 KB
#4411	`EnrichedLog`	netcoreapp3.1	3.55μs	1.66ns	6.21ns	0.0213	1.58 KB
#4411	`EnrichedLog`	net472	3.99μs	1.61ns	6.25ns	0.31	1.96 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4411

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.182	370.39	437.86

Faster 🎉 in #4411

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472	1.141	823.16	721.18

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	370ns	0.128ns	0.48ns	0.00754	536 B
master	`StartFinishSpan`	netcoreapp3.1	539ns	0.339ns	1.17ns	0.00722	536 B
master	`StartFinishSpan`	net472	663ns	0.127ns	0.493ns	0.0854	538 B
master	`StartFinishScope`	net6.0	537ns	0.206ns	0.771ns	0.00909	656 B
master	`StartFinishScope`	netcoreapp3.1	722ns	0.186ns	0.696ns	0.00868	656 B
master	`StartFinishScope`	net472	823ns	0.209ns	0.809ns	0.0978	618 B
#4411	`StartFinishSpan`	net6.0	438ns	0.141ns	0.529ns	0.00741	536 B
#4411	`StartFinishSpan`	netcoreapp3.1	574ns	0.257ns	0.892ns	0.00724	536 B
#4411	`StartFinishSpan`	net472	644ns	0.212ns	0.735ns	0.0854	538 B
#4411	`StartFinishScope`	net6.0	511ns	0.0907ns	0.327ns	0.00917	656 B
#4411	`StartFinishScope`	netcoreapp3.1	692ns	0.395ns	1.48ns	0.00903	656 B
#4411	`StartFinishScope`	net472	722ns	0.588ns	2.28ns	0.098	618 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4411

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.117	522.91	584.05

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	523ns	0.148ns	0.555ns	0.00913	656 B
master	`RunOnMethodBegin`	netcoreapp3.1	758ns	0.334ns	1.25ns	0.00909	656 B
master	`RunOnMethodBegin`	net472	934ns	0.216ns	0.779ns	0.0979	618 B
#4411	`RunOnMethodBegin`	net6.0	584ns	0.234ns	0.876ns	0.00936	656 B
#4411	`RunOnMethodBegin`	netcoreapp3.1	749ns	1.42ns	5.12ns	0.00904	656 B
#4411	`RunOnMethodBegin`	net472	963ns	0.367ns	1.42ns	0.0979	618 B

andrewlock · 2023-07-18T15:29:07Z

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4411) (11.410M)   : 0, 11410333
    master (11.146M)   : 0, 11146357
    benchmarks/2.33.0 (10.950M)   : 0, 10949608
    benchmarks/2.9.0 (11.142M)   : 0, 11141669

    section Automatic
    This PR (4411) (7.897M)   : 0, 7897414
    master (7.839M)   : 0, 7839183
    benchmarks/2.33.0 (7.621M)   : 0, 7620707
    benchmarks/2.9.0 (7.959M)   : 0, 7958947

    section Trace stats
    master (7.839M)   : 0, 7838681
    benchmarks/2.33.0 (7.602M)   : 0, 7602038

    section Manual
    This PR (4411) (10.072M)   : 0, 10072121
    master (10.041M)   : 0, 10040618
    benchmarks/2.33.0 (9.770M)   : 0, 9770235

    section Manual + Automatic
    This PR (4411) (7.529M)   : 0, 7528889
    master (7.524M)   : 0, 7524092
    benchmarks/2.33.0 (7.189M)   : 0, 7188658

    section Version Conflict
    master (6.689M)   : 0, 6689193
    benchmarks/2.33.0 (6.553M)   : 0, 6553316

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4411) (9.618M)   : 0, 9617614
    master (9.820M)   : 0, 9819765
    benchmarks/2.33.0 (9.426M)   : 0, 9425603
    benchmarks/2.9.0 (9.842M)   : 0, 9842216

    section Automatic
    This PR (4411) (6.728M)   : 0, 6728466
    master (6.584M)   : 0, 6583758
    benchmarks/2.33.0 (6.803M)   : 0, 6802932

    section Trace stats
    master (6.837M)   : 0, 6837403
    benchmarks/2.33.0 (6.810M)   : 0, 6810447

    section Manual
    This PR (4411) (8.597M)   : 0, 8596619
    master (8.483M)   : 0, 8483262
    benchmarks/2.33.0 (8.435M)   : 0, 8434569

    section Manual + Automatic
    This PR (4411) (6.581M)   : 0, 6581038
    master (6.460M)   : 0, 6460201
    benchmarks/2.33.0 (6.465M)   : 0, 6465149

    section Version Conflict
    master (5.942M)   : 0, 5941596
    benchmarks/2.33.0 (5.807M)   : 0, 5806690

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4411) (9.759M)   : 0, 9758680
    master (10.170M)   : 0, 10169653
    benchmarks/2.33.0 (9.426M)   : 0, 9426080
    benchmarks/2.9.0 (10.688M)   : 0, 10687842

    section Automatic
    This PR (4411) (6.990M)   : 0, 6990083
    master (7.216M)   : 0, 7216200
    benchmarks/2.33.0 (6.469M)   : 0, 6468939
    benchmarks/2.9.0 (7.829M)   : 0, 7828890

    section Trace stats
    master (7.287M)   : 0, 7287082
    benchmarks/2.33.0 (6.830M)   : 0, 6830314

    section Manual
    This PR (4411) (8.762M)   : 0, 8761857
    master (9.190M)   : 0, 9189578
    benchmarks/2.33.0 (8.108M)   : 0, 8108026

    section Manual + Automatic
    This PR (4411) (6.849M)   : 0, 6848899
    master (6.995M)   : 0, 6994675
    benchmarks/2.33.0 (6.856M)   : 0, 6856379

    section Version Conflict
    master (6.398M)   : 0, 6397504
    benchmarks/2.33.0 (5.893M)   : 0, 5893016

gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4411) (7.565M)   : 0, 7565156
    master (7.563M)   : 0, 7563381
    benchmarks/2.33.0 (7.765M)   : 0, 7764607
    benchmarks/2.9.0 (7.769M)   : 0, 7768524

    section No attack
    This PR (4411) (2.122M)   : 0, 2121610
    master (2.137M)   : 0, 2137122
    benchmarks/2.33.0 (2.209M)   : 0, 2208725
    benchmarks/2.9.0 (3.174M)   : 0, 3174364

    section Attack
    This PR (4411) (1.821M)   : 0, 1820786
    master (1.861M)   : 0, 1860845
    benchmarks/2.33.0 (1.911M)   : 0, 1910909
    benchmarks/2.9.0 (2.518M)   : 0, 2517764

    section Blocking
    This PR (4411) (3.670M)   : 0, 3669642
    master (3.675M)   : 0, 3675283
    benchmarks/2.33.0 (3.861M)   : 0, 3860900

NachoEchevarria · 2023-07-19T09:06:31Z

Thank you for your feedback and reviews!

andrewlock

I think you need to UtcNow

andrewlock · 2023-07-19T09:11:33Z

tracer/src/Datadog.Trace/Iast/HashBasedDeduplication.cs

    {
+        _cacheClearedTime = currentTime ?? DateTime.Now;


I think this should use DateTime.UtcNow, otherwise you could run into time-zone issues with daylight savings etc 🤔

Sorry, I did not notice this comment before merging. I guess that, potentially, a time correction for daylight savings could occur and affect the calculations. I will correct that. Thank you!

andrewlock · 2023-07-19T09:11:55Z

tracer/src/Datadog.Trace/Iast/HashBasedDeduplication.cs

    {
        var hashCode = vulnerability.GetHashCode();
+        var currentTime = addTime ?? DateTime.Now;


Same, I think this should use DateTime.UtcNow

* Add 1hr timer to deduplication * Update tracer/src/Datadog.Trace/Iast/HashBasedDeduplication.cs Co-authored-by: Daniel Romano <108014683+daniel-romano-DD@users.noreply.github.com> * Remove empty space --------- Co-authored-by: Daniel Romano <108014683+daniel-romano-DD@users.noreply.github.com>

Add 1hr timer to deduplication

88300bb

This comment has been minimized.

Sign in to view

NachoEchevarria changed the title ~~Add 1hr timer to deduplication~~ [ASM] Added 1 hour timer to vulnerability deduplication Jul 18, 2023

This comment has been minimized.

Sign in to view

NachoEchevarria added area:asm area:asm-iast labels Jul 18, 2023

This comment has been minimized.

Sign in to view

NachoEchevarria marked this pull request as ready for review July 18, 2023 10:34

NachoEchevarria requested a review from a team as a code owner July 18, 2023 10:34

daniel-romano-DD reviewed Jul 18, 2023

View reviewed changes

tracer/src/Datadog.Trace/Iast/HashBasedDeduplication.cs Show resolved Hide resolved

daniel-romano-DD approved these changes Jul 18, 2023

View reviewed changes

robertpi approved these changes Jul 18, 2023

View reviewed changes

NachoEchevarria and others added 3 commits July 18, 2023 14:29

Update tracer/src/Datadog.Trace/Iast/HashBasedDeduplication.cs

a9a93f4

Co-authored-by: Daniel Romano <108014683+daniel-romano-DD@users.noreply.github.com>

Merge branch 'master' into nacho/deduplicationTimeout

2863111

Remove empty space

66b9961

NachoEchevarria merged commit 55d50ce into master Jul 19, 2023

NachoEchevarria deleted the nacho/deduplicationTimeout branch July 19, 2023 09:06

github-actions bot added this to the vNext milestone Jul 19, 2023

andrewlock reviewed Jul 19, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ASM] Added 1 hour timer to vulnerability deduplication #4411

[ASM] Added 1 hour timer to vulnerability deduplication #4411

NachoEchevarria commented Jul 18, 2023 •

edited

Loading

datadog-ddstaging bot commented Jul 18, 2023 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

daniel-romano-DD left a comment

andrewlock commented Jul 18, 2023

andrewlock commented Jul 18, 2023

Raw results

Slower ⚠️ in #4411

Raw results

Raw results

More allocations ⚠️ in #4411

Raw results

Raw results

Slower ⚠️ in #4411

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #4411

Faster 🎉 in #4411

Raw results

Slower ⚠️ in #4411

Raw results

andrewlock commented Jul 18, 2023

NachoEchevarria commented Jul 19, 2023

andrewlock left a comment

andrewlock Jul 19, 2023

NachoEchevarria Jul 19, 2023

andrewlock Jul 19, 2023

[ASM] Added 1 hour timer to vulnerability deduplication #4411

[ASM] Added 1 hour timer to vulnerability deduplication #4411

Conversation

NachoEchevarria commented Jul 18, 2023 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Jul 18, 2023 • edited Loading

Datadog Report

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

daniel-romano-DD left a comment

Choose a reason for hiding this comment

andrewlock commented Jul 18, 2023

Execution-Time Benchmarks Report ⏱️

andrewlock commented Jul 18, 2023

Benchmarks Report 🐌

Benchmark details

Raw results

Slower ⚠️ in #4411

Raw results

Raw results

More allocations ⚠️ in #4411

Raw results

Raw results

Slower ⚠️ in #4411

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #4411

Faster 🎉 in #4411

Raw results

Slower ⚠️ in #4411

Raw results

andrewlock commented Jul 18, 2023

Throughput/Crank Report:zap:

NachoEchevarria commented Jul 19, 2023

andrewlock left a comment

Choose a reason for hiding this comment

andrewlock Jul 19, 2023

Choose a reason for hiding this comment

NachoEchevarria Jul 19, 2023

Choose a reason for hiding this comment

andrewlock Jul 19, 2023

Choose a reason for hiding this comment

NachoEchevarria commented Jul 18, 2023 •

edited

Loading

datadog-ddstaging bot commented Jul 18, 2023 •

edited

Loading