[ASM] Add processors and scanners to ruleset #5248

anna-git · 2024-02-28T20:05:05Z

Summary of changes

Add processors and scanners to asm_dd payload.

Reason for change

We don't have them, they're necessary to api security when we deserialize from rmote config.

Implementation details

Test coverage

There's already tests for api security and tests for rcm and ruleset taken into account. I dont think another test for rcm and api security is necessary.

Other details

datadog-ddstaging · 2024-02-28T20:22:43Z

Datadog Report

Branch report: anna/asm/add-processors-scanners
Commit report: ad1b416
Test service: dd-trace-dotnet

✅ 0 Failed, 337457 Passed, 1549 Skipped, 41m 49.87s Wall Time
❄️ 3 New Flaky

New Flaky Tests (3)

GraphQL7SchemaV0Tests.SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests - Last Failure

Expand for error

 Results do not match.
 Differences:
 Received: GraphQL7Tests.SubmitsTraces.SchemaV0.received.txt
 Verified: GraphQL7Tests.SubmitsTraces.SchemaV0.verified.txt
 Received Content:
 [
   {
     TraceId: Id_1,
     SpanId: Id_2,
     Name: aspnet_core.request,
 ...

CleanUri_HttpUrlTag - Datadog.Trace.ClrProfiler.Managed.Tests.ScopeFactoryTests - Last Failure

Expand for error

 Assert.Equal() Failure
                                  ↓ (pos 36)
 Expected: ···om/path/to/file.aspx?query=1
 Actual:   ···om/path/to/file.aspx
                                  ↑ (pos 36)

CleanUri_HttpUrlTag - Datadog.Trace.ClrProfiler.Managed.Tests.ScopeFactoryTests - Last Failure

Expand for error

 Assert.Equal() Failure
                                  ↓ (pos 42)
 Expected: ···00/path/to/file.aspx?query=1
 Actual:   ···00/path/to/file.aspx
                                  ↑ (pos 42)

andrewlock · 2024-02-28T20:29:03Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5248) - mean (72ms)  : 66, 78
     .   : milestone, 72,
    master - mean (73ms)  : 65, 82
     .   : milestone, 73,

    section CallTarget+Inlining+NGEN
    This PR (5248) - mean (984ms)  : 958, 1010
     .   : milestone, 984,
    master - mean (981ms)  : 960, 1002
     .   : milestone, 981,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5248) - mean (111ms)  : 106, 115
     .   : milestone, 111,
    master - mean (110ms)  : 107, 113
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (5248) - mean (717ms)  : 691, 743
     .   : milestone, 717,
    master - mean (711ms)  : 688, 733
     .   : milestone, 711,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5248) - mean (96ms)  : 92, 100
     .   : milestone, 96,
    master - mean (96ms)  : 92, 99
     .   : milestone, 96,

    section CallTarget+Inlining+NGEN
    This PR (5248) - mean (675ms)  : 653, 697
     .   : milestone, 675,
    master - mean (671ms)  : 640, 702
     .   : milestone, 671,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5248) - mean (188ms)  : 186, 191
     .   : milestone, 188,
    master - mean (188ms)  : 185, 191
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (5248) - mean (1,065ms)  : 1042, 1088
     .   : milestone, 1065,
    master - mean (1,064ms)  : 1039, 1090
     .   : milestone, 1064,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5248) - mean (271ms)  : 266, 276
     .   : milestone, 271,
    master - mean (271ms)  : 267, 275
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (5248) - mean (874ms)  : 843, 904
     .   : milestone, 874,
    master - mean (867ms)  : 839, 895
     .   : milestone, 867,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5248) - mean (260ms)  : 256, 265
     .   : milestone, 260,
    master - mean (260ms)  : 257, 264
     .   : milestone, 260,

    section CallTarget+Inlining+NGEN
    This PR (5248) - mean (856ms)  : 827, 884
     .   : milestone, 856,
    master - mean (854ms)  : 828, 881
     .   : milestone, 854,

andrewlock · 2024-02-28T20:55:12Z

Benchmarks Report 🐌

Benchmarks for #5248 compared to master:

2 benchmarks are faster, with geometric mean 1.160
1 benchmarks are slower, with geometric mean 1.199
3 benchmarks have fewer allocations
2 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.83μs	46.6ns	223ns	0.0215	0.00859	0	7.5 KB
master	`StartStopWithChild`	netcoreapp3.1	10.5μs	56.7ns	300ns	0.0258	0.0103	0	7.58 KB
master	`StartStopWithChild`	net472	16.9μs	81.7ns	316ns	1.35	0.363	0.132	7.95 KB
#5248	`StartStopWithChild`	net6.0	8.74μs	46.9ns	244ns	0.0182	0.0091	0	7.48 KB
#5248	`StartStopWithChild`	netcoreapp3.1	10.5μs	57.1ns	323ns	0.0316	0.0158	0	7.59 KB
#5248	`StartStopWithChild`	net472	17.2μs	57.6ns	223ns	1.34	0.364	0.13	7.95 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	457μs	491ns	1.9μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	624μs	198ns	766ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	818μs	390ns	1.51μs	0.411	3.3 KB
#5248	`WriteAndFlushEnrichedTraces`	net6.0	470μs	175ns	654ns	0	2.7 KB
#5248	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	642μs	264ns	1.02μs	0	2.7 KB
#5248	`WriteAndFlushEnrichedTraces`	net472	813μs	557ns	2.16μs	0.406	3.3 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5248

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1	1.183	245.33	207.41

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	40.4μs	16.9ns	61ns	0.0202	0	2.36 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	43.7μs	22ns	79.3ns	0.0219	0	2.34 KB
master	`AllCycleSimpleBody`	net472	47μs	13.4ns	50.1ns	0.375	0	2.41 KB
master	`AllCycleMoreComplexBody`	net6.0	213μs	89.6ns	347ns	0.106	0	9.84 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	224μs	183ns	686ns	0.112	0	9.73 KB
master	`AllCycleMoreComplexBody`	net472	240μs	90.4ns	350ns	1.56	0	9.91 KB
master	`ObjectExtractorSimpleBody`	net6.0	141ns	0.058ns	0.209ns	0.00395	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	245ns	0.1ns	0.376ns	0.00369	0	272 B
master	`ObjectExtractorSimpleBody`	net472	172ns	0.111ns	0.428ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3.04μs	1.67ns	6.23ns	0.0532	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.03μs	1.75ns	6.55ns	0.0503	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	4.44μs	2.23ns	8.62ns	0.603	0.00665	3.8 KB
#5248	`AllCycleSimpleBody`	net6.0	40.3μs	17.8ns	69ns	0.02	0	2.36 KB
#5248	`AllCycleSimpleBody`	netcoreapp3.1	44.2μs	16.3ns	63.3ns	0.0224	0	2.34 KB
#5248	`AllCycleSimpleBody`	net472	47.4μs	38ns	147ns	0.378	0	2.41 KB
#5248	`AllCycleMoreComplexBody`	net6.0	212μs	51.7ns	193ns	0.106	0	9.84 KB
#5248	`AllCycleMoreComplexBody`	netcoreapp3.1	223μs	65.9ns	247ns	0.112	0	9.73 KB
#5248	`AllCycleMoreComplexBody`	net472	240μs	208ns	807ns	1.55	0	9.91 KB
#5248	`ObjectExtractorSimpleBody`	net6.0	142ns	0.0806ns	0.279ns	0.0039	0	280 B
#5248	`ObjectExtractorSimpleBody`	netcoreapp3.1	207ns	0.0833ns	0.312ns	0.00377	0	272 B
#5248	`ObjectExtractorSimpleBody`	net472	171ns	0.0407ns	0.158ns	0.0446	0	281 B
#5248	`ObjectExtractorMoreComplexBody`	net6.0	3.06μs	1.58ns	5.91ns	0.0534	0	3.78 KB
#5248	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.04μs	1.64ns	6.36ns	0.0504	0	3.69 KB
#5248	`ObjectExtractorMoreComplexBody`	net472	4.31μs	6.79ns	25.4ns	0.602	0.00645	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EncodeArgs`	net6.0	53.5μs	74.6ns	289ns	0.429	0	30.94 KB
master	`EncodeArgs`	netcoreapp3.1	70.9μs	58.6ns	219ns	0.425	0	31.47 KB
master	`EncodeArgs`	net472	84.8μs	38.2ns	138ns	5.12	0.086	32.27 KB
master	`EncodeLegacyArgs`	net6.0	128μs	154ns	578ns	0.444	0	33.89 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	155μs	157ns	608ns	0.453	0	34.09 KB
master	`EncodeLegacyArgs`	net472	214μs	557ns	2.16μs	5.47	0.429	34.99 KB
#5248	`EncodeArgs`	net6.0	50.3μs	70.3ns	272ns	0.424	0	30.94 KB
#5248	`EncodeArgs`	netcoreapp3.1	70.4μs	55.4ns	207ns	0.421	0	31.47 KB
#5248	`EncodeArgs`	net472	85.7μs	80.4ns	311ns	5.13	0.0854	32.27 KB
#5248	`EncodeLegacyArgs`	net6.0	127μs	397ns	1.43μs	0.474	0	33.89 KB
#5248	`EncodeLegacyArgs`	netcoreapp3.1	158μs	345ns	1.34μs	0.467	0	34.09 KB
#5248	`EncodeLegacyArgs`	net472	217μs	806ns	3.12μs	5.53	0.434	35 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunWafRealisticBenchmark`	net6.0	190μs	526ns	2.04μs	0.0953	6.51 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	207μs	446ns	1.73μs	0	6.49 KB
master	`RunWafRealisticBenchmark`	net472	224μs	113ns	438ns	1.01	6.59 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	125μs	81.6ns	294ns	0.0627	4.15 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	135μs	216ns	835ns	0	4.15 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	147μs	163ns	630ns	0.66	4.19 KB
#5248	`RunWafRealisticBenchmark`	net6.0	188μs	80.3ns	311ns	0.094	6.51 KB
#5248	`RunWafRealisticBenchmark`	netcoreapp3.1	201μs	296ns	1.15μs	0	6.49 KB
#5248	`RunWafRealisticBenchmark`	net472	224μs	155ns	602ns	1.01	6.59 KB
#5248	`RunWafRealisticBenchmarkWithAttack`	net6.0	126μs	273ns	1.06μs	0.063	4.15 KB
#5248	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	134μs	118ns	456ns	0	4.15 KB
#5248	`RunWafRealisticBenchmarkWithAttack`	net472	148μs	50.2ns	194ns	0.665	4.19 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	175μs	205ns	794ns	0.174	18.26 KB
master	`SendRequest`	netcoreapp3.1	195μs	271ns	1.05μs	0.195	20.42 KB
master	`SendRequest`	net472	0.000804ns	0.000236ns	0.000883ns	0	0 b
#5248	`SendRequest`	net6.0	173μs	188ns	726ns	0.256	18.26 KB
#5248	`SendRequest`	netcoreapp3.1	190μs	293ns	1.13μs	0.189	20.42 KB
#5248	`SendRequest`	net472	0.000323ns	0.000179ns	0.000692ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #5248

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.69 KB	41.47 KB	-224 B	-0.54%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	553μs	801ns	3.1μs	0.551	0	0	41.69 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	646μs	320ns	1.11μs	0.326	0	0	41.67 KB
master	`WriteAndFlushEnrichedTraces`	net472	838μs	2.08μs	8.07μs	8.45	2.53	0.422	53.24 KB
#5248	`WriteAndFlushEnrichedTraces`	net6.0	539μs	1.48μs	5.73μs	0.556	0	0	41.47 KB
#5248	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	661μs	555ns	2.15μs	0.332	0	0	41.74 KB
#5248	`WriteAndFlushEnrichedTraces`	net472	849μs	4.04μs	15.6μs	8.33	2.5	0.417	53.23 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.12μs	0.385ns	1.39ns	0.0112	776 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.5μs	1.75ns	6.79ns	0.00992	776 B
master	`ExecuteNonQuery`	net472	1.75μs	0.917ns	3.18ns	0.117	738 B
#5248	`ExecuteNonQuery`	net6.0	1.02μs	0.852ns	3.3ns	0.0108	776 B
#5248	`ExecuteNonQuery`	netcoreapp3.1	1.5μs	0.772ns	2.78ns	0.0105	776 B
#5248	`ExecuteNonQuery`	net472	1.71μs	2.48ns	9.6ns	0.117	738 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.18μs	0.597ns	2.31ns	0.0131	944 B
master	`CallElasticsearch`	netcoreapp3.1	1.54μs	1.12ns	4.18ns	0.0124	944 B
master	`CallElasticsearch`	net472	2.51μs	0.926ns	3.59ns	0.153	963 B
master	`CallElasticsearchAsync`	net6.0	1.3μs	0.595ns	2.23ns	0.013	920 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.64μs	1.33ns	5.15ns	0.0133	992 B
master	`CallElasticsearchAsync`	net472	2.63μs	0.924ns	3.58ns	0.162	1.02 KB
#5248	`CallElasticsearch`	net6.0	1.21μs	0.578ns	2.09ns	0.0134	944 B
#5248	`CallElasticsearch`	netcoreapp3.1	1.56μs	1.3ns	5.04ns	0.0124	944 B
#5248	`CallElasticsearch`	net472	2.53μs	2.28ns	8.85ns	0.152	963 B
#5248	`CallElasticsearchAsync`	net6.0	1.27μs	0.544ns	1.96ns	0.0127	920 B
#5248	`CallElasticsearchAsync`	netcoreapp3.1	1.66μs	0.895ns	3.47ns	0.0132	992 B
#5248	`CallElasticsearchAsync`	net472	2.66μs	1.42ns	5.51ns	0.162	1.02 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.19μs	1.41ns	5.45ns	0.0132	920 B
master	`ExecuteAsync`	netcoreapp3.1	1.63μs	0.968ns	3.75ns	0.0123	920 B
master	`ExecuteAsync`	net472	1.82μs	0.915ns	3.54ns	0.14	883 B
#5248	`ExecuteAsync`	net6.0	1.26μs	0.397ns	1.49ns	0.0129	920 B
#5248	`ExecuteAsync`	netcoreapp3.1	1.53μs	0.785ns	2.94ns	0.0123	920 B
#5248	`ExecuteAsync`	net472	1.85μs	0.624ns	2.42ns	0.14	883 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.03μs	1.83ns	6.85ns	0.0282	2.1 KB
master	`SendAsync`	netcoreapp3.1	5.04μs	2.56ns	9.57ns	0.0355	2.64 KB
master	`SendAsync`	net472	7.81μs	1.92ns	7.42ns	0.524	3.31 KB
#5248	`SendAsync`	net6.0	4.01μs	0.924ns	3.46ns	0.0281	2.1 KB
#5248	`SendAsync`	netcoreapp3.1	4.97μs	1.8ns	6.97ns	0.0346	2.64 KB
#5248	`SendAsync`	net472	7.74μs	4.73ns	18.3ns	0.526	3.31 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5248

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	202.26 KB	212.46 KB	10.21 KB	5.05%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	203.6 KB	207.09 KB	3.49 KB	1.71%

Fewer allocations 🎉 in #5248

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	60.95 KB	59.18 KB	-1.78 KB	-2.91%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472	229.38 KB	221.18 KB	-8.19 KB	-3.57%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	61.3μs	1.04μs	10.2μs	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	53.1μs	258ns	1.06μs	42.64 KB
master	`StringConcatBenchmark`	net472	37.9μs	93.5ns	350ns	60.95 KB
master	`StringConcatAspectBenchmark`	net6.0	279μs	1.02μs	3.52μs	203.6 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	282μs	1.51μs	8.92μs	202.26 KB
master	`StringConcatAspectBenchmark`	net472	226μs	1.2μs	6.48μs	229.38 KB
#5248	`StringConcatBenchmark`	net6.0	57.3μs	748ns	7.21μs	43.44 KB
#5248	`StringConcatBenchmark`	netcoreapp3.1	57.2μs	576ns	5.74μs	42.64 KB
#5248	`StringConcatBenchmark`	net472	37.6μs	78.3ns	282ns	59.18 KB
#5248	`StringConcatAspectBenchmark`	net6.0	277μs	5.91μs	57.9μs	207.09 KB
#5248	`StringConcatAspectBenchmark`	netcoreapp3.1	274μs	2.84μs	27.7μs	212.46 KB
#5248	`StringConcatAspectBenchmark`	net472	242μs	3.83μs	36.1μs	221.18 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.42μs	0.666ns	2.49ns	0.022	1.58 KB
master	`EnrichedLog`	netcoreapp3.1	2.26μs	0.925ns	3.58ns	0.0217	1.58 KB
master	`EnrichedLog`	net472	2.59μs	3.23ns	12.5ns	0.239	1.51 KB
#5248	`EnrichedLog`	net6.0	1.49μs	0.821ns	3.07ns	0.0221	1.58 KB
#5248	`EnrichedLog`	netcoreapp3.1	2.15μs	1.21ns	4.51ns	0.0217	1.58 KB
#5248	`EnrichedLog`	net472	2.54μs	1.37ns	5.14ns	0.238	1.51 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	113μs	117ns	454ns	0.0566	0	4.22 KB
master	`EnrichedLog`	netcoreapp3.1	119μs	204ns	790ns	0	0	4.22 KB
master	`EnrichedLog`	net472	148μs	212ns	822ns	0.67	0.223	4.4 KB
#5248	`EnrichedLog`	net6.0	112μs	113ns	424ns	0.0563	0	4.22 KB
#5248	`EnrichedLog`	netcoreapp3.1	119μs	188ns	727ns	0.0594	0	4.22 KB
#5248	`EnrichedLog`	net472	146μs	112ns	433ns	0.652	0.217	4.4 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.01μs	1.82ns	6.82ns	0.0301	2.14 KB
master	`EnrichedLog`	netcoreapp3.1	4.15μs	1.39ns	5.2ns	0.0291	2.14 KB
master	`EnrichedLog`	net472	4.84μs	2.06ns	7.99ns	0.309	1.95 KB
#5248	`EnrichedLog`	net6.0	2.99μs	1.18ns	4.41ns	0.03	2.14 KB
#5248	`EnrichedLog`	netcoreapp3.1	4.2μs	2.12ns	7.93ns	0.0276	2.14 KB
#5248	`EnrichedLog`	net472	4.84μs	1.35ns	5.24ns	0.309	1.95 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.4μs	0.516ns	2ns	0.0155	1.11 KB
master	`SendReceive`	netcoreapp3.1	1.77μs	0.867ns	3.36ns	0.015	1.11 KB
master	`SendReceive`	net472	2.11μs	1.3ns	4.88ns	0.178	1.12 KB
#5248	`SendReceive`	net6.0	1.44μs	0.687ns	2.57ns	0.0158	1.11 KB
#5248	`SendReceive`	netcoreapp3.1	1.81μs	0.992ns	3.84ns	0.0145	1.11 KB
#5248	`SendReceive`	net472	2.12μs	1.01ns	3.76ns	0.178	1.12 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.78μs	1.1ns	4.27ns	0.0222	1.54 KB
master	`EnrichedLog`	netcoreapp3.1	3.86μs	2.82ns	10.9ns	0.0212	1.58 KB
master	`EnrichedLog`	net472	4.3μs	1.84ns	7.14ns	0.313	1.97 KB
#5248	`EnrichedLog`	net6.0	2.77μs	0.884ns	3.19ns	0.0209	1.54 KB
#5248	`EnrichedLog`	netcoreapp3.1	3.96μs	1.33ns	5.15ns	0.0198	1.58 KB
#5248	`EnrichedLog`	net472	4.32μs	2.5ns	9.67ns	0.312	1.97 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5248

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.137	584.86	514.27

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	585ns	0.524ns	2.03ns	0.00773	544 B
master	`StartFinishSpan`	netcoreapp3.1	833ns	1.81ns	7ns	0.00704	544 B
master	`StartFinishSpan`	net472	755ns	1.83ns	7.08ns	0.0864	546 B
master	`StartFinishScope`	net6.0	652ns	0.822ns	3.18ns	0.00946	664 B
master	`StartFinishScope`	netcoreapp3.1	780ns	1.61ns	6.25ns	0.00893	664 B
master	`StartFinishScope`	net472	984ns	2.35ns	9.08ns	0.0993	626 B
#5248	`StartFinishSpan`	net6.0	514ns	0.691ns	2.59ns	0.00773	544 B
#5248	`StartFinishSpan`	netcoreapp3.1	759ns	1.24ns	4.66ns	0.00738	544 B
#5248	`StartFinishSpan`	net472	746ns	1.92ns	7.43ns	0.0865	546 B
#5248	`StartFinishScope`	net6.0	598ns	0.801ns	3.1ns	0.00918	664 B
#5248	`StartFinishScope`	netcoreapp3.1	794ns	1.8ns	6.95ns	0.00889	664 B
#5248	`StartFinishScope`	net472	1.04μs	2.59ns	10ns	0.099	626 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5248

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.199	590.27	707.83

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	589ns	1.01ns	3.9ns	0.00924	664 B
master	`RunOnMethodBegin`	netcoreapp3.1	1E+03ns	1.69ns	6.55ns	0.00896	664 B
master	`RunOnMethodBegin`	net472	1.09μs	2.02ns	7.83ns	0.0993	626 B
#5248	`RunOnMethodBegin`	net6.0	708ns	1.23ns	4.77ns	0.00923	664 B
#5248	`RunOnMethodBegin`	netcoreapp3.1	993ns	2.02ns	7.84ns	0.00899	664 B
#5248	`RunOnMethodBegin`	net472	1.11μs	1.28ns	4.96ns	0.0991	626 B

andrewlock · 2024-02-28T21:13:26Z

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5248) (11.076M)   : 0, 11076362
    master (10.889M)   : 0, 10888546
    benchmarks/2.9.0 (11.031M)   : 0, 11030962

    section Automatic
    This PR (5248) (7.610M)   : 0, 7610224
    master (7.559M)   : 0, 7559000
    benchmarks/2.9.0 (8.233M)   : 0, 8232945

    section Trace stats
    This PR (5248) (7.866M)   : 0, 7865596
    master (7.865M)   : 0, 7864929

    section Manual
    This PR (5248) (9.557M)   : 0, 9556873
    master (9.614M)   : 0, 9614359

    section Manual + Automatic
    This PR (5248) (7.163M)   : 0, 7163033
    master (7.123M)   : 0, 7123171

    section Version Conflict
    This PR (5248) (6.487M)   : 0, 6487031
    master (6.519M)   : 0, 6518649

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5248) (9.621M)   : 0, 9621230
    master (9.750M)   : 0, 9749527
    benchmarks/2.9.0 (9.465M)   : 0, 9464836

    section Automatic
    This PR (5248) (6.734M)   : 0, 6733759
    master (6.700M)   : 0, 6699639

    section Trace stats
    This PR (5248) (6.906M)   : 0, 6906076
    master (6.926M)   : 0, 6926113

    section Manual
    This PR (5248) (8.296M)   : 0, 8295506
    master (8.151M)   : 0, 8151388

    section Manual + Automatic
    This PR (5248) (6.210M)   : 0, 6210443
    master (6.123M)   : 0, 6123363

    section Version Conflict
    This PR (5248) (5.734M)   : 0, 5734346
    master (5.610M)   : 0, 5609787

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5248) (10.222M)   : 0, 10221560
    master (8.843M)   : 0, 8843308
    benchmarks/2.9.0 (10.140M)   : 0, 10140137

    section Automatic
    This PR (5248) (7.263M)   : 0, 7263039
    master (6.324M)   : 0, 6324042
    benchmarks/2.9.0 (7.489M)   : 0, 7488546

    section Trace stats
    This PR (5248) (7.664M)   : 0, 7664401
    master (6.530M)   : 0, 6529901

    section Manual
    This PR (5248) (8.915M)   : 0, 8915329
    master (8.269M)   : 0, 8268747

    section Manual + Automatic
    This PR (5248) (7.053M)   : 0, 7052552
    master (6.578M)   : 0, 6578082

    section Version Conflict
    This PR (5248) (6.271M)   : 0, 6270787
    master (6.028M)   : 0, 6027669

gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5248) (7.442M)   : 0, 7441541
    master (7.392M)   : 0, 7391587
    benchmarks/2.9.0 (7.871M)   : 0, 7870513

    section No attack
    This PR (5248) (1.850M)   : 0, 1849580
    master (1.840M)   : 0, 1839987
    benchmarks/2.9.0 (3.232M)   : 0, 3232114

    section Attack
    This PR (5248) (1.454M)   : 0, 1454285
    master (1.447M)   : 0, 1446581
    benchmarks/2.9.0 (2.539M)   : 0, 2539351

    section Blocking
    This PR (5248) (3.126M)   : 0, 3125595
    master (3.183M)   : 0, 3183450

    section IAST default
    This PR (5248) (6.459M)   : 0, 6458725
    master (6.508M)   : 0, 6508124

    section IAST full
    This PR (5248) (5.584M)   : 0, 5583644
    master (5.529M)   : 0, 5529393

    section Base vuln
    This PR (5248) (0.917M)   : 0, 917081
    master (0.907M)   : 0, 906898

    section IAST vuln
    This PR (5248) (0.843M)   : 0, 842774
    master (0.854M)   : 0, 854318

add processors and scanners

ad1b416

github-actions bot added the area:asm label Feb 28, 2024

anna-git marked this pull request as ready for review February 29, 2024 08:45

anna-git requested a review from a team as a code owner February 29, 2024 08:45

NachoEchevarria approved these changes Feb 29, 2024

View reviewed changes

daniel-romano-DD approved these changes Feb 29, 2024

View reviewed changes

anna-git merged commit c224f63 into master Feb 29, 2024
55 checks passed

anna-git deleted the anna/asm/add-processors-scanners branch February 29, 2024 16:10

github-actions bot added this to the vNext milestone Feb 29, 2024

anna-git mentioned this pull request Apr 15, 2024

[dotnet] activate external waf header and apisec processors/scanners tests DataDog/system-tests#2344

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ASM] Add processors and scanners to ruleset #5248

[ASM] Add processors and scanners to ruleset #5248

anna-git commented Feb 28, 2024 •

edited

Loading

datadog-ddstaging bot commented Feb 28, 2024 •

edited

Loading

andrewlock commented Feb 28, 2024

andrewlock commented Feb 28, 2024

Raw results

Raw results

Faster 🎉 in #5248

Raw results

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #5248

Raw results

Raw results

Raw results

Raw results

Raw results

More allocations ⚠️ in #5248

Fewer allocations 🎉 in #5248

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5248

Raw results

Slower ⚠️ in #5248

Raw results

andrewlock commented Feb 28, 2024

[ASM] Add processors and scanners to ruleset #5248

[ASM] Add processors and scanners to ruleset #5248

Conversation

anna-git commented Feb 28, 2024 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Feb 28, 2024 • edited Loading

Datadog Report

New Flaky Tests (3)

andrewlock commented Feb 28, 2024

Execution-Time Benchmarks Report ⏱️

andrewlock commented Feb 28, 2024

Benchmarks Report 🐌

Benchmark details

Raw results

Raw results

Faster 🎉 in #5248

Raw results

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #5248

Raw results

Raw results

Raw results

Raw results

Raw results

More allocations ⚠️ in #5248

Fewer allocations 🎉 in #5248

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5248

Raw results

Slower ⚠️ in #5248

Raw results

andrewlock commented Feb 28, 2024

Throughput/Crank Report:zap:

anna-git commented Feb 28, 2024 •

edited

Loading

datadog-ddstaging bot commented Feb 28, 2024 •

edited

Loading