Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update querystring obfuscationregex (2) #4764

Merged
merged 3 commits into from
Oct 26, 2023
Merged

Update querystring obfuscationregex (2) #4764

merged 3 commits into from
Oct 26, 2023

Conversation

anna-git
Copy link
Contributor

Summary of changes

We realized some important keys were missing.

Reason for change

Implementation details

Test coverage

updated unit test with new key

@anna-git anna-git requested a review from a team as a code owner October 25, 2023 16:24
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Oct 25, 2023 •
edited
Loading

Datadog Report

Branch report: anna/update-querystring-obfuscationregex
Commit report: 3c799bd

✅ dd-trace-dotnet: 0 Failed, 0 New Flaky, 304166 Passed, 941 Skipped, 36m 31.85s Wall Time

@andrewlock
Copy link
Member

andrewlock commented Oct 25, 2023 •
edited
Loading

Benchmarks Report 🐌

Benchmarks for #4764 compared to master:

  • 2 benchmarks are slower, with geometric mean 1.188
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.21μs 43.6ns 222ns 0.0279 0.012 0 7.29 KB
master StartStopWithChild netcoreapp3.1 10.3μs 54.5ns 288ns 0.0251 0.01 0 7.39 KB
master StartStopWithChild net472 15.6μs 41.4ns 160ns 1.28 0.302 0.0928 7.66 KB
#4764 StartStopWithChild net6.0 8.05μs 41.9ns 210ns 0.0158 0.00789 0 7.29 KB
#4764 StartStopWithChild netcoreapp3.1 10.3μs 52ns 244ns 0.0252 0.0101 0 7.38 KB
#4764 StartStopWithChild net472 15.5μs 46.5ns 180ns 1.29 0.317 0.101 7.67 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 467μs 270ns 1.04μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 640μs 234ns 906ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 784μs 218ns 815ns 0.391 0 0 3.3 KB
#4764 WriteAndFlushEnrichedTraces net6.0 476μs 202ns 782ns 0 0 0 2.7 KB
#4764 WriteAndFlushEnrichedTraces netcoreapp3.1 623μs 186ns 697ns 0 0 0 2.7 KB
#4764 WriteAndFlushEnrichedTraces net472 784μs 359ns 1.39μs 0.391 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 43.5μs 46.5ns 180ns 0.0221 0 0 2.03 KB
master AllCycleSimpleBody netcoreapp3.1 45.7μs 147ns 569ns 0.023 0 0 2.01 KB
master AllCycleSimpleBody net472 47μs 70.2ns 272ns 0.321 0 0 2.08 KB
master AllCycleMoreComplexBody net6.0 228μs 58.9ns 220ns 0.114 0 0 8.63 KB
master AllCycleMoreComplexBody netcoreapp3.1 233μs 195ns 755ns 0 0 0 8.52 KB
master AllCycleMoreComplexBody net472 239μs 94.7ns 367ns 1.31 0 0 8.7 KB
master ObjectExtractorSimpleBody net6.0 119ns 0.114ns 0.443ns 0.00398 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 176ns 0.128ns 0.498ns 0.0037 0 0 272 B
master ObjectExtractorSimpleBody net472 149ns 0.0851ns 0.33ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.05μs 11.7ns 45.4ns 0.0539 0 0 3.88 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.06μs 1.7ns 6.58ns 0.0528 0 0 3.78 KB
master ObjectExtractorMoreComplexBody net472 4.12μs 3.32ns 12.9ns 0.617 0.00615 0 3.89 KB
#4764 AllCycleSimpleBody net6.0 43.8μs 84.9ns 329ns 0.022 0 0 2.03 KB
#4764 AllCycleSimpleBody netcoreapp3.1 45.7μs 75.8ns 293ns 0.0227 0 0 2.01 KB
#4764 AllCycleSimpleBody net472 47μs 37.9ns 147ns 0.325 0 0 2.08 KB
#4764 AllCycleMoreComplexBody net6.0 230μs 59.3ns 222ns 0.115 0 0 8.63 KB
#4764 AllCycleMoreComplexBody netcoreapp3.1 235μs 78.1ns 282ns 0.118 0 0 8.52 KB
#4764 AllCycleMoreComplexBody net472 240μs 107ns 415ns 1.31 0 0 8.7 KB
#4764 ObjectExtractorSimpleBody net6.0 124ns 0.0428ns 0.16ns 0.00397 0 0 280 B
#4764 ObjectExtractorSimpleBody netcoreapp3.1 175ns 0.149ns 0.579ns 0.00369 0 0 272 B
#4764 ObjectExtractorSimpleBody net472 144ns 0.122ns 0.471ns 0.0446 0 0 281 B
#4764 ObjectExtractorMoreComplexBody net6.0 3.03μs 0.807ns 3.13ns 0.0545 0 0 3.88 KB
#4764 ObjectExtractorMoreComplexBody netcoreapp3.1 4.01μs 1.38ns 5.34ns 0.0521 0 0 3.78 KB
#4764 ObjectExtractorMoreComplexBody net472 4.08μs 3.05ns 11ns 0.618 0.0061 0 3.89 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 12.6μs 3.16ns 11.8ns 0.134 0 0 9.42 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 19.5μs 36.2ns 140ns 0.126 0 0 9.42 KB
master RunWaf(args=NestedMap (10)) net472 28.7μs 8.43ns 31.5ns 1.5 0 0 9.48 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 76.4μs 34.8ns 126ns 0.193 0 0 15.77 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 86μs 54.9ns 205ns 0.173 0 0 15.72 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 96.8μs 47.6ns 178ns 2.52 0 0 16.04 KB
master RunWaf(args=NestedMap (100)) net6.0 24.1μs 7.66ns 29.7ns 0.277 0 0 19.66 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 38μs 18.9ns 73.3ns 0.267 0 0 20.42 KB
master RunWaf(args=NestedMap (100)) net472 56.7μs 20ns 77.3ns 3.25 0.0283 0 20.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 90.9μs 45.7ns 158ns 0.359 0 0 26.01 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 110μs 92.4ns 320ns 0.324 0 0 26.72 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 132μs 222ns 860ns 4.3 0.0652 0 27.19 KB
master RunWaf(args=NestedMap (20)) net6.0 24.2μs 10.4ns 40.1ns 0.276 0 0 19.4 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 37.4μs 21.4ns 82.7ns 0.263 0 0 19.84 KB
master RunWaf(args=NestedMap (20)) net472 53.7μs 22.7ns 87.8ns 3.17 0.0538 0 20.04 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 90.2μs 55.5ns 215ns 0.361 0 0 25.74 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 110μs 112ns 386ns 0.328 0 0 26.14 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 131μs 90.7ns 351ns 4.22 0.065 0 26.6 KB
#4764 RunWaf(args=NestedMap (10)) net6.0 12.7μs 4.09ns 15.8ns 0.128 0 0 9.42 KB
#4764 RunWaf(args=NestedMap (10)) netcoreapp3.1 19.4μs 7.35ns 28.5ns 0.126 0 0 9.42 KB
#4764 RunWaf(args=NestedMap (10)) net472 28.2μs 8.39ns 31.4ns 1.49 0 0 9.48 KB
#4764 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 76.1μs 32.2ns 120ns 0.19 0 0 15.77 KB
#4764 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 85μs 183ns 708ns 0.169 0 0 15.72 KB
#4764 RunWafWithAttack(args=Neste(...)tack) [22]) net472 97.8μs 152ns 590ns 2.52 0 0 16.04 KB
#4764 RunWaf(args=NestedMap (100)) net6.0 24.9μs 23.5ns 87.9ns 0.274 0 0 19.66 KB
#4764 RunWaf(args=NestedMap (100)) netcoreapp3.1 38μs 10.6ns 39.6ns 0.268 0 0 20.42 KB
#4764 RunWaf(args=NestedMap (100)) net472 54.5μs 15.2ns 56.9ns 3.27 0.0273 0 20.63 KB
#4764 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 93.4μs 61.5ns 238ns 0.328 0 0 26.01 KB
#4764 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 109μs 232ns 837ns 0.328 0 0 26.72 KB
#4764 RunWafWithAttack(args=Neste(...)tack) [23]) net472 131μs 151ns 584ns 4.27 0.0646 0 27.19 KB
#4764 RunWaf(args=NestedMap (20)) net6.0 24μs 13.3ns 49.6ns 0.264 0 0 19.4 KB
#4764 RunWaf(args=NestedMap (20)) netcoreapp3.1 37.9μs 17.8ns 64.1ns 0.265 0 0 19.84 KB
#4764 RunWaf(args=NestedMap (20)) net472 54μs 13.5ns 52.1ns 3.18 0.0539 0 20.04 KB
#4764 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 90.5μs 39.8ns 143ns 0.364 0 0 25.74 KB
#4764 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 108μs 168ns 650ns 0.32 0 0 26.14 KB
#4764 RunWafWithAttack(args=Neste(...)tack) [22]) net472 133μs 72.9ns 273ns 4.19 0.0665 0 26.6 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 165μs 143ns 554ns 0.251 0 0 18.04 KB
master SendRequest netcoreapp3.1 192μs 425ns 1.65μs 0.19 0 0 20.2 KB
master SendRequest net472 8.7E‑05ns 5.18E‑05ns 0.000201ns 0 0 0 0 b
#4764 SendRequest net6.0 170μs 123ns 459ns 0.255 0 0 18.04 KB
#4764 SendRequest netcoreapp3.1 191μs 234ns 875ns 0.189 0 0 20.2 KB
#4764 SendRequest net472 1.66E‑05ns 1.66E‑05ns 6.2E‑05ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #4764

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.51 KB 41.74 KB 229 B 0.55%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 557μs 2.07μs 8μs 0.558 0 0 41.51 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 664μs 620ns 2.4μs 0.336 0 0 41.81 KB
master WriteAndFlushEnrichedTraces net472 837μs 4.13μs 18μs 8.22 2.47 0.411 53.25 KB
#4764 WriteAndFlushEnrichedTraces net6.0 567μs 1.55μs 6μs 0.568 0 0 41.74 KB
#4764 WriteAndFlushEnrichedTraces netcoreapp3.1 652μs 1.22μs 4.74μs 0.329 0 0 42.01 KB
#4764 WriteAndFlushEnrichedTraces net472 867μs 3.88μs 15μs 8.36 2.64 0.44 53.26 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.04μs 0.488ns 1.89ns 0.0104 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.29μs 1.03ns 3.99ns 0.0102 0 0 768 B
master ExecuteNonQuery net472 1.62μs 0.673ns 2.61ns 0.116 0 0 730 B
#4764 ExecuteNonQuery net6.0 1.04μs 0.419ns 1.62ns 0.0104 0 0 768 B
#4764 ExecuteNonQuery netcoreapp3.1 1.37μs 4.62ns 17.9ns 0.00989 0 0 768 B
#4764 ExecuteNonQuery net472 1.73μs 1.34ns 4.82ns 0.116 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.16μs 0.547ns 2.05ns 0.0133 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.39μs 1.03ns 3.84ns 0.0125 0 0 936 B
master CallElasticsearch net472 2.36μs 1ns 3.75ns 0.151 0 0 955 B
master CallElasticsearchAsync net6.0 1.26μs 0.541ns 2.09ns 0.0125 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.56μs 3.94ns 14.7ns 0.0136 0 0 984 B
master CallElasticsearchAsync net472 2.61μs 2.86ns 10.7ns 0.16 0 0 1.01 KB
#4764 CallElasticsearch net6.0 1.23μs 0.682ns 2.46ns 0.0129 0 0 936 B
#4764 CallElasticsearch netcoreapp3.1 1.45μs 1.27ns 4.93ns 0.0124 0 0 936 B
#4764 CallElasticsearch net472 2.31μs 1.22ns 4.55ns 0.152 0 0 955 B
#4764 CallElasticsearchAsync net6.0 1.27μs 0.719ns 2.69ns 0.0126 0 0 912 B
#4764 CallElasticsearchAsync netcoreapp3.1 1.46μs 1.14ns 4.27ns 0.0132 0 0 984 B
#4764 CallElasticsearchAsync net472 2.37μs 1.2ns 4.66ns 0.16 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.19μs 1.26ns 4.88ns 0.013 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.51μs 0.246ns 0.886ns 0.0121 0 0 912 B
master ExecuteAsync net472 1.7μs 0.911ns 3.53ns 0.139 0 0 875 B
#4764 ExecuteAsync net6.0 1.19μs 0.458ns 1.71ns 0.0126 0 0 912 B
#4764 ExecuteAsync netcoreapp3.1 1.57μs 0.471ns 1.76ns 0.0119 0 0 912 B
#4764 ExecuteAsync net472 1.76μs 0.711ns 2.76ns 0.139 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 3.75μs 1.46ns 5.64ns 0.0263 0 0 1.9 KB
master SendAsync netcoreapp3.1 4.46μs 1.81ns 6.77ns 0.0335 0 0 2.43 KB
master SendAsync net472 7.08μs 1.66ns 6.44ns 0.476 0 0 2.99 KB
#4764 SendAsync net6.0 3.85μs 1.24ns 4.62ns 0.025 0 0 1.9 KB
#4764 SendAsync netcoreapp3.1 4.32μs 2.31ns 8.95ns 0.0323 0 0 2.43 KB
#4764 SendAsync net472 7.07μs 2.38ns 9.22ns 0.475 0 0 2.99 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.29μs 0.949ns 3.29ns 0.022 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 1.98μs 1.05ns 4.06ns 0.0218 0 0 1.57 KB
master EnrichedLog net472 2.21μs 1.04ns 4.04ns 0.236 0 0 1.49 KB
#4764 EnrichedLog net6.0 1.36μs 0.615ns 2.3ns 0.022 0 0 1.57 KB
#4764 EnrichedLog netcoreapp3.1 1.99μs 1.15ns 4.44ns 0.0209 0 0 1.57 KB
#4764 EnrichedLog net472 2.3μs 2.93ns 11.3ns 0.237 0 0 1.49 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 113μs 68.1ns 255ns 0.0566 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 117μs 105ns 392ns 0 0 0 4.21 KB
master EnrichedLog net472 149μs 78.3ns 293ns 0.668 0.223 0 4.38 KB
#4764 EnrichedLog net6.0 113μs 97.1ns 376ns 0.0561 0 0 4.21 KB
#4764 EnrichedLog netcoreapp3.1 118μs 111ns 415ns 0 0 0 4.21 KB
#4764 EnrichedLog net472 150μs 234ns 906ns 0.673 0.224 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.94μs 1.44ns 5.59ns 0.0295 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 3.9μs 1.18ns 4.41ns 0.0272 0 0 2.13 KB
master EnrichedLog net472 4.44μs 1.12ns 4.17ns 0.306 0 0 1.93 KB
#4764 EnrichedLog net6.0 2.85μs 1.11ns 4.17ns 0.0299 0 0 2.13 KB
#4764 EnrichedLog netcoreapp3.1 3.74μs 0.954ns 3.7ns 0.028 0 0 2.13 KB
#4764 EnrichedLog net472 4.58μs 2.91ns 10.9ns 0.305 0 0 1.93 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.29μs 1.12ns 4.19ns 0.0154 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.64μs 0.689ns 2.67ns 0.0148 0 0 1.1 KB
master SendReceive net472 1.96μs 1.41ns 5.27ns 0.177 0 0 1.12 KB
#4764 SendReceive net6.0 1.3μs 0.62ns 2.32ns 0.0151 0 0 1.1 KB
#4764 SendReceive netcoreapp3.1 1.57μs 0.734ns 2.75ns 0.0151 0 0 1.1 KB
#4764 SendReceive net472 2.07μs 3.55ns 13.7ns 0.177 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.6μs 0.848ns 3.17ns 0.0208 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.59μs 2.32ns 8.97ns 0.0197 0 0 1.58 KB
master EnrichedLog net472 4.05μs 1.42ns 5.32ns 0.309 0 0 1.96 KB
#4764 EnrichedLog net6.0 2.62μs 0.911ns 3.41ns 0.021 0 0 1.53 KB
#4764 EnrichedLog netcoreapp3.1 3.68μs 1.59ns 6.17ns 0.0202 0 0 1.58 KB
#4764 EnrichedLog net472 3.77μs 2.15ns 8.33ns 0.311 0 0 1.96 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4764

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.225 520.27 637.16

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 411ns 0.312ns 1.17ns 0.00744 0 0 536 B
master StartFinishSpan netcoreapp3.1 554ns 0.406ns 1.57ns 0.00719 0 0 536 B
master StartFinishSpan net472 635ns 0.124ns 0.482ns 0.0852 0 0 538 B
master StartFinishScope net6.0 520ns 0.144ns 0.52ns 0.00912 0 0 656 B
master StartFinishScope netcoreapp3.1 733ns 0.204ns 0.736ns 0.00881 0 0 656 B
master StartFinishScope net472 853ns 0.347ns 1.34ns 0.098 0 0 618 B
#4764 StartFinishSpan net6.0 393ns 0.192ns 0.719ns 0.0075 0 0 536 B
#4764 StartFinishSpan netcoreapp3.1 608ns 0.691ns 2.67ns 0.00727 0 0 536 B
#4764 StartFinishSpan net472 647ns 0.356ns 1.33ns 0.0852 0 0 538 B
#4764 StartFinishScope net6.0 637ns 0.2ns 0.774ns 0.00925 0 0 656 B
#4764 StartFinishScope netcoreapp3.1 758ns 0.598ns 2.32ns 0.00865 0 0 656 B
#4764 StartFinishScope net472 802ns 0.47ns 1.82ns 0.0978 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4764

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 1.151 913.93 1,052.35

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 608ns 0.291ns 1.13ns 0.00938 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 776ns 0.585ns 2.19ns 0.00898 0 0 656 B
master RunOnMethodBegin net472 914ns 0.408ns 1.53ns 0.0978 0 0 618 B
#4764 RunOnMethodBegin net6.0 574ns 0.192ns 0.744ns 0.00943 0 0 656 B
#4764 RunOnMethodBegin netcoreapp3.1 861ns 0.677ns 2.62ns 0.00875 0 0 656 B
#4764 RunOnMethodBegin net472 1.05μs 0.285ns 1.11ns 0.0978 0 0 618 B

@andrewlock
Copy link
Member

andrewlock commented Oct 25, 2023 •
edited
Loading

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4764) (11.267M)   : 0, 11266745
    master (10.849M)   : 0, 10848650
    benchmarks/2.38.0 (11.867M)   : 0, 11866956
    benchmarks/2.9.0 (11.404M)   : 0, 11403964

    section Automatic
    This PR (4764) (7.815M)   : 0, 7815259
    master (7.689M)   : 0, 7688765
    benchmarks/2.38.0 (8.176M)   : 0, 8175636
    benchmarks/2.9.0 (8.122M)   : 0, 8121934

    section Trace stats
    This PR (4764) (8.115M)   : 0, 8114680
    master (7.974M)   : 0, 7973986
    benchmarks/2.38.0 (8.450M)   : 0, 8450370

    section Manual
    This PR (4764) (9.883M)   : 0, 9883139
    master (9.776M)   : 0, 9775925
    benchmarks/2.38.0 (10.334M)   : 0, 10334368

    section Manual + Automatic
    This PR (4764) (7.401M)   : 0, 7401259
    master (7.300M)   : 0, 7299779
    benchmarks/2.38.0 (7.750M)   : 0, 7750484

    section Version Conflict
    This PR (4764) (6.860M)   : 0, 6859708
    master (6.575M)   : 0, 6574617
    benchmarks/2.38.0 (7.137M)   : 0, 7136691
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4764) (9.540M)   : 0, 9540041
    master (9.534M)   : 0, 9534375
    benchmarks/2.38.0 (9.548M)   : 0, 9548121
    benchmarks/2.9.0 (9.656M)   : 0, 9655806

    section Automatic
    This PR (4764) (6.801M)   : 0, 6801347
    master (6.740M)   : 0, 6739842
    benchmarks/2.38.0 (6.747M)   : 0, 6747227

    section Trace stats
    This PR (4764) (6.951M)   : 0, 6950634
    master (6.775M)   : 0, 6774905
    benchmarks/2.38.0 (6.815M)   : 0, 6814846

    section Manual
    This PR (4764) (8.352M)   : 0, 8351680
    master (8.302M)   : 0, 8302488
    benchmarks/2.38.0 (8.263M)   : 0, 8263131

    section Manual + Automatic
    This PR (4764) (6.344M)   : 0, 6343513
    master (6.349M)   : 0, 6348627
    benchmarks/2.38.0 (6.275M)   : 0, 6275411

    section Version Conflict
    This PR (4764) (5.883M)   : 0, 5882920
    master (5.749M)   : 0, 5749334
    benchmarks/2.38.0 (5.670M)   : 0, 5669744
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4764) (9.410M)   : 0, 9409595
    master (9.450M)   : 0, 9449674
    benchmarks/2.38.0 (9.007M)   : 0, 9007066
    benchmarks/2.9.0 (9.731M)   : 0, 9730905

    section Automatic
    This PR (4764) (6.774M)   : 0, 6774344
    master (6.635M)   : 0, 6634884
    benchmarks/2.38.0 (6.423M)   : 0, 6422756
    benchmarks/2.9.0 (6.908M)   : 0, 6908188

    section Trace stats
    This PR (4764) (7.045M)   : 0, 7045086
    master (6.864M)   : 0, 6863822
    benchmarks/2.38.0 (6.721M)   : 0, 6720838

    section Manual
    This PR (4764) (8.444M)   : 0, 8444112
    master (8.250M)   : 0, 8250092
    benchmarks/2.38.0 (7.977M)   : 0, 7977494

    section Manual + Automatic
    This PR (4764) (6.601M)   : 0, 6601258
    master (6.462M)   : 0, 6462400
    benchmarks/2.38.0 (6.349M)   : 0, 6349066

    section Version Conflict
    This PR (4764) (5.886M)   : 0, 5886450
    master (5.794M)   : 0, 5794375
    benchmarks/2.38.0 (5.682M)   : 0, 5681654
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.751M)   : 0, 7750914
    benchmarks/2.38.0 (7.578M)   : 0, 7577544
    benchmarks/2.9.0 (7.904M)   : 0, 7903721

    section No attack
    master (2.190M)   : 0, 2190399
    benchmarks/2.38.0 (2.184M)   : 0, 2184279
    benchmarks/2.9.0 (3.239M)   : 0, 3238804

    section Attack
    master (1.724M)   : 0, 1723833
    benchmarks/2.38.0 (1.711M)   : 0, 1710542
    benchmarks/2.9.0 (2.459M)   : 0, 2458912

    section Blocking
    master (3.465M)   : 0, 3464934
    benchmarks/2.38.0 (3.486M)   : 0, 3486124

    section IAST default
    master (6.848M)   : 0, 6847970

    section IAST full
    master (6.223M)   : 0, 6222736

    section Base vuln
    master (0.962M)   : 0, 961814

    section IAST vuln
    master (0.904M)   : 0, 903897
Loading

@andrewlock
Copy link
Member

andrewlock commented Oct 26, 2023 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4764) - mean (71ms)  : 64, 78
     .   : milestone, 71,
    master - mean (77ms)  : 67, 88
     .   : milestone, 77,

    section CallTarget+Inlining+NGEN
    This PR (4764) - mean (979ms)  : 963, 995
     .   : milestone, 979,
    master - mean (977ms)  : 954, 999
     .   : milestone, 977,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4764) - mean (106ms)  : 104, 108
     .   : milestone, 106,
    master - mean (106ms)  : 102, 109
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (4764) - mean (672ms)  : 648, 695
     .   : milestone, 672,
    master - mean (671ms)  : 652, 689
     .   : milestone, 671,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4764) - mean (90ms)  : 88, 92
     .   : milestone, 90,
    master - mean (90ms)  : 88, 92
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (4764) - mean (637ms)  : 621, 652
     .   : milestone, 637,
    master - mean (630ms)  : 617, 642
     .   : milestone, 630,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4764) - mean (187ms)  : 185, 189
     .   : milestone, 187,
    master - mean (188ms)  : 185, 190
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (4764) - mean (1,140ms)  : 1100, 1179
     .   : milestone, 1140,
    master - mean (1,127ms)  : 1108, 1145
     .   : milestone, 1127,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4764) - mean (274ms)  : 267, 280
     .   : milestone, 274,
    master - mean (271ms)  : 267, 275
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (4764) - mean (1,085ms)  : 1062, 1107
     .   : milestone, 1085,
    master - mean (1,091ms)  : 1068, 1113
     .   : milestone, 1091,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4764) - mean (262ms)  : 259, 264
     .   : milestone, 262,
    master - mean (261ms)  : 257, 265
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (4764) - mean (1,056ms)  : 1028, 1083
     .   : milestone, 1056,
    master - mean (1,049ms)  : 1016, 1082
     .   : milestone, 1049,
Loading

@anna-git anna-git merged commit 8c23d88 into master Oct 26, 2023
@anna-git anna-git deleted the anna/update-querystring-obfuscationregex branch October 26, 2023 16:13
@github-actions github-actions bot added this to the vNext milestone Oct 26, 2023
@andrewlock andrewlock added type:enhancement Improvement to an existing feature area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) area:asm labels Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NachoEchevarria NachoEchevarria NachoEchevarria approved these changes

@andrewlock andrewlock andrewlock approved these changes

Assignees
No one assigned
Labels
area:asm area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) type:enhancement Improvement to an existing feature
Projects
None yet
Milestone
2.41.0
Development

Successfully merging this pull request may close these issues.
3 participants
@anna-git @andrewlock @NachoEchevarria