Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[IAST] Folder casing fix #5035

Merged
merged 1 commit into from
Jan 10, 2024
Merged

[IAST] Folder casing fix #5035

merged 1 commit into from
Jan 10, 2024

Conversation

daniel-romano-DD
Copy link
Contributor

Summary of changes

Fix wrong duplicated folder casing

Reason for change

Implementation details

Test coverage

Other details

@daniel-romano-DD daniel-romano-DD requested review from a team as code owners January 9, 2024 12:20
@github-actions github-actions bot added the area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) label Jan 9, 2024
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Jan 9, 2024 •
edited
Loading

Datadog Report

Branch report: dani/asm/folder_casing
Commit report: eac0de4
Test service: dd-trace-dotnet

✅ 0 Failed, 305231 Passed, 999 Skipped, 39m 22.27s Wall Time

@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5035) - mean (71ms)  : 62, 80
     .   : milestone, 71,
    master - mean (71ms)  : 62, 80
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (5035) - mean (1,019ms)  : 995, 1044
     .   : milestone, 1019,
    master - mean (1,023ms)  : 995, 1052
     .   : milestone, 1023,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5035) - mean (105ms)  : 100, 110
     .   : milestone, 105,
    master - mean (106ms)  : 102, 110
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (5035) - mean (726ms)  : 697, 756
     .   : milestone, 726,
    master - mean (728ms)  : 704, 753
     .   : milestone, 728,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5035) - mean (90ms)  : 86, 94
     .   : milestone, 90,
    master - mean (90ms)  : 88, 93
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (5035) - mean (688ms)  : 666, 710
     .   : milestone, 688,
    master - mean (688ms)  : 667, 708
     .   : milestone, 688,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5035) - mean (189ms)  : 185, 192
     .   : milestone, 189,
    master - mean (189ms)  : 186, 193
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (5035) - mean (1,130ms)  : 1110, 1151
     .   : milestone, 1130,
    master - mean (1,132ms)  : 1110, 1153
     .   : milestone, 1132,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5035) - mean (272ms)  : 268, 276
     .   : milestone, 272,
    master - mean (273ms)  : 269, 277
     .   : milestone, 273,

    section CallTarget+Inlining+NGEN
    This PR (5035) - mean (1,082ms)  : 1055, 1110
     .   : milestone, 1082,
    master - mean (1,081ms)  : 1057, 1104
     .   : milestone, 1081,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5035) - mean (262ms)  : 259, 266
     .   : milestone, 262,
    master - mean (263ms)  : 259, 267
     .   : milestone, 263,

    section CallTarget+Inlining+NGEN
    This PR (5035) - mean (1,053ms)  : 1022, 1085
     .   : milestone, 1053,
    master - mean (1,054ms)  : 1028, 1080
     .   : milestone, 1054,
Loading

@andrewlock
Copy link
Member

Benchmarks Report 🐌

Benchmarks for #5035 compared to master:

  • 3 benchmarks are faster, with geometric mean 1.137
  • 2 benchmarks are slower, with geometric mean 1.235
  • 4 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.85μs 48.6ns 295ns 0.0214 0.00857 0 7.48 KB
master StartStopWithChild netcoreapp3.1 10.9μs 57.3ns 309ns 0.0273 0.0109 0 7.57 KB
master StartStopWithChild net472 16.9μs 52.7ns 204ns 1.36 0.375 0.128 7.96 KB
#5035 StartStopWithChild net6.0 8.49μs 46.1ns 252ns 0.0344 0.0172 0 7.48 KB
#5035 StartStopWithChild netcoreapp3.1 10.8μs 60.1ns 394ns 0.0216 0.0108 0 7.58 KB
#5035 StartStopWithChild net472 17μs 51.1ns 198ns 1.32 0.366 0.0997 7.95 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 456μs 226ns 847ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 636μs 120ns 464ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 784μs 368ns 1.38μs 0.391 0 0 3.3 KB
#5035 WriteAndFlushEnrichedTraces net6.0 474μs 183ns 710ns 0 0 0 2.7 KB
#5035 WriteAndFlushEnrichedTraces netcoreapp3.1 636μs 297ns 1.11μs 0 0 0 2.7 KB
#5035 WriteAndFlushEnrichedTraces net472 777μs 123ns 461ns 0.391 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5035

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 1.160 248.56 214.37

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 38.9μs 25.7ns 99.4ns 0.0194 0 0 1.77 KB
master AllCycleSimpleBody netcoreapp3.1 41.6μs 19.9ns 71.8ns 0.0207 0 0 1.74 KB
master AllCycleSimpleBody net472 44.9μs 16.9ns 65.4ns 0.287 0 0 1.81 KB
master AllCycleMoreComplexBody net6.0 200μs 147ns 569ns 0.0994 0 0 9.25 KB
master AllCycleMoreComplexBody netcoreapp3.1 211μs 223ns 864ns 0.105 0 0 9.14 KB
master AllCycleMoreComplexBody net472 225μs 94ns 352ns 1.46 0 0 9.32 KB
master ObjectExtractorSimpleBody net6.0 135ns 0.0635ns 0.237ns 0.00391 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 249ns 0.171ns 0.662ns 0.00362 0 0 272 B
master ObjectExtractorSimpleBody net472 164ns 0.121ns 0.454ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.01μs 1.09ns 4.08ns 0.0536 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.06μs 2.26ns 8.77ns 0.0508 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.73μs 2.18ns 8.44ns 0.603 0.0056 0 3.8 KB
#5035 AllCycleSimpleBody net6.0 38.7μs 77.2ns 299ns 0.0192 0 0 1.77 KB
#5035 AllCycleSimpleBody netcoreapp3.1 42.3μs 44.8ns 173ns 0.021 0 0 1.74 KB
#5035 AllCycleSimpleBody net472 44.8μs 17ns 65.8ns 0.272 0 0 1.81 KB
#5035 AllCycleMoreComplexBody net6.0 198μs 350ns 1.36μs 0.0987 0 0 9.25 KB
#5035 AllCycleMoreComplexBody netcoreapp3.1 213μs 80.9ns 303ns 0.105 0 0 9.14 KB
#5035 AllCycleMoreComplexBody net472 225μs 130ns 503ns 1.45 0 0 9.32 KB
#5035 ObjectExtractorSimpleBody net6.0 135ns 0.0335ns 0.125ns 0.00395 0 0 280 B
#5035 ObjectExtractorSimpleBody netcoreapp3.1 214ns 0.163ns 0.63ns 0.00374 0 0 272 B
#5035 ObjectExtractorSimpleBody net472 164ns 0.0791ns 0.307ns 0.0446 0 0 281 B
#5035 ObjectExtractorMoreComplexBody net6.0 2.98μs 1.41ns 5.28ns 0.0524 0 0 3.78 KB
#5035 ObjectExtractorMoreComplexBody netcoreapp3.1 3.94μs 1.41ns 5.44ns 0.0513 0 0 3.69 KB
#5035 ObjectExtractorMoreComplexBody net472 3.7μs 1.43ns 5.55ns 0.602 0.00559 0 3.8 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5035

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWaf(args: NestedMap (10))‑net6.0 1.125 50,691.24 57,007.84

Faster 🎉 in #5035

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWaf(args: NestedMap (100))‑net6.0 1.127 110,832.26 98,361.46

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 50.9μs 116ns 435ns 0.228 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 70.3μs 390ns 2.56μs 0.22 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) net472 96.5μs 43.8ns 164ns 2.54 0.0959 0 16.14 KB
master RunWafTwice(args=NestedMap (10)) net6.0 53.7μs 118ns 486ns 0.211 0 0 16.6 KB
master RunWafTwice(args=NestedMap (10)) netcoreapp3.1 71.1μs 185ns 783ns 0.225 0 0 16.58 KB
master RunWafTwice(args=NestedMap (10)) net472 108μs 243ns 940ns 2.65 0.104 0 16.69 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 108μs 35ns 135ns 0.27 0 0 22.41 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 131μs 615ns 2.54μs 0.272 0 0 22.36 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 164μs 117ns 454ns 3.53 0.164 0 22.7 KB
master RunWaf(args=NestedMap (100)) net6.0 109μs 618ns 4.33μs 0.444 0 0 32.76 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 135μs 675ns 3.16μs 0.399 0 0 33.33 KB
master RunWaf(args=NestedMap (100)) net472 194μs 633ns 2.45μs 5.29 0.371 0 33.67 KB
master RunWafTwice(args=NestedMap (100)) net6.0 111μs 401ns 1.55μs 0.458 0 0 33.3 KB
master RunWafTwice(args=NestedMap (100)) netcoreapp3.1 139μs 723ns 3.68μs 0.413 0 0 33.86 KB
master RunWafTwice(args=NestedMap (100)) net472 194μs 80ns 310ns 5.43 0.388 0 34.23 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 156μs 41.4ns 160ns 0.548 0 0 39.1 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 198μs 156ns 603ns 0.494 0 0 39.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 254μs 125ns 469ns 6.34 0.507 0 40.23 KB
master RunWaf(args=NestedMap (20)) net6.0 103μs 556ns 3.15μs 0.441 0 0 32.18 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 132μs 718ns 4.18μs 0.447 0 0 32.3 KB
master RunWaf(args=NestedMap (20)) net472 189μs 963ns 4.2μs 5.18 0.37 0 32.63 KB
master RunWafTwice(args=NestedMap (20)) net6.0 106μs 620ns 5.58μs 0.457 0 0 32.72 KB
master RunWafTwice(args=NestedMap (20)) netcoreapp3.1 141μs 720ns 3.3μs 0.402 0 0 32.82 KB
master RunWafTwice(args=NestedMap (20)) net472 197μs 810ns 3.14μs 5.24 0.388 0 33.19 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 156μs 129ns 447ns 0.543 0 0 38.53 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 198μs 803ns 3.11μs 0.489 0 0 38.6 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 260μs 1.27μs 5.08μs 6.13 0.522 0 39.2 KB
#5035 RunWaf(args=NestedMap (10)) net6.0 57μs 45.4ns 176ns 0.224 0 0 16.06 KB
#5035 RunWaf(args=NestedMap (10)) netcoreapp3.1 70.2μs 384ns 2.33μs 0.218 0 0 16.06 KB
#5035 RunWaf(args=NestedMap (10)) net472 100μs 53.5ns 207ns 2.53 0.0956 0 16.14 KB
#5035 RunWafTwice(args=NestedMap (10)) net6.0 53.5μs 296ns 1.82μs 0.212 0 0 16.6 KB
#5035 RunWafTwice(args=NestedMap (10)) netcoreapp3.1 73μs 390ns 2.1μs 0.224 0 0 16.58 KB
#5035 RunWafTwice(args=NestedMap (10)) net472 108μs 63.1ns 236ns 2.64 0.104 0 16.69 KB
#5035 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 117μs 54.1ns 210ns 0.328 0 0 22.41 KB
#5035 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 130μs 651ns 3.05μs 0.252 0 0 22.36 KB
#5035 RunWafWithAttack(args=Neste(...)tack) [22]) net472 158μs 60.9ns 228ns 3.56 0.158 0 22.7 KB
#5035 RunWaf(args=NestedMap (100)) net6.0 98.3μs 24ns 89.8ns 0.442 0 0 32.76 KB
#5035 RunWaf(args=NestedMap (100)) netcoreapp3.1 136μs 715ns 3.5μs 0.447 0 0 33.33 KB
#5035 RunWaf(args=NestedMap (100)) net472 186μs 134ns 518ns 5.29 0.371 0 33.67 KB
#5035 RunWafTwice(args=NestedMap (100)) net6.0 108μs 44.6ns 173ns 0.436 0 0 33.3 KB
#5035 RunWafTwice(args=NestedMap (100)) netcoreapp3.1 138μs 330ns 1.19μs 0.413 0 0 33.86 KB
#5035 RunWafTwice(args=NestedMap (100)) net472 200μs 659ns 2.47μs 5.42 0.387 0 34.23 KB
#5035 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 164μs 885ns 4.69μs 0.559 0 0 39.1 KB
#5035 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 197μs 983ns 4.28μs 0.489 0 0 39.63 KB
#5035 RunWafWithAttack(args=Neste(...)tack) [23]) net472 257μs 256ns 957ns 6.39 0.511 0 40.23 KB
#5035 RunWaf(args=NestedMap (20)) net6.0 106μs 29.6ns 115ns 0.439 0 0 32.18 KB
#5035 RunWaf(args=NestedMap (20)) netcoreapp3.1 133μs 761ns 5.64μs 0.411 0 0 32.3 KB
#5035 RunWaf(args=NestedMap (20)) net472 192μs 109ns 408ns 5.11 0.371 0 32.63 KB
#5035 RunWafTwice(args=NestedMap (20)) net6.0 111μs 648ns 5.68μs 0.455 0 0 32.72 KB
#5035 RunWafTwice(args=NestedMap (20)) netcoreapp3.1 144μs 328ns 1.23μs 0.419 0 0 32.82 KB
#5035 RunWafTwice(args=NestedMap (20)) net472 193μs 84.4ns 327ns 5.27 0.383 0 33.19 KB
#5035 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 165μs 80.9ns 303ns 0.564 0 0 38.53 KB
#5035 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 199μs 1.02μs 3.96μs 0.502 0 0 38.6 KB
#5035 RunWafWithAttack(args=Neste(...)tack) [22]) net472 255μs 316ns 1.22μs 6.12 0.51 0 39.2 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 170μs 132ns 511ns 0.17 0 0 18.25 KB
master SendRequest netcoreapp3.1 193μs 247ns 925ns 0.193 0 0 20.41 KB
master SendRequest net472 0.00135ns 0.000386ns 0.0015ns 0 0 0 0 b
#5035 SendRequest net6.0 168μs 123ns 445ns 0.252 0 0 18.25 KB
#5035 SendRequest netcoreapp3.1 190μs 378ns 1.46μs 0.192 0 0 20.41 KB
#5035 SendRequest net472 0.000131ns 0.00012ns 0.00045ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5035

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.51 KB 41.88 KB 371 B 0.89%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.69 KB 41.97 KB 284 B 0.68%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 538μs 2.33μs 8.4μs 0.532 0 0 41.51 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 668μs 1.48μs 5.73μs 0.334 0 0 41.69 KB
master WriteAndFlushEnrichedTraces net472 820μs 3.95μs 16.8μs 8.39 2.52 0.419 53.23 KB
#5035 WriteAndFlushEnrichedTraces net6.0 559μs 2.51μs 9.71μs 0.551 0 0 41.88 KB
#5035 WriteAndFlushEnrichedTraces netcoreapp3.1 654μs 1.04μs 4.03μs 0.321 0 0 41.97 KB
#5035 WriteAndFlushEnrichedTraces net472 860μs 2.82μs 10.9μs 8.13 2.57 0.428 53.22 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.08μs 0.671ns 2.6ns 0.0108 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.48μs 0.812ns 3.14ns 0.0105 0 0 768 B
master ExecuteNonQuery net472 1.8μs 0.784ns 3.03ns 0.116 0 0 730 B
#5035 ExecuteNonQuery net6.0 1.07μs 0.865ns 3.35ns 0.0107 0 0 768 B
#5035 ExecuteNonQuery netcoreapp3.1 1.55μs 0.753ns 2.91ns 0.0106 0 0 768 B
#5035 ExecuteNonQuery net472 1.76μs 2.84ns 11ns 0.115 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.19μs 0.53ns 1.98ns 0.0131 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.52μs 1.6ns 6ns 0.0129 0 0 936 B
master CallElasticsearch net472 2.47μs 0.425ns 1.59ns 0.151 0 0 955 B
master CallElasticsearchAsync net6.0 1.3μs 1.99ns 7.45ns 0.013 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.62μs 1.24ns 4.81ns 0.013 0 0 984 B
master CallElasticsearchAsync net472 2.74μs 1.03ns 4ns 0.16 0 0 1.01 KB
#5035 CallElasticsearch net6.0 1.22μs 0.459ns 1.78ns 0.0134 0 0 936 B
#5035 CallElasticsearch netcoreapp3.1 1.56μs 0.702ns 2.72ns 0.0125 0 0 936 B
#5035 CallElasticsearch net472 2.53μs 0.49ns 1.77ns 0.151 0 0 955 B
#5035 CallElasticsearchAsync net6.0 1.29μs 0.347ns 1.3ns 0.013 0 0 912 B
#5035 CallElasticsearchAsync netcoreapp3.1 1.67μs 1.34ns 5.02ns 0.0133 0 0 984 B
#5035 CallElasticsearchAsync net472 2.65μs 1.11ns 4.3ns 0.16 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.34μs 1.17ns 4.53ns 0.013 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.69μs 0.641ns 2.48ns 0.0119 0 0 912 B
master ExecuteAsync net472 1.72μs 1.61ns 6.25ns 0.139 0 0 875 B
#5035 ExecuteAsync net6.0 1.32μs 1.19ns 4.62ns 0.0127 0 0 912 B
#5035 ExecuteAsync netcoreapp3.1 1.66μs 0.62ns 2.32ns 0.0123 0 0 912 B
#5035 ExecuteAsync net472 1.83μs 0.798ns 3.09ns 0.138 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.17μs 2.57ns 9.95ns 0.0288 0 0 2.1 KB
master SendAsync netcoreapp3.1 4.94μs 2.04ns 7.9ns 0.0347 0 0 2.63 KB
master SendAsync net472 7.62μs 4.02ns 15.6ns 0.522 0 0 3.31 KB
#5035 SendAsync net6.0 3.92μs 2.29ns 8.58ns 0.0293 0 0 2.1 KB
#5035 SendAsync netcoreapp3.1 4.92μs 2.11ns 8.19ns 0.0343 0 0 2.63 KB
#5035 SendAsync net472 7.74μs 4.8ns 18.6ns 0.522 0 0 3.31 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5035

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472 57.34 KB 81.92 KB 24.58 KB 42.86%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472 58.52 KB 62.65 KB 4.13 KB 7.05%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 55μs 283ns 1.3μs 0 0 0 43.44 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 53μs 201ns 752ns 0 0 0 42.64 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 37.6μs 97.7ns 366ns 0 0 0 58.52 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 65μs 250ns 936ns 0 0 0 43.29 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 101μs 3.88μs 38.4μs 0 0 0 42.64 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 63.5μs 253ns 912ns 0 0 0 57.34 KB
#5035 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 61.5μs 841ns 8.15μs 0 0 0 43.44 KB
#5035 StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 53.2μs 255ns 1.02μs 0 0 0 42.64 KB
#5035 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 37.2μs 81.3ns 293ns 0 0 0 62.65 KB
#5035 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 94.1μs 2.55μs 25.5μs 0 0 0 43.29 KB
#5035 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 97.3μs 3.88μs 38.6μs 0 0 0 42.64 KB
#5035 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 60.4μs 837ns 8.37μs 0 0 0 81.92 KB
Benchmarks.Trace.ILoggerBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5035

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 1.125 2,329.35 2,071.03

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.44μs 0.494ns 1.91ns 0.0223 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 2.33μs 2.12ns 7.94ns 0.0209 0 0 1.57 KB
master EnrichedLog net472 2.52μs 3.65ns 14.1ns 0.238 0 0 1.5 KB
#5035 EnrichedLog net6.0 1.49μs 0.754ns 2.82ns 0.0218 0 0 1.57 KB
#5035 EnrichedLog netcoreapp3.1 2.07μs 1.01ns 3.65ns 0.0207 0 0 1.57 KB
#5035 EnrichedLog net472 2.56μs 3.06ns 11.4ns 0.237 0 0 1.5 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 112μs 124ns 463ns 0 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 121μs 196ns 757ns 0.0602 0 0 4.21 KB
master EnrichedLog net472 150μs 134ns 517ns 0.671 0.224 0 4.39 KB
#5035 EnrichedLog net6.0 115μs 94.4ns 353ns 0.0576 0 0 4.21 KB
#5035 EnrichedLog netcoreapp3.1 119μs 225ns 871ns 0 0 0 4.21 KB
#5035 EnrichedLog net472 147μs 130ns 486ns 0.664 0.221 0 4.39 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.14μs 1.26ns 4.88ns 0.0294 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 4.13μs 1.92ns 7.43ns 0.0289 0 0 2.13 KB
master EnrichedLog net472 4.91μs 2.43ns 9.41ns 0.308 0 0 1.95 KB
#5035 EnrichedLog net6.0 3.02μs 0.701ns 2.71ns 0.0301 0 0 2.13 KB
#5035 EnrichedLog netcoreapp3.1 4.15μs 1.13ns 4.07ns 0.0291 0 0 2.13 KB
#5035 EnrichedLog net472 4.64μs 2.21ns 8.27ns 0.308 0 0 1.95 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.4μs 0.755ns 2.92ns 0.0154 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.75μs 0.822ns 3.18ns 0.0149 0 0 1.1 KB
master SendReceive net472 2.13μs 3.25ns 12.6ns 0.177 0 0 1.12 KB
#5035 SendReceive net6.0 1.29μs 0.843ns 3.15ns 0.0157 0 0 1.1 KB
#5035 SendReceive netcoreapp3.1 1.77μs 0.53ns 2.05ns 0.0151 0 0 1.1 KB
#5035 SendReceive net472 2.14μs 1.11ns 4.16ns 0.177 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.67μs 1.12ns 4.35ns 0.0214 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.81μs 1.02ns 3.97ns 0.0207 0 0 1.58 KB
master EnrichedLog net472 4.29μs 0.997ns 3.59ns 0.311 0 0 1.97 KB
#5035 EnrichedLog net6.0 2.78μs 1.19ns 4.45ns 0.0208 0 0 1.53 KB
#5035 EnrichedLog netcoreapp3.1 4.07μs 1.5ns 5.63ns 0.0204 0 0 1.58 KB
#5035 EnrichedLog net472 4.31μs 1.63ns 6.33ns 0.311 0 0 1.97 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 456ns 0.347ns 1.3ns 0.00759 0 0 536 B
master StartFinishSpan netcoreapp3.1 712ns 0.272ns 1.05ns 0.0072 0 0 536 B
master StartFinishSpan net472 766ns 0.384ns 1.49ns 0.0852 0 0 538 B
master StartFinishScope net6.0 524ns 0.268ns 1ns 0.00918 0 0 656 B
master StartFinishScope netcoreapp3.1 852ns 0.461ns 1.73ns 0.00892 0 0 656 B
master StartFinishScope net472 964ns 1.79ns 6.93ns 0.0979 0 0 618 B
#5035 StartFinishSpan net6.0 445ns 0.384ns 1.44ns 0.00752 0 0 536 B
#5035 StartFinishSpan netcoreapp3.1 733ns 0.156ns 0.584ns 0.00725 0 0 536 B
#5035 StartFinishSpan net472 836ns 0.254ns 0.984ns 0.0853 0 0 538 B
#5035 StartFinishScope net6.0 518ns 0.158ns 0.591ns 0.0091 0 0 656 B
#5035 StartFinishScope netcoreapp3.1 811ns 0.28ns 1.08ns 0.00873 0 0 656 B
#5035 StartFinishScope net472 981ns 0.66ns 2.56ns 0.098 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5035

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.356 573.54 777.46

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 574ns 0.142ns 0.548ns 0.0094 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 988ns 1.92ns 7.44ns 0.0088 0 0 656 B
master RunOnMethodBegin net472 1.1μs 0.99ns 3.83ns 0.098 0 0 618 B
#5035 RunOnMethodBegin net6.0 778ns 0.293ns 1.1ns 0.00896 0 0 656 B
#5035 RunOnMethodBegin netcoreapp3.1 966ns 0.349ns 1.35ns 0.00836 0 0 656 B
#5035 RunOnMethodBegin net472 1.15μs 0.624ns 2.42ns 0.0979 0 0 618 B

@andrewlock
Copy link
Member

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5035) (9.017M)   : 0, 9016913
    master (11.753M)   : 0, 11753262
    benchmarks/2.9.0 (11.691M)   : 0, 11690725

    section Automatic
    This PR (5035) (7.542M)   : crit ,0, 7542349
    master (7.966M)   : 0, 7965560
    benchmarks/2.9.0 (8.196M)   : 0, 8195550

    section Trace stats
    This PR (5035) (7.792M)   : crit ,0, 7792414
    master (8.343M)   : 0, 8343276

    section Manual
    This PR (5035) (9.576M)   : crit ,0, 9575664
    master (10.388M)   : 0, 10388041

    section Manual + Automatic
    This PR (5035) (7.136M)   : crit ,0, 7136407
    master (7.588M)   : 0, 7587504

    section Version Conflict
    This PR (5035) (6.436M)   : crit ,0, 6436118
    master (6.966M)   : 0, 6965765
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5035) (9.644M)   : 0, 9643664
    master (9.242M)   : 0, 9241504
    benchmarks/2.9.0 (9.533M)   : 0, 9533360

    section Automatic
    This PR (5035) (6.756M)   : 0, 6755534
    master (6.483M)   : 0, 6483329

    section Trace stats
    This PR (5035) (6.791M)   : 0, 6791140
    master (6.779M)   : 0, 6778543

    section Manual
    This PR (5035) (8.302M)   : 0, 8301921
    master (8.348M)   : 0, 8347735

    section Manual + Automatic
    This PR (5035) (6.231M)   : 0, 6230684
    master (6.187M)   : 0, 6186728

    section Version Conflict
    This PR (5035) (5.703M)   : 0, 5702721
    master (5.682M)   : 0, 5681803
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5035) (10.893M)   : 0, 10893130
    master (10.602M)   : 0, 10601787
    benchmarks/2.9.0 (10.006M)   : 0, 10005933

    section Automatic
    This PR (5035) (7.507M)   : 0, 7507463
    master (7.375M)   : 0, 7375201
    benchmarks/2.9.0 (7.242M)   : 0, 7242466

    section Trace stats
    This PR (5035) (7.295M)   : 0, 7294795
    master (7.665M)   : 0, 7665372

    section Manual
    This PR (5035) (9.168M)   : 0, 9168365
    master (8.718M)   : 0, 8717600

    section Manual + Automatic
    This PR (5035) (7.029M)   : 0, 7028635
    master (7.040M)   : 0, 7040484

    section Version Conflict
    This PR (5035) (6.288M)   : 0, 6288499
    master (6.300M)   : 0, 6299859
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5035) (7.458M)   : 0, 7458141
    master (7.358M)   : 0, 7358395
    benchmarks/2.9.0 (7.870M)   : 0, 7869823

    section No attack
    This PR (5035) (1.756M)   : 0, 1755508
    master (1.765M)   : 0, 1764782
    benchmarks/2.9.0 (3.239M)   : 0, 3239486

    section Attack
    This PR (5035) (1.409M)   : 0, 1408731
    master (1.411M)   : 0, 1411188
    benchmarks/2.9.0 (2.556M)   : 0, 2556231

    section Blocking
    This PR (5035) (3.119M)   : 0, 3119325
    master (3.099M)   : 0, 3099104

    section IAST default
    This PR (5035) (6.490M)   : 0, 6490492
    master (6.495M)   : 0, 6495419

    section IAST full
    This PR (5035) (5.690M)   : 0, 5690200
    master (5.855M)   : 0, 5855342

    section Base vuln
    This PR (5035) (0.992M)   : 0, 992169
    master (0.970M)   : 0, 969506

    section IAST vuln
    This PR (5035) (0.896M)   : 0, 895685
    master (0.870M)   : 0, 870485
Loading

@daniel-romano-DD daniel-romano-DD merged commit d88e255 into master Jan 10, 2024
50 of 55 checks passed
@daniel-romano-DD daniel-romano-DD deleted the dani/asm/folder_casing branch January 10, 2024 15:21
@github-actions github-actions bot added this to the vNext milestone Jan 10, 2024
@andrewlock andrewlock added area:builds project files, build scripts, pipelines, versioning, releases, packages and removed area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) labels Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NachoEchevarria NachoEchevarria NachoEchevarria approved these changes

@andrewlock andrewlock andrewlock approved these changes

Assignees
No one assigned
Labels
area:builds project files, build scripts, pipelines, versioning, releases, packages
Projects
None yet
Milestone
2.46.0
Development

Successfully merging this pull request may close these issues.
3 participants
@daniel-romano-DD @andrewlock @NachoEchevarria