Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Always log rejit errors #5061

Merged
merged 1 commit into from
Jan 16, 2024
Merged

Always log rejit errors #5061

merged 1 commit into from
Jan 16, 2024

Conversation

kevingosse
Copy link
Collaborator

Summary of changes

Currently we log rejit errors only if the profiler is not initialized, which... doesn't seem quite right.

Reason for change

There are some rejit errors on the automapper tests.

Implementation details

Moved a line.

@kevingosse kevingosse requested a review from a team as a code owner January 15, 2024 14:32
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Jan 15, 2024 •
edited
Loading

Datadog Report

Branch report: kevin/rejitError
Commit report: 20e1b3c
Test service: dd-trace-dotnet

✅ 0 Failed, 311809 Passed, 1451 Skipped, 50m 55.69s Wall Time
❄️ 1 New Flaky

New Flaky Tests (1)

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.AspNetMvc4TestsCallTargetClassic - Last Failure

    Expand for error 
     Gave up waiting for IIS Express.

@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5061) - mean (71ms)  : 61, 81
     .   : milestone, 71,
    master - mean (72ms)  : 62, 82
     .   : milestone, 72,

    section CallTarget+Inlining+NGEN
    This PR (5061) - mean (1,019ms)  : 990, 1048
     .   : milestone, 1019,
    master - mean (1,021ms)  : 995, 1047
     .   : milestone, 1021,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5061) - mean (105ms)  : 101, 109
     .   : milestone, 105,
    master - mean (106ms)  : 102, 110
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (5061) - mean (727ms)  : 710, 745
     .   : milestone, 727,
    master - mean (727ms)  : 705, 750
     .   : milestone, 727,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5061) - mean (89ms)  : 85, 93
     .   : milestone, 89,
    master - mean (90ms)  : 86, 93
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (5061) - mean (690ms)  : 665, 715
     .   : milestone, 690,
    master - mean (691ms)  : 666, 717
     .   : milestone, 691,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5061) - mean (188ms)  : 185, 191
     .   : milestone, 188,
    master - mean (188ms)  : 185, 190
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (5061) - mean (1,132ms)  : 1110, 1155
     .   : milestone, 1132,
    master - mean (1,127ms)  : 1103, 1151
     .   : milestone, 1127,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5061) - mean (271ms)  : 266, 276
     .   : milestone, 271,
    master - mean (271ms)  : 267, 276
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (5061) - mean (1,093ms)  : 1073, 1113
     .   : milestone, 1093,
    master - mean (1,083ms)  : 1063, 1104
     .   : milestone, 1083,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5061) - mean (261ms)  : 258, 264
     .   : milestone, 261,
    master - mean (261ms)  : 258, 264
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (5061) - mean (1,051ms)  : 1028, 1073
     .   : milestone, 1051,
    master - mean (1,052ms)  : 1023, 1082
     .   : milestone, 1052,
Loading

@andrewlock
Copy link
Member

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5061) (10.873M)   : 0, 10872783
    master (11.780M)   : 0, 11779759
    benchmarks/2.9.0 (11.164M)   : 0, 11164134

    section Automatic
    This PR (5061) (7.455M)   : crit ,0, 7454644
    master (8.058M)   : 0, 8058094
    benchmarks/2.9.0 (8.136M)   : 0, 8135871

    section Trace stats
    This PR (5061) (8.129M)   : 0, 8129196
    master (8.439M)   : 0, 8439082

    section Manual
    This PR (5061) (9.926M)   : 0, 9925693
    master (10.242M)   : 0, 10242271

    section Manual + Automatic
    This PR (5061) (7.260M)   : crit ,0, 7260368
    master (7.659M)   : 0, 7658903

    section Version Conflict
    This PR (5061) (6.574M)   : 0, 6574449
    master (6.892M)   : 0, 6891771
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5061) (9.630M)   : 0, 9629510
    master (9.578M)   : 0, 9578150
    benchmarks/2.9.0 (9.823M)   : 0, 9823469

    section Automatic
    This PR (5061) (6.577M)   : 0, 6577349
    master (6.571M)   : 0, 6571022

    section Trace stats
    This PR (5061) (6.922M)   : 0, 6922324
    master (6.876M)   : 0, 6876077

    section Manual
    This PR (5061) (8.347M)   : 0, 8347477
    master (8.283M)   : 0, 8282694

    section Manual + Automatic
    This PR (5061) (6.076M)   : 0, 6076328
    master (6.177M)   : 0, 6176931

    section Version Conflict
    This PR (5061) (5.457M)   : crit ,0, 5456553
    master (5.793M)   : 0, 5793005
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5061) (10.308M)   : 0, 10307807
    master (9.912M)   : 0, 9912439
    benchmarks/2.9.0 (9.958M)   : 0, 9958018

    section Automatic
    This PR (5061) (6.962M)   : 0, 6962364
    master (7.122M)   : 0, 7121793
    benchmarks/2.9.0 (7.298M)   : 0, 7297998

    section Trace stats
    This PR (5061) (7.333M)   : 0, 7332805
    master (7.489M)   : 0, 7489063

    section Manual
    This PR (5061) (9.004M)   : 0, 9004425
    master (8.728M)   : 0, 8728136

    section Manual + Automatic
    This PR (5061) (6.829M)   : 0, 6828625
    master (6.827M)   : 0, 6826805

    section Version Conflict
    This PR (5061) (6.086M)   : 0, 6085550
    master (6.138M)   : 0, 6138255
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.525M)   : 0, 7524901
    benchmarks/2.9.0 (6.929M)   : 0, 6928745

    section No attack
    master (1.776M)   : 0, 1776039
    benchmarks/2.9.0 (3.016M)   : 0, 3016270

    section Attack
    master (1.417M)   : 0, 1417213
    benchmarks/2.9.0 (2.468M)   : 0, 2467885

    section Blocking
    master (3.155M)   : 0, 3154797

    section IAST default
    master (6.580M)   : 0, 6579676

    section IAST full
    master (5.776M)   : 0, 5775711

    section Base vuln
    master (0.981M)   : 0, 980889

    section IAST vuln
    master (0.893M)   : 0, 892652
Loading

@andrewlock
Copy link
Member

Benchmarks Report 🐌

Benchmarks for #5061 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.124
  • 3 benchmarks are slower, with geometric mean 1.273
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.71μs 45.9ns 230ns 0.021 0.00841 0 7.48 KB
master StartStopWithChild netcoreapp3.1 10.9μs 60.2ns 376ns 0.0263 0.0105 0 7.58 KB
master StartStopWithChild net472 17μs 54.1ns 195ns 1.34 0.346 0.118 7.95 KB
#5061 StartStopWithChild net6.0 8.6μs 48.1ns 337ns 0.0246 0.00819 0 7.48 KB
#5061 StartStopWithChild netcoreapp3.1 10.9μs 59.2ns 335ns 0.0263 0.0105 0 7.58 KB
#5061 StartStopWithChild net472 17.3μs 56.5ns 219ns 1.31 0.329 0.0953 7.94 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 468μs 386ns 1.5μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 627μs 85.1ns 295ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 804μs 338ns 1.31μs 0.398 0 0 3.3 KB
#5061 WriteAndFlushEnrichedTraces net6.0 451μs 203ns 761ns 0 0 0 2.7 KB
#5061 WriteAndFlushEnrichedTraces netcoreapp3.1 635μs 301ns 1.17μs 0 0 0 2.7 KB
#5061 WriteAndFlushEnrichedTraces net472 802μs 408ns 1.58μs 0.401 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 38.6μs 58.8ns 228ns 0.0193 0 0 1.77 KB
master AllCycleSimpleBody netcoreapp3.1 41.9μs 25.2ns 97.8ns 0.0209 0 0 1.74 KB
master AllCycleSimpleBody net472 44.5μs 10.1ns 39ns 0.267 0 0 1.81 KB
master AllCycleMoreComplexBody net6.0 201μs 414ns 1.6μs 0.0997 0 0 9.25 KB
master AllCycleMoreComplexBody netcoreapp3.1 213μs 289ns 1.12μs 0.106 0 0 9.14 KB
master AllCycleMoreComplexBody net472 227μs 71.2ns 276ns 1.48 0 0 9.32 KB
master ObjectExtractorSimpleBody net6.0 141ns 0.0495ns 0.185ns 0.00392 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 197ns 0.1ns 0.388ns 0.00368 0 0 272 B
master ObjectExtractorSimpleBody net472 170ns 0.102ns 0.394ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.05μs 0.782ns 3.03ns 0.0534 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.89μs 1.06ns 4.12ns 0.0485 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.81μs 0.922ns 3.45ns 0.603 0.00574 0 3.8 KB
#5061 AllCycleSimpleBody net6.0 38.7μs 22.7ns 87.9ns 0.0194 0 0 1.77 KB
#5061 AllCycleSimpleBody netcoreapp3.1 41.7μs 59.1ns 229ns 0.0205 0 0 1.74 KB
#5061 AllCycleSimpleBody net472 44.4μs 12.1ns 46.7ns 0.288 0 0 1.81 KB
#5061 AllCycleMoreComplexBody net6.0 200μs 50.6ns 189ns 0.0998 0 0 9.25 KB
#5061 AllCycleMoreComplexBody netcoreapp3.1 212μs 180ns 698ns 0.106 0 0 9.14 KB
#5061 AllCycleMoreComplexBody net472 226μs 121ns 467ns 1.47 0 0 9.32 KB
#5061 ObjectExtractorSimpleBody net6.0 145ns 0.168ns 0.65ns 0.00396 0 0 280 B
#5061 ObjectExtractorSimpleBody netcoreapp3.1 191ns 0.0533ns 0.206ns 0.00365 0 0 272 B
#5061 ObjectExtractorSimpleBody net472 170ns 0.0762ns 0.295ns 0.0446 0 0 281 B
#5061 ObjectExtractorMoreComplexBody net6.0 3.02μs 1.56ns 6.05ns 0.0529 0 0 3.78 KB
#5061 ObjectExtractorMoreComplexBody netcoreapp3.1 4.03μs 1.36ns 5.28ns 0.0505 0 0 3.69 KB
#5061 ObjectExtractorMoreComplexBody net472 3.76μs 1.21ns 4.54ns 0.603 0.00564 0 3.8 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 56.3μs 316ns 2.26μs 0.228 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 69.7μs 363ns 1.85μs 0.21 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) net472 94.8μs 25.6ns 95.9ns 2.54 0.0941 0 16.14 KB
master RunWafTwice(args=NestedMap (10)) net6.0 57.1μs 330ns 2.64μs 0.234 0 0 16.6 KB
master RunWafTwice(args=NestedMap (10)) netcoreapp3.1 74.2μs 426ns 3.41μs 0.222 0 0 16.58 KB
master RunWafTwice(args=NestedMap (10)) net472 103μs 40.7ns 158ns 2.64 0.104 0 16.69 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 110μs 52ns 201ns 0.322 0 0 22.41 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 128μs 657ns 3.08μs 0.252 0 0 22.36 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 159μs 100ns 389ns 3.56 0.158 0 22.7 KB
master RunWaf(args=NestedMap (100)) net6.0 103μs 23ns 83.1ns 0.436 0 0 32.76 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 136μs 498ns 1.93μs 0.401 0 0 33.33 KB
master RunWaf(args=NestedMap (100)) net472 187μs 92ns 356ns 5.33 0.374 0 33.67 KB
master RunWafTwice(args=NestedMap (100)) net6.0 104μs 485ns 1.94μs 0.472 0 0 33.3 KB
master RunWafTwice(args=NestedMap (100)) netcoreapp3.1 142μs 682ns 2.97μs 0.425 0 0 33.86 KB
master RunWafTwice(args=NestedMap (100)) net472 194μs 71.8ns 269ns 5.4 0.386 0 34.23 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 159μs 53.2ns 206ns 0.553 0 0 39.1 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 204μs 546ns 2.12μs 0.486 0 0 39.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 253μs 49.1ns 184ns 6.3 0.504 0 40.23 KB
master RunWaf(args=NestedMap (20)) net6.0 106μs 23.5ns 90.9ns 0.435 0 0 32.18 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 132μs 749ns 5.35μs 0.397 0 0 32.3 KB
master RunWaf(args=NestedMap (20)) net472 187μs 765ns 2.96μs 5.1 0.371 0 32.63 KB
master RunWafTwice(args=NestedMap (20)) net6.0 110μs 396ns 1.53μs 0.456 0 0 32.72 KB
master RunWafTwice(args=NestedMap (20)) netcoreapp3.1 139μs 147ns 570ns 0.394 0 0 32.82 KB
master RunWafTwice(args=NestedMap (20)) net472 195μs 171ns 663ns 5.27 0.39 0 33.19 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 169μs 72.3ns 280ns 0.507 0 0 38.53 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 193μs 82.8ns 321ns 0.512 0 0 38.6 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 256μs 754ns 2.92μs 6.16 0.503 0 39.2 KB
#5061 RunWaf(args=NestedMap (10)) net6.0 55.6μs 341ns 3.41μs 0.229 0 0 16.06 KB
#5061 RunWaf(args=NestedMap (10)) netcoreapp3.1 69.2μs 395ns 2.9μs 0.219 0 0 16.06 KB
#5061 RunWaf(args=NestedMap (10)) net472 94.7μs 49ns 190ns 2.55 0.0945 0 16.14 KB
#5061 RunWafTwice(args=NestedMap (10)) net6.0 52.1μs 57.9ns 200ns 0.234 0 0 16.6 KB
#5061 RunWafTwice(args=NestedMap (10)) netcoreapp3.1 73.9μs 372ns 1.71μs 0.19 0 0 16.58 KB
#5061 RunWafTwice(args=NestedMap (10)) net472 103μs 41.3ns 160ns 2.63 0.103 0 16.69 KB
#5061 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 107μs 45ns 174ns 0.283 0 0 22.41 KB
#5061 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 128μs 572ns 2.22μs 0.311 0 0 22.36 KB
#5061 RunWafWithAttack(args=Neste(...)tack) [22]) net472 161μs 792ns 3.36μs 3.54 0.157 0 22.7 KB
#5061 RunWaf(args=NestedMap (100)) net6.0 99.3μs 36.7ns 142ns 0.444 0 0 32.76 KB
#5061 RunWaf(args=NestedMap (100)) netcoreapp3.1 133μs 740ns 4.56μs 0.442 0 0 33.33 KB
#5061 RunWaf(args=NestedMap (100)) net472 189μs 933ns 3.85μs 5.34 0.368 0 33.67 KB
#5061 RunWafTwice(args=NestedMap (100)) net6.0 114μs 32ns 120ns 0.456 0 0 33.3 KB
#5061 RunWafTwice(args=NestedMap (100)) netcoreapp3.1 142μs 673ns 2.69μs 0.426 0 0 33.86 KB
#5061 RunWafTwice(args=NestedMap (100)) net472 193μs 117ns 452ns 5.43 0.388 0 34.23 KB
#5061 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 164μs 79.2ns 296ns 0.547 0 0 39.1 KB
#5061 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 204μs 1.05μs 5.27μs 0.49 0 0 39.63 KB
#5061 RunWafWithAttack(args=Neste(...)tack) [23]) net472 254μs 304ns 1.18μs 6.3 0.504 0 40.23 KB
#5061 RunWaf(args=NestedMap (20)) net6.0 109μs 211ns 816ns 0.445 0 0 32.18 KB
#5061 RunWaf(args=NestedMap (20)) netcoreapp3.1 130μs 688ns 3.37μs 0.408 0 0 32.3 KB
#5061 RunWaf(args=NestedMap (20)) net472 184μs 69.6ns 260ns 5.15 0.368 0 32.63 KB
#5061 RunWafTwice(args=NestedMap (20)) net6.0 99.9μs 28.9ns 126ns 0.428 0 0 32.72 KB
#5061 RunWafTwice(args=NestedMap (20)) netcoreapp3.1 133μs 739ns 4.44μs 0.398 0 0 32.82 KB
#5061 RunWafTwice(args=NestedMap (20)) net472 195μs 772ns 2.99μs 5.19 0.385 0 33.19 KB
#5061 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 166μs 550ns 2.13μs 0.484 0 0 38.53 KB
#5061 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 197μs 306ns 1.19μs 0.494 0 0 38.6 KB
#5061 RunWafWithAttack(args=Neste(...)tack) [22]) net472 254μs 231ns 894ns 6.21 0.507 0 39.2 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 174μs 238ns 922ns 0.174 0 0 18.25 KB
master SendRequest netcoreapp3.1 193μs 292ns 1.13μs 0.194 0 0 20.41 KB
master SendRequest net472 0.000618ns 0.000303ns 0.00113ns 0 0 0 0 b
#5061 SendRequest net6.0 175μs 450ns 1.62μs 0.173 0 0 18.25 KB
#5061 SendRequest netcoreapp3.1 194μs 267ns 1.03μs 0.192 0 0 20.41 KB
#5061 SendRequest net472 0.00152ns 0.000481ns 0.00186ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5061

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.5 KB 41.74 KB 242 B 0.58%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 546μs 1.06μs 4.12μs 0.546 0 0 41.5 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 649μs 1.14μs 4.41μs 0.331 0 0 41.63 KB
master WriteAndFlushEnrichedTraces net472 864μs 4.39μs 19.1μs 8.36 2.64 0.44 53.26 KB
#5061 WriteAndFlushEnrichedTraces net6.0 552μs 228ns 883ns 0.532 0 0 41.74 KB
#5061 WriteAndFlushEnrichedTraces netcoreapp3.1 663μs 1.52μs 5.89μs 0.332 0 0 41.76 KB
#5061 WriteAndFlushEnrichedTraces net472 810μs 4.04μs 18.1μs 8.22 2.47 0.411 53.25 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.23μs 0.584ns 2.26ns 0.0104 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.44μs 1.09ns 4.21ns 0.0101 0 0 768 B
master ExecuteNonQuery net472 1.78μs 1.4ns 5.43ns 0.116 0 0 730 B
#5061 ExecuteNonQuery net6.0 1.12μs 0.362ns 1.4ns 0.0108 0 0 768 B
#5061 ExecuteNonQuery netcoreapp3.1 1.45μs 0.698ns 2.7ns 0.0101 0 0 768 B
#5061 ExecuteNonQuery net472 1.71μs 0.994ns 3.85ns 0.115 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.24μs 0.664ns 2.49ns 0.0131 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.56μs 0.449ns 1.68ns 0.0125 0 0 936 B
master CallElasticsearch net472 2.58μs 0.594ns 2.22ns 0.152 0 0 955 B
master CallElasticsearchAsync net6.0 1.32μs 1.65ns 6.17ns 0.0128 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.63μs 0.664ns 2.48ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.63μs 1.04ns 3.89ns 0.161 0.00131 0 1.01 KB
#5061 CallElasticsearch net6.0 1.23μs 0.53ns 2.05ns 0.0133 0 0 936 B
#5061 CallElasticsearch netcoreapp3.1 1.52μs 2.97ns 11.1ns 0.0129 0 0 936 B
#5061 CallElasticsearch net472 2.46μs 1.45ns 5.62ns 0.152 0.00123 0 955 B
#5061 CallElasticsearchAsync net6.0 1.28μs 1.13ns 4.21ns 0.0124 0 0 912 B
#5061 CallElasticsearchAsync netcoreapp3.1 1.6μs 1.45ns 5.41ns 0.0135 0 0 984 B
#5061 CallElasticsearchAsync net472 2.66μs 0.505ns 1.89ns 0.159 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.28μs 0.752ns 2.91ns 0.0127 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.58μs 0.535ns 2ns 0.0125 0 0 912 B
master ExecuteAsync net472 1.72μs 0.776ns 2.8ns 0.139 0 0 875 B
#5061 ExecuteAsync net6.0 1.27μs 0.701ns 2.72ns 0.0127 0 0 912 B
#5061 ExecuteAsync netcoreapp3.1 1.71μs 0.84ns 3.25ns 0.0121 0 0 912 B
#5061 ExecuteAsync net472 1.81μs 1.01ns 3.78ns 0.138 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.07μs 1.94ns 7.51ns 0.0285 0 0 2.1 KB
master SendAsync netcoreapp3.1 5.02μs 4.42ns 15.9ns 0.0351 0 0 2.63 KB
master SendAsync net472 7.81μs 2.48ns 8.96ns 0.522 0 0 3.31 KB
#5061 SendAsync net6.0 4.13μs 1.38ns 5.18ns 0.029 0 0 2.1 KB
#5061 SendAsync netcoreapp3.1 5.12μs 2.47ns 9.25ns 0.0361 0 0 2.63 KB
#5061 SendAsync net472 7.99μs 3.64ns 14.1ns 0.521 0 0 3.31 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5061

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472 1.247 49,200.00 61,350.00

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 60.2μs 668ns 6.65μs 0 0 0 43.44 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 52.6μs 263ns 1.08μs 0 0 0 42.64 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 37.3μs 127ns 457ns 0 0 0 62.66 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 65.5μs 255ns 956ns 0 0 0 43.29 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 96.8μs 3.8μs 37.6μs 0 0 0 42.64 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 49.1μs 92.8ns 321ns 0 0 0 57.34 KB
#5061 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 54.7μs 235ns 1.29μs 0 0 0 43.44 KB
#5061 StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 53.3μs 89.2ns 345ns 0 0 0 42.64 KB
#5061 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 38μs 118ns 443ns 0 0 0 62.47 KB
#5061 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 89.5μs 2.44μs 24.4μs 0 0 0 43.29 KB
#5061 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 102μs 3.02μs 30.2μs 0 0 0 42.64 KB
#5061 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 61.3μs 143ns 533ns 0 0 0 57.34 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.51μs 0.735ns 2.75ns 0.022 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 2.11μs 0.747ns 2.69ns 0.0211 0 0 1.57 KB
master EnrichedLog net472 2.73μs 2.08ns 8.07ns 0.237 0 0 1.5 KB
#5061 EnrichedLog net6.0 1.45μs 0.663ns 2.48ns 0.0222 0 0 1.57 KB
#5061 EnrichedLog netcoreapp3.1 2.21μs 5.3ns 20.5ns 0.0209 0 0 1.57 KB
#5061 EnrichedLog net472 2.66μs 0.992ns 3.71ns 0.237 0 0 1.5 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 112μs 99.6ns 373ns 0.0564 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 118μs 108ns 419ns 0.0587 0 0 4.21 KB
master EnrichedLog net472 148μs 48.5ns 181ns 0.666 0.222 0 4.39 KB
#5061 EnrichedLog net6.0 114μs 187ns 698ns 0.0561 0 0 4.21 KB
#5061 EnrichedLog netcoreapp3.1 118μs 174ns 652ns 0 0 0 4.21 KB
#5061 EnrichedLog net472 148μs 68.7ns 257ns 0.672 0.224 0 4.39 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.05μs 1.41ns 5.47ns 0.0305 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 4.26μs 1.9ns 7.36ns 0.0277 0 0 2.13 KB
master EnrichedLog net472 4.82μs 3.07ns 11.9ns 0.308 0 0 1.95 KB
#5061 EnrichedLog net6.0 3.04μs 1.15ns 4.46ns 0.0291 0 0 2.13 KB
#5061 EnrichedLog netcoreapp3.1 4.32μs 3.9ns 15.1ns 0.0282 0 0 2.13 KB
#5061 EnrichedLog net472 4.9μs 1.66ns 6.44ns 0.309 0 0 1.95 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.34μs 0.594ns 2.14ns 0.0154 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.76μs 1.18ns 4.57ns 0.0148 0 0 1.1 KB
master SendReceive net472 2.14μs 2.06ns 7.69ns 0.177 0 0 1.12 KB
#5061 SendReceive net6.0 1.37μs 0.352ns 1.32ns 0.0156 0 0 1.1 KB
#5061 SendReceive netcoreapp3.1 1.7μs 0.775ns 2.9ns 0.0145 0 0 1.1 KB
#5061 SendReceive net472 2.12μs 1.49ns 5.59ns 0.177 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.79μs 1.09ns 4.08ns 0.0211 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.82μs 1.21ns 4.69ns 0.0212 0 0 1.58 KB
master EnrichedLog net472 4.32μs 1.87ns 7.25ns 0.311 0 0 1.97 KB
#5061 EnrichedLog net6.0 2.87μs 1.23ns 4.58ns 0.0216 0 0 1.53 KB
#5061 EnrichedLog netcoreapp3.1 3.91μs 3.15ns 12.2ns 0.0218 0 0 1.58 KB
#5061 EnrichedLog net472 4.3μs 7.83ns 30.3ns 0.312 0 0 1.97 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5061

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.238 509.21 630.29

Faster 🎉 in #5061

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.124 974.13 866.62

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 461ns 0.142ns 0.512ns 0.00743 0 0 536 B
master StartFinishSpan netcoreapp3.1 694ns 0.367ns 1.42ns 0.00705 0 0 536 B
master StartFinishSpan net472 724ns 0.293ns 1.14ns 0.0853 0 0 538 B
master StartFinishScope net6.0 509ns 0.111ns 0.429ns 0.0092 0 0 656 B
master StartFinishScope netcoreapp3.1 974ns 0.296ns 1.15ns 0.00868 0 0 656 B
master StartFinishScope net472 942ns 2.93ns 11.3ns 0.0979 0 0 618 B
#5061 StartFinishSpan net6.0 456ns 0.145ns 0.56ns 0.00738 0 0 536 B
#5061 StartFinishSpan netcoreapp3.1 727ns 0.248ns 0.96ns 0.00697 0 0 536 B
#5061 StartFinishSpan net472 755ns 0.622ns 2.33ns 0.0851 0 0 538 B
#5061 StartFinishScope net6.0 630ns 0.0909ns 0.328ns 0.00919 0 0 656 B
#5061 StartFinishScope netcoreapp3.1 867ns 0.426ns 1.59ns 0.00848 0 0 656 B
#5061 StartFinishScope net472 988ns 0.582ns 2.26ns 0.0978 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5061

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.338 581.34 777.60

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 581ns 0.292ns 1.13ns 0.00909 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 976ns 0.377ns 1.46ns 0.00868 0 0 656 B
master RunOnMethodBegin net472 1.17μs 0.341ns 1.32ns 0.0978 0 0 618 B
#5061 RunOnMethodBegin net6.0 778ns 0.225ns 0.87ns 0.00935 0 0 656 B
#5061 RunOnMethodBegin netcoreapp3.1 940ns 0.132ns 0.475ns 0.00876 0 0 656 B
#5061 RunOnMethodBegin net472 1.1μs 0.606ns 2.35ns 0.0981 0 0 618 B

@kevingosse kevingosse merged commit 4a142f0 into master Jan 16, 2024
60 checks passed
@kevingosse kevingosse deleted the kevin/rejitError branch January 16, 2024 14:17
@github-actions github-actions bot added this to the vNext milestone Jan 16, 2024
@andrewlock andrewlock added the area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) label Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewlock andrewlock andrewlock approved these changes

@tonyredondo tonyredondo tonyredondo approved these changes

Assignees
No one assigned
Labels
area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native)
Projects
None yet
Milestone
2.46.0
Development

Successfully merging this pull request may close these issues.
3 participants
@kevingosse @andrewlock @tonyredondo