Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable Datadog static analysis #5057

Merged
merged 3 commits into from
Feb 29, 2024
Merged

Enable Datadog static analysis #5057

merged 3 commits into from
Feb 29, 2024

Conversation

andrewlock
Copy link
Member

Summary of changes

Adds a GitHub workflow to use the Datadog C# static analysis

Reason for change

We'd like to 🐶🥫

Implementation details

Added the workflow as described at: https://app.datadoghq.com/ci/setup/static-analysis

Test coverage

Let's see what it says!

Other details

@andrewlock andrewlock added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Jan 12, 2024
@andrewlock andrewlock requested a review from a team as a code owner January 12, 2024 17:12
markazerdd
markazerdd suggested changes Jan 12, 2024
View reviewed changes
.github/static-analysis.datadog.yml Outdated
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is actually unrelated to GitHub Actions and must be in the root directory of the repo

@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Jan 16, 2024 •
edited
Loading

Datadog Report

Branch report: andrew/ci/add-static-analysis
Commit report: 40a52ba
Test service: dd-trace-dotnet

❌ 398 Failed (1 Known Flaky), 218647 Passed, 898 Skipped, 57m 20.18s Wall Time

❌ Failed Tests (398)

This report shows up to 5 failed tests.

  • GraphQL2SchemaV0Tests.SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests - Details

    Expand for error 
     Address not available Address not available
 ---- System.Net.Http.HttpRequestException : Address not available
 -------- System.Net.Sockets.SocketException : Address not available

  • IntegrationDisabled - Datadog.Trace.ClrProfiler.IntegrationTests.AdoNet.SystemDataSqliteTests - Details

    Expand for error 
     No space left on device

  • SubmitsTracesV0 - Datadog.Trace.ClrProfiler.IntegrationTests.AdoNet.SystemDataSqliteTests - Details

    Expand for error 
     No space left on device

  • SubmitsTracesV1 - Datadog.Trace.ClrProfiler.IntegrationTests.AdoNet.SystemDataSqliteTests - Details

    Expand for error 
     No space left on device

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.AgentMalfunctionTests - Details

    Expand for error 
     No space left on device

@andrewlock
Copy link
Member Author

andrewlock commented Jan 16, 2024 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5057) - mean (74ms)  : 63, 85
     .   : milestone, 74,
    master - mean (74ms)  : 66, 82
     .   : milestone, 74,

    section CallTarget+Inlining+NGEN
    This PR (5057) - mean (989ms)  : 963, 1015
     .   : milestone, 989,
    master - mean (997ms)  : 975, 1018
     .   : milestone, 997,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5057) - mean (111ms)  : 106, 117
     .   : milestone, 111,
    master - mean (111ms)  : 108, 115
     .   : milestone, 111,

    section CallTarget+Inlining+NGEN
    This PR (5057) - mean (728ms)  : 705, 751
     .   : milestone, 728,
    master - mean (720ms)  : 702, 737
     .   : milestone, 720,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5057) - mean (96ms)  : 92, 100
     .   : milestone, 96,
    master - mean (95ms)  : 91, 99
     .   : milestone, 95,

    section CallTarget+Inlining+NGEN
    This PR (5057) - mean (677ms)  : 655, 698
     .   : milestone, 677,
    master - mean (679ms)  : 651, 707
     .   : milestone, 679,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5057) - mean (188ms)  : 185, 191
     .   : milestone, 188,
    master - mean (187ms)  : 185, 190
     .   : milestone, 187,

    section CallTarget+Inlining+NGEN
    This PR (5057) - mean (1,060ms)  : 1040, 1080
     .   : milestone, 1060,
    master - mean (1,062ms)  : 1041, 1083
     .   : milestone, 1062,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5057) - mean (271ms)  : 266, 275
     .   : milestone, 271,
    master - mean (271ms)  : 266, 275
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (5057) - mean (1,065ms)  : 1039, 1090
     .   : milestone, 1065,
    master - mean (1,056ms)  : 1034, 1078
     .   : milestone, 1056,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5057) - mean (261ms)  : 257, 264
     .   : milestone, 261,
    master - mean (257ms)  : 252, 262
     .   : milestone, 257,

    section CallTarget+Inlining+NGEN
    This PR (5057) - mean (1,002ms)  : 984, 1020
     .   : milestone, 1002,
    master - mean (1,000ms)  : 976, 1024
     .   : milestone, 1000,
Loading

@andrewlock
Copy link
Member Author

Benchmarks Report 🐌

Benchmarks for #5057 compared to master:

  • 3 benchmarks are faster, with geometric mean 1.140
  • 1 benchmarks are slower, with geometric mean 1.214
  • 1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.79μs 47.8ns 279ns 0.0139 0.00465 0 7.49 KB
master StartStopWithChild netcoreapp3.1 10.7μs 58.6ns 342ns 0.0327 0.0163 0 7.58 KB
master StartStopWithChild net472 17.1μs 53.8ns 208ns 1.35 0.347 0.11 7.94 KB
#5057 StartStopWithChild net6.0 8.73μs 46.4ns 254ns 0.025 0.0125 0 7.49 KB
#5057 StartStopWithChild netcoreapp3.1 10.9μs 58.7ns 342ns 0.033 0.0165 0 7.57 KB
#5057 StartStopWithChild net472 17.2μs 70.8ns 274ns 1.32 0.346 0.11 7.95 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 461μs 147ns 571ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 629μs 160ns 619ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 802μs 246ns 954ns 0.396 0 0 3.3 KB
#5057 WriteAndFlushEnrichedTraces net6.0 480μs 132ns 513ns 0 0 0 2.7 KB
#5057 WriteAndFlushEnrichedTraces netcoreapp3.1 640μs 124ns 464ns 0 0 0 2.7 KB
#5057 WriteAndFlushEnrichedTraces net472 789μs 778ns 3.01μs 0.393 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5057

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody‑net6.0 1.214 2,981.06 3,619.32

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 38.7μs 68ns 264ns 0.019 0 0 1.77 KB
master AllCycleSimpleBody netcoreapp3.1 41.8μs 91.8ns 356ns 0.0206 0 0 1.74 KB
master AllCycleSimpleBody net472 44.2μs 40ns 155ns 0.286 0 0 1.81 KB
master AllCycleMoreComplexBody net6.0 201μs 64.7ns 251ns 0.1 0 0 9.25 KB
master AllCycleMoreComplexBody netcoreapp3.1 212μs 151ns 585ns 0.106 0 0 9.14 KB
master AllCycleMoreComplexBody net472 225μs 73.4ns 284ns 1.46 0 0 9.32 KB
master ObjectExtractorSimpleBody net6.0 144ns 0.0612ns 0.237ns 0.00393 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 220ns 0.0932ns 0.336ns 0.00365 0 0 272 B
master ObjectExtractorSimpleBody net472 170ns 0.0501ns 0.194ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 2.98μs 1.69ns 6.33ns 0.0538 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.06μs 0.96ns 3.59ns 0.0488 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.81μs 2.21ns 8.57ns 0.602 0.0057 0 3.8 KB
#5057 AllCycleSimpleBody net6.0 39μs 17.8ns 66.7ns 0.0194 0 0 1.77 KB
#5057 AllCycleSimpleBody netcoreapp3.1 42μs 45.5ns 176ns 0.0211 0 0 1.74 KB
#5057 AllCycleSimpleBody net472 44.6μs 14.4ns 54ns 0.288 0 0 1.81 KB
#5057 AllCycleMoreComplexBody net6.0 199μs 41.3ns 149ns 0.0995 0 0 9.25 KB
#5057 AllCycleMoreComplexBody netcoreapp3.1 212μs 280ns 1.09μs 0.106 0 0 9.14 KB
#5057 AllCycleMoreComplexBody net472 226μs 89ns 345ns 1.46 0 0 9.32 KB
#5057 ObjectExtractorSimpleBody net6.0 148ns 0.0892ns 0.322ns 0.00394 0 0 280 B
#5057 ObjectExtractorSimpleBody netcoreapp3.1 202ns 0.12ns 0.448ns 0.00367 0 0 272 B
#5057 ObjectExtractorSimpleBody net472 163ns 0.0557ns 0.209ns 0.0446 0 0 281 B
#5057 ObjectExtractorMoreComplexBody net6.0 3.62μs 1.24ns 4.8ns 0.0524 0 0 3.78 KB
#5057 ObjectExtractorMoreComplexBody netcoreapp3.1 4.09μs 1.52ns 5.89ns 0.0492 0 0 3.69 KB
#5057 ObjectExtractorMoreComplexBody net472 3.8μs 1.54ns 5.78ns 0.603 0.0057 0 3.8 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5057

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafTwice(args: NestedMap (20))‑net6.0 1.126 113,359.40 100,664.22

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 50.3μs 17.8ns 64.3ns 0.227 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 70.3μs 386ns 2.29μs 0.198 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) net472 95.5μs 55.2ns 199ns 2.53 0.0953 0 16.14 KB
master RunWafTwice(args=NestedMap (10)) net6.0 57μs 334ns 3.11μs 0.237 0 0 16.6 KB
master RunWafTwice(args=NestedMap (10)) netcoreapp3.1 74μs 429ns 1.92μs 0.189 0 0 16.58 KB
master RunWafTwice(args=NestedMap (10)) net472 104μs 44.7ns 173ns 2.6 0.104 0 16.69 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 109μs 54.7ns 212ns 0.287 0 0 22.41 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 129μs 600ns 2.55μs 0.316 0 0 22.36 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 164μs 793ns 3.46μs 3.55 0.158 0 22.7 KB
master RunWaf(args=NestedMap (100)) net6.0 105μs 582ns 3.73μs 0.464 0 0 32.76 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 131μs 740ns 4.97μs 0.408 0 0 33.33 KB
master RunWaf(args=NestedMap (100)) net472 184μs 64.2ns 240ns 5.26 0.369 0 33.67 KB
master RunWafTwice(args=NestedMap (100)) net6.0 99.6μs 27ns 101ns 0.449 0 0 33.3 KB
master RunWafTwice(args=NestedMap (100)) netcoreapp3.1 137μs 730ns 3.93μs 0.418 0 0 33.86 KB
master RunWafTwice(args=NestedMap (100)) net472 193μs 211ns 817ns 5.38 0.384 0 34.23 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 157μs 134ns 482ns 0.546 0 0 39.1 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 201μs 989ns 4.2μs 0.479 0 0 39.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 260μs 425ns 1.65μs 6.35 0.508 0 40.23 KB
master RunWaf(args=NestedMap (20)) net6.0 104μs 292ns 1.13μs 0.438 0 0 32.18 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 127μs 525ns 1.89μs 0.377 0 0 32.3 KB
master RunWaf(args=NestedMap (20)) net472 190μs 874ns 3.39μs 5.13 0.367 0 32.63 KB
master RunWafTwice(args=NestedMap (20)) net6.0 113μs 81.6ns 316ns 0.45 0 0 32.72 KB
master RunWafTwice(args=NestedMap (20)) netcoreapp3.1 141μs 686ns 3.07μs 0.413 0 0 32.82 KB
master RunWafTwice(args=NestedMap (20)) net472 195μs 805ns 3.12μs 5.22 0.386 0 33.19 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 163μs 70.3ns 272ns 0.543 0 0 38.53 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 200μs 1.01μs 4.5μs 0.484 0 0 38.6 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 250μs 128ns 497ns 6.23 0.498 0 39.2 KB
#5057 RunWaf(args=NestedMap (10)) net6.0 55.3μs 26.4ns 98.9ns 0.209 0 0 16.06 KB
#5057 RunWaf(args=NestedMap (10)) netcoreapp3.1 74.7μs 74.6ns 289ns 0.21 0 0 16.06 KB
#5057 RunWaf(args=NestedMap (10)) net472 98.4μs 506ns 2.26μs 2.55 0.0946 0 16.14 KB
#5057 RunWafTwice(args=NestedMap (10)) net6.0 55.6μs 19.1ns 71.5ns 0.223 0 0 16.6 KB
#5057 RunWafTwice(args=NestedMap (10)) netcoreapp3.1 75.2μs 386ns 1.81μs 0.187 0 0 16.58 KB
#5057 RunWafTwice(args=NestedMap (10)) net472 104μs 63.4ns 245ns 2.63 0.103 0 16.69 KB
#5057 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 112μs 609ns 3.39μs 0.285 0 0 22.41 KB
#5057 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 125μs 85.1ns 329ns 0.253 0 0 22.36 KB
#5057 RunWafWithAttack(args=Neste(...)tack) [22]) net472 165μs 738ns 2.86μs 3.59 0.16 0 22.7 KB
#5057 RunWaf(args=NestedMap (100)) net6.0 111μs 46.7ns 181ns 0.427 0 0 32.76 KB
#5057 RunWaf(args=NestedMap (100)) netcoreapp3.1 137μs 375ns 1.45μs 0.441 0 0 33.33 KB
#5057 RunWaf(args=NestedMap (100)) net472 185μs 124ns 480ns 5.29 0.371 0 33.67 KB
#5057 RunWafTwice(args=NestedMap (100)) net6.0 102μs 36.1ns 140ns 0.457 0 0 33.3 KB
#5057 RunWafTwice(args=NestedMap (100)) netcoreapp3.1 138μs 738ns 3.83μs 0.427 0 0 33.86 KB
#5057 RunWafTwice(args=NestedMap (100)) net472 199μs 695ns 2.69μs 5.39 0.385 0 34.23 KB
#5057 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 160μs 448ns 1.61μs 0.558 0 0 39.1 KB
#5057 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 205μs 969ns 3.99μs 0.473 0 0 39.63 KB
#5057 RunWafWithAttack(args=Neste(...)tack) [23]) net472 264μs 264ns 1.02μs 6.38 0.51 0 40.23 KB
#5057 RunWaf(args=NestedMap (20)) net6.0 108μs 477ns 1.85μs 0.44 0 0 32.18 KB
#5057 RunWaf(args=NestedMap (20)) netcoreapp3.1 133μs 630ns 2.6μs 0.398 0 0 32.3 KB
#5057 RunWaf(args=NestedMap (20)) net472 187μs 117ns 452ns 5.12 0.373 0 32.63 KB
#5057 RunWafTwice(args=NestedMap (20)) net6.0 101μs 24.6ns 88.8ns 0.428 0 0 32.72 KB
#5057 RunWafTwice(args=NestedMap (20)) netcoreapp3.1 143μs 152ns 588ns 0.454 0 0 32.82 KB
#5057 RunWafTwice(args=NestedMap (20)) net472 193μs 148ns 574ns 5.18 0.384 0 33.19 KB
#5057 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 162μs 851ns 4.25μs 0.469 0 0 38.53 KB
#5057 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 199μs 926ns 3.7μs 0.498 0 0 38.6 KB
#5057 RunWafWithAttack(args=Neste(...)tack) [22]) net472 252μs 87.6ns 339ns 6.16 0.503 0 39.2 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 171μs 145ns 542ns 0.171 0 0 18.25 KB
master SendRequest netcoreapp3.1 193μs 247ns 958ns 0.191 0 0 20.41 KB
master SendRequest net472 0.0013ns 0.00036ns 0.00135ns 0 0 0 0 b
#5057 SendRequest net6.0 171μs 138ns 534ns 0.172 0 0 18.25 KB
#5057 SendRequest netcoreapp3.1 195μs 272ns 1.05μs 0.194 0 0 20.41 KB
#5057 SendRequest net472 0.000706ns 0.000294ns 0.0011ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 555μs 1.2μs 4.49μs 0.553 0 0 41.86 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 661μs 1.38μs 5.36μs 0.332 0 0 41.68 KB
master WriteAndFlushEnrichedTraces net472 850μs 2.88μs 11.1μs 8.45 2.53 0.422 53.23 KB
#5057 WriteAndFlushEnrichedTraces net6.0 573μs 2.73μs 10.6μs 0.566 0 0 41.77 KB
#5057 WriteAndFlushEnrichedTraces netcoreapp3.1 672μs 575ns 2.23μs 0.338 0 0 41.62 KB
#5057 WriteAndFlushEnrichedTraces net472 832μs 3.02μs 11.3μs 8.12 2.44 0.406 53.26 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.03μs 0.682ns 2.64ns 0.0105 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.49μs 1.24ns 4.79ns 0.0104 0 0 768 B
master ExecuteNonQuery net472 1.82μs 3.21ns 12.4ns 0.115 0 0 730 B
#5057 ExecuteNonQuery net6.0 1.1μs 0.618ns 2.39ns 0.0105 0 0 768 B
#5057 ExecuteNonQuery netcoreapp3.1 1.55μs 1.14ns 4.41ns 0.01 0 0 768 B
#5057 ExecuteNonQuery net472 1.72μs 1.2ns 4.66ns 0.116 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.21μs 1.15ns 4.44ns 0.0127 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.52μs 0.587ns 2.27ns 0.0129 0 0 936 B
master CallElasticsearch net472 2.45μs 0.894ns 3.46ns 0.152 0 0 955 B
master CallElasticsearchAsync net6.0 1.27μs 0.618ns 2.39ns 0.0128 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.65μs 0.671ns 2.51ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.57μs 1.55ns 6ns 0.16 0 0 1.01 KB
#5057 CallElasticsearch net6.0 1.19μs 0.77ns 2.98ns 0.0132 0 0 936 B
#5057 CallElasticsearch netcoreapp3.1 1.51μs 0.816ns 2.94ns 0.0129 0 0 936 B
#5057 CallElasticsearch net472 2.4μs 1.04ns 4.02ns 0.152 0 0 955 B
#5057 CallElasticsearchAsync net6.0 1.29μs 0.699ns 2.62ns 0.0125 0 0 912 B
#5057 CallElasticsearchAsync netcoreapp3.1 1.62μs 1.58ns 5.91ns 0.0129 0 0 984 B
#5057 CallElasticsearchAsync net472 2.65μs 0.645ns 2.41ns 0.16 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.33μs 0.459ns 1.78ns 0.0127 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.63μs 1.03ns 3.84ns 0.0122 0 0 912 B
master ExecuteAsync net472 1.84μs 0.612ns 2.29ns 0.139 0 0 875 B
#5057 ExecuteAsync net6.0 1.27μs 0.938ns 3.51ns 0.0128 0 0 912 B
#5057 ExecuteAsync netcoreapp3.1 1.69μs 0.552ns 1.99ns 0.0118 0 0 912 B
#5057 ExecuteAsync net472 1.88μs 1.55ns 5.82ns 0.139 0.000938 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.04μs 0.739ns 2.66ns 0.0285 0 0 2.1 KB
master SendAsync netcoreapp3.1 4.93μs 8.96ns 34.7ns 0.0368 0 0 2.63 KB
master SendAsync net472 7.66μs 4.19ns 15.7ns 0.522 0 0 3.31 KB
#5057 SendAsync net6.0 4.1μs 2.22ns 8.31ns 0.0287 0 0 2.1 KB
#5057 SendAsync netcoreapp3.1 4.96μs 2.62ns 9.81ns 0.0346 0 0 2.63 KB
#5057 SendAsync net472 7.74μs 3.2ns 12.4ns 0.522 0 0 3.31 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #5057

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472 60.13 KB 58.52 KB -1.61 KB -2.67%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 51.4μs 226ns 846ns 0 0 0 43.44 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 53μs 263ns 1.09μs 0 0 0 42.64 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 37.7μs 131ns 491ns 0 0 0 60.13 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 66.4μs 180ns 696ns 0 0 0 43.29 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 109μs 3.58μs 35.5μs 0 0 0 42.64 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 64μs 224ns 807ns 0 0 0 57.34 KB
#5057 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 54.7μs 184ns 687ns 0 0 0 43.44 KB
#5057 StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 53.5μs 177ns 638ns 0 0 0 42.64 KB
#5057 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 38μs 84.3ns 315ns 0 0 0 58.52 KB
#5057 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 66.4μs 260ns 972ns 0 0 0 43.29 KB
#5057 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 108μs 3.96μs 39.2μs 0 0 0 42.64 KB
#5057 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 61.4μs 260ns 1.01μs 0 0 0 57.34 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.43μs 0.811ns 3.14ns 0.0222 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 2.18μs 0.715ns 2.77ns 0.0207 0 0 1.57 KB
master EnrichedLog net472 2.63μs 2.36ns 9.15ns 0.237 0 0 1.5 KB
#5057 EnrichedLog net6.0 1.47μs 0.551ns 2.06ns 0.0222 0 0 1.57 KB
#5057 EnrichedLog netcoreapp3.1 2.34μs 0.811ns 2.92ns 0.0211 0 0 1.57 KB
#5057 EnrichedLog net472 2.77μs 6.05ns 23.4ns 0.237 0 0 1.5 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 114μs 212ns 820ns 0.0568 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 118μs 167ns 647ns 0.0586 0 0 4.21 KB
master EnrichedLog net472 148μs 85.5ns 331ns 0.664 0.221 0 4.39 KB
#5057 EnrichedLog net6.0 113μs 106ns 410ns 0.0564 0 0 4.21 KB
#5057 EnrichedLog netcoreapp3.1 118μs 100ns 375ns 0.0593 0 0 4.21 KB
#5057 EnrichedLog net472 147μs 78.2ns 303ns 0.663 0.221 0 4.39 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.94μs 1.09ns 4.2ns 0.0295 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 4.16μs 2ns 7.21ns 0.0295 0 0 2.13 KB
master EnrichedLog net472 4.87μs 1.36ns 5.1ns 0.307 0 0 1.95 KB
#5057 EnrichedLog net6.0 3.09μs 1.12ns 4.21ns 0.0295 0 0 2.13 KB
#5057 EnrichedLog netcoreapp3.1 4.19μs 1.36ns 4.73ns 0.0292 0 0 2.13 KB
#5057 EnrichedLog net472 4.98μs 1.73ns 6.68ns 0.309 0 0 1.95 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.42μs 0.802ns 3ns 0.0156 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.68μs 0.97ns 3.76ns 0.0153 0 0 1.1 KB
master SendReceive net472 2.04μs 1ns 3.74ns 0.177 0 0 1.12 KB
#5057 SendReceive net6.0 1.41μs 0.818ns 3.06ns 0.0154 0 0 1.1 KB
#5057 SendReceive netcoreapp3.1 1.76μs 0.748ns 2.8ns 0.015 0 0 1.1 KB
#5057 SendReceive net472 2.13μs 2.86ns 11.1ns 0.176 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.77μs 1.2ns 4.5ns 0.0208 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.97μs 1.37ns 5.12ns 0.02 0 0 1.58 KB
master EnrichedLog net472 4.28μs 2.27ns 8.8ns 0.312 0 0 1.97 KB
#5057 EnrichedLog net6.0 2.73μs 6.38ns 23.9ns 0.0206 0 0 1.53 KB
#5057 EnrichedLog netcoreapp3.1 3.8μs 0.91ns 3.41ns 0.0211 0 0 1.58 KB
#5057 EnrichedLog net472 4.32μs 1.92ns 7.44ns 0.311 0 0 1.97 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 512ns 0.148ns 0.571ns 0.00743 0 0 536 B
master StartFinishSpan netcoreapp3.1 747ns 0.76ns 2.84ns 0.00729 0 0 536 B
master StartFinishSpan net472 785ns 0.404ns 1.57ns 0.0852 0 0 538 B
master StartFinishScope net6.0 633ns 0.459ns 1.72ns 0.00921 0 0 656 B
master StartFinishScope netcoreapp3.1 891ns 0.256ns 0.957ns 0.00893 0 0 656 B
master StartFinishScope net472 1.02μs 1.02ns 3.96ns 0.0978 0 0 618 B
#5057 StartFinishSpan net6.0 508ns 0.104ns 0.361ns 0.00762 0 0 536 B
#5057 StartFinishSpan netcoreapp3.1 691ns 0.266ns 1.03ns 0.00691 0 0 536 B
#5057 StartFinishSpan net472 763ns 0.383ns 1.43ns 0.0852 0 0 538 B
#5057 StartFinishScope net6.0 598ns 0.387ns 1.5ns 0.00933 0 0 656 B
#5057 StartFinishScope netcoreapp3.1 866ns 0.594ns 2.3ns 0.00872 0 0 656 B
#5057 StartFinishScope net472 936ns 0.542ns 2.1ns 0.0979 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5057

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.151 746.03 648.36
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 1.143 1,150.58 1,006.69

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 746ns 0.346ns 1.34ns 0.00918 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 924ns 0.474ns 1.83ns 0.00884 0 0 656 B
master RunOnMethodBegin net472 1.15μs 0.542ns 2.03ns 0.0982 0 0 618 B
#5057 RunOnMethodBegin net6.0 649ns 0.355ns 1.38ns 0.0093 0 0 656 B
#5057 RunOnMethodBegin netcoreapp3.1 1μs 0.37ns 1.43ns 0.00846 0 0 656 B
#5057 RunOnMethodBegin net472 1.01μs 0.42ns 1.51ns 0.0981 0 0 618 B

@andrewlock andrewlock force-pushed the andrew/ci/add-static-analysis branch from 40a52ba to 6469130 Compare February 15, 2024 13:45
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Feb 15, 2024 •
edited
Loading

Datadog Report

Branch report: andrew/ci/add-static-analysis
Commit report: baf82eb
Test service: dd-trace-dotnet

✅ 0 Failed, 310377 Passed, 1555 Skipped, 40m 47.58s Wall Time
⌛ 4 Performance Regressions

⌛ Performance Regressions vs Default Branch (4)

  • Profiler_cpu_walltime - scenarios 4.5s (+98.36ms, +2%) - Details
  • Profiler_walltime - scenarios 3.38s (+74.18ms, +2%) - Details
  • Baseline_allocation - scenarios 2.26s (+98.86ms, +5%) - Details
  • Profiler_cpu_walltime - scenarios 3.39s (+88.47ms, +3%) - Details

@andrewlock
Copy link
Member Author

andrewlock commented Feb 15, 2024 •
edited
Loading

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5057) (11.097M)   : 0, 11097236
    master (10.975M)   : 0, 10975378
    benchmarks/2.9.0 (11.163M)   : 0, 11162756

    section Automatic
    This PR (5057) (7.685M)   : 0, 7684671
    master (7.712M)   : 0, 7712004
    benchmarks/2.9.0 (8.268M)   : 0, 8267853

    section Trace stats
    This PR (5057) (7.995M)   : 0, 7995469
    master (7.900M)   : 0, 7900048

    section Manual
    This PR (5057) (9.841M)   : 0, 9840669
    master (9.590M)   : 0, 9589996

    section Manual + Automatic
    This PR (5057) (7.218M)   : 0, 7217617
    master (7.199M)   : 0, 7199081

    section Version Conflict
    This PR (5057) (6.863M)   : 0, 6862577
    master (6.595M)   : 0, 6594857
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5057) (9.723M)   : 0, 9723379
    master (9.559M)   : 0, 9559250
    benchmarks/2.9.0 (9.367M)   : 0, 9367466

    section Automatic
    This PR (5057) (6.619M)   : 0, 6618810
    master (6.430M)   : 0, 6429697

    section Trace stats
    This PR (5057) (6.827M)   : 0, 6827310
    master (6.913M)   : 0, 6912867

    section Manual
    This PR (5057) (8.239M)   : 0, 8238898
    master (8.113M)   : 0, 8112975

    section Manual + Automatic
    This PR (5057) (6.220M)   : 0, 6220354
    master (6.193M)   : 0, 6193054

    section Version Conflict
    This PR (5057) (5.707M)   : 0, 5707487
    master (5.718M)   : 0, 5718277
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5057) (10.131M)   : 0, 10130720
    master (10.425M)   : 0, 10425449
    benchmarks/2.9.0 (10.212M)   : 0, 10211874

    section Automatic
    This PR (5057) (7.145M)   : 0, 7144855
    master (7.239M)   : 0, 7239076
    benchmarks/2.9.0 (7.605M)   : 0, 7605074

    section Trace stats
    This PR (5057) (7.484M)   : 0, 7484051
    master (7.593M)   : 0, 7592694

    section Manual
    This PR (5057) (8.857M)   : 0, 8856532
    master (9.072M)   : 0, 9071898

    section Manual + Automatic
    This PR (5057) (6.879M)   : 0, 6878938
    master (7.058M)   : 0, 7057785

    section Version Conflict
    This PR (5057) (6.152M)   : 0, 6152140
    master (6.298M)   : 0, 6298452
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.275M)   : 0, 7275478
    benchmarks/2.9.0 (7.838M)   : 0, 7838185

    section No attack
    master (1.838M)   : 0, 1838305
    benchmarks/2.9.0 (3.252M)   : 0, 3252197

    section Attack
    master (1.464M)   : 0, 1463853
    benchmarks/2.9.0 (2.508M)   : 0, 2508032

    section Blocking
    master (3.176M)   : 0, 3176014

    section IAST default
    master (6.410M)   : 0, 6409716

    section IAST full
    master (5.594M)   : 0, 5594312

    section Base vuln
    master (0.955M)   : 0, 954641

    section IAST vuln
    master (0.875M)   : 0, 874734
Loading

@andrewlock andrewlock force-pushed the andrew/ci/add-static-analysis branch from 6469130 to baf82eb Compare February 16, 2024 17:23
@andrewlock andrewlock merged commit 031a405 into master Feb 29, 2024
53 of 56 checks passed
@andrewlock andrewlock deleted the andrew/ci/add-static-analysis branch February 29, 2024 14:51
@github-actions github-actions bot added this to the vNext milestone Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@markazerdd markazerdd markazerdd approved these changes

@kevingosse kevingosse kevingosse approved these changes

Assignees
No one assigned
Labels
area:builds project files, build scripts, pipelines, versioning, releases, packages
Projects
None yet
Milestone
2.49.0
Development

Successfully merging this pull request may close these issues.
3 participants
@andrewlock @kevingosse @markazerdd