Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Tracer] Elasticsearch Refactor Part 1 (AIT-8764) #4767

Merged
merged 4 commits into from
Nov 1, 2023
Merged

[Tracer] Elasticsearch Refactor Part 1 (AIT-8764) #4767

merged 4 commits into from
Nov 1, 2023

Conversation

zacharycmontoya
Copy link
Collaborator

Summary of changes

Sets up snapshot testing for Elasticsearch and removes the use of Elasticsearch path as a fallback value for the resource name

Reason for change

The main motivation was to see if we could remove the Elasticsearch path as a fallback since the RequestParameters should always be present, and the snapshot tests were added to validate the before/after.

Implementation details

  • Removes the Elasticsearch path as a fallback value for the resource name
  • Refactors the RequestData proxies since the duck type shapes are now consistent between library major versions
  • Adds snapshot testing with normalization of the service name and the Elasticsearch host name

Test coverage

Added snapshot testing

Other details

Part 1 implies a Part 2 😆 With snapshot testing, I want to try a refactor to guarantee that we always have the command name. This can be done in a follow-up PR in case it is unsuccessful

@zacharycmontoya zacharycmontoya requested a review from a team as a code owner October 26, 2023 01:14
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Oct 26, 2023 •
edited
Loading

Datadog Report

Branch report: zach/elasticsearch/refactor-1
Commit report: 5c045df

❄️ dd-trace-dotnet: 132 Failed (0 Known Flaky), 2 New Flaky, 363527 Passed, 1002 Skipped, 47m 50.73s Wall Time

❌ Failed Tests (132)

This report shows up to 5 failed tests.

  • IntegrationDisabled - Datadog.Trace.ClrProfiler.IntegrationTests.Elasticsearch5Tests - Details

    Expand for error 
     application not found: D:\a\1\s\tracer\test\test-applications\integrations\Samples.Elasticsearch.V5\bin\Release\net6.0\Samples.Elasticsearch.V5.dll

  • IntegrationDisabled - Datadog.Trace.ClrProfiler.IntegrationTests.Elasticsearch5Tests - Details

    Expand for error 
     application not found: D:\a\1\s\tracer\test\test-applications\integrations\Samples.Elasticsearch.V5\bin\Release\net7.0\Samples.Elasticsearch.V5.dll

  • IntegrationDisabled - Datadog.Trace.ClrProfiler.IntegrationTests.Elasticsearch5Tests - Details

    Expand for error 
     application not found: D:\a\1\s\tracer\test\test-applications\integrations\Samples.Elasticsearch.V5\bin\Release\net462\Samples.Elasticsearch.V5.exe

  • IntegrationDisabled - Datadog.Trace.ClrProfiler.IntegrationTests.Elasticsearch5Tests - Details

    Expand for error 
     application not found: D:\a\1\s\tracer\test\test-applications\integrations\Samples.Elasticsearch.V5\bin\Release\net462\Samples.Elasticsearch.V5.exe

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.Elasticsearch5Tests - Details

    Expand for error 
     application not found: D:\a\1\s\tracer\test\test-applications\integrations\Samples.Elasticsearch.V5\bin\Release\net462\Samples.Elasticsearch.V5.exe

New Flaky Tests (2)

  • ObfuscateWithDefaultPattern - Datadog.Trace.Tests.Util.Http.QueryStringObfuscatorTests - Last Failure

    Expand for error 
     Expected string to be "http://google.fr/waf?key1=val1&key2=val2&key3=val3" with a length of 50, but "" has a length of 0, differs near "" (index 0).

  • ObfuscateWithDefaultPattern - Datadog.Trace.Tests.Util.Http.QueryStringObfuscatorTests - Last Failure

    Expand for error 
     Expected string to be "http://google.fr/waf?json=%7B%20<redacted>%7D" with a length of 45, but "" has a length of 0, differs near "" (index 0).

@andrewlock
Copy link
Member

andrewlock commented Oct 26, 2023 •
edited
Loading

Benchmarks Report 🐌

Benchmarks for #4767 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.310
  • 3 benchmarks are slower, with geometric mean 1.139
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.02μs 39.5ns 176ns 0.0315 0.0118 0 7.3 KB
master StartStopWithChild netcoreapp3.1 10.3μs 56.4ns 334ns 0.0242 0.00968 0 7.38 KB
master StartStopWithChild net472 15.5μs 55.6ns 215ns 1.29 0.328 0.109 7.66 KB
#4767 StartStopWithChild net6.0 8.02μs 43.1ns 228ns 0.0233 0.0117 0 7.29 KB
#4767 StartStopWithChild netcoreapp3.1 10.2μs 37.9ns 142ns 0.0333 0.0143 0 7.39 KB
#4767 StartStopWithChild net472 15.8μs 40.7ns 158ns 1.27 0.309 0.1 7.66 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 481μs 223ns 862ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 644μs 206ns 798ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 793μs 296ns 1.15μs 0.396 0 0 3.3 KB
#4767 WriteAndFlushEnrichedTraces net6.0 474μs 143ns 515ns 0 0 0 2.7 KB
#4767 WriteAndFlushEnrichedTraces netcoreapp3.1 634μs 326ns 1.26μs 0 0 0 2.7 KB
#4767 WriteAndFlushEnrichedTraces net472 791μs 208ns 777ns 0.393 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 43.1μs 42.2ns 163ns 0.0218 0 0 2.03 KB
master AllCycleSimpleBody netcoreapp3.1 45.6μs 21.5ns 77.5ns 0.0225 0 0 2.01 KB
master AllCycleSimpleBody net472 47μs 19.8ns 76.5ns 0.328 0 0 2.08 KB
master AllCycleMoreComplexBody net6.0 228μs 146ns 566ns 0 0 0 8.63 KB
master AllCycleMoreComplexBody netcoreapp3.1 233μs 63.9ns 247ns 0 0 0 8.52 KB
master AllCycleMoreComplexBody net472 239μs 121ns 469ns 1.3 0 0 8.7 KB
master ObjectExtractorSimpleBody net6.0 119ns 0.0562ns 0.21ns 0.00392 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 173ns 0.235ns 0.91ns 0.00371 0 0 272 B
master ObjectExtractorSimpleBody net472 146ns 0.116ns 0.45ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.07μs 0.85ns 3.18ns 0.055 0 0 3.88 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.19μs 2.87ns 11.1ns 0.0522 0 0 3.78 KB
master ObjectExtractorMoreComplexBody net472 4.13μs 4.99ns 19.3ns 0.618 0.00622 0 3.89 KB
#4767 AllCycleSimpleBody net6.0 44μs 35.4ns 137ns 0.0222 0 0 2.03 KB
#4767 AllCycleSimpleBody netcoreapp3.1 46.2μs 61.1ns 229ns 0.0227 0 0 2.01 KB
#4767 AllCycleSimpleBody net472 46.8μs 38.7ns 140ns 0.327 0 0 2.08 KB
#4767 AllCycleMoreComplexBody net6.0 228μs 57.8ns 209ns 0.114 0 0 8.63 KB
#4767 AllCycleMoreComplexBody netcoreapp3.1 235μs 65.5ns 245ns 0.117 0 0 8.52 KB
#4767 AllCycleMoreComplexBody net472 240μs 129ns 501ns 1.32 0 0 8.7 KB
#4767 ObjectExtractorSimpleBody net6.0 127ns 0.0589ns 0.22ns 0.00394 0 0 280 B
#4767 ObjectExtractorSimpleBody netcoreapp3.1 176ns 0.126ns 0.488ns 0.00374 0 0 272 B
#4767 ObjectExtractorSimpleBody net472 146ns 0.168ns 0.65ns 0.0446 0 0 281 B
#4767 ObjectExtractorMoreComplexBody net6.0 3.02μs 0.918ns 3.44ns 0.0538 0 0 3.88 KB
#4767 ObjectExtractorMoreComplexBody netcoreapp3.1 4.04μs 2.28ns 8.82ns 0.0506 0 0 3.78 KB
#4767 ObjectExtractorMoreComplexBody net472 4.12μs 1.73ns 6.47ns 0.618 0.00616 0 3.89 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 13.1μs 61.7ns 296ns 0.135 0 0 9.42 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 19.4μs 5.58ns 21.6ns 0.126 0 0 9.42 KB
master RunWaf(args=NestedMap (10)) net472 28.8μs 11.7ns 43.8ns 1.5 0 0 9.48 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 76.4μs 49.4ns 191ns 0.191 0 0 15.77 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 86.6μs 156ns 582ns 0.214 0 0 15.72 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 97.9μs 194ns 751ns 2.5 0.0482 0 16.04 KB
master RunWaf(args=NestedMap (100)) net6.0 23.7μs 12.5ns 48.6ns 0.273 0 0 19.66 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 38.6μs 15.6ns 58.3ns 0.269 0 0 20.42 KB
master RunWaf(args=NestedMap (100)) net472 55.3μs 48.8ns 189ns 3.27 0.0277 0 20.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 91μs 57.7ns 223ns 0.363 0 0 26.01 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 108μs 153ns 553ns 0.323 0 0 26.72 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 131μs 151ns 584ns 4.29 0.065 0 27.19 KB
master RunWaf(args=NestedMap (20)) net6.0 23.7μs 10.9ns 42.2ns 0.272 0 0 19.4 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 38μs 15.5ns 59.9ns 0.266 0 0 19.84 KB
master RunWaf(args=NestedMap (20)) net472 56.5μs 52.6ns 204ns 3.17 0.0566 0 20.04 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 92.8μs 59.5ns 231ns 0.322 0 0 25.74 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 106μs 200ns 721ns 0.319 0 0 26.14 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 132μs 208ns 805ns 4.19 0.0665 0 26.6 KB
#4767 RunWaf(args=NestedMap (10)) net6.0 12.7μs 5.88ns 22ns 0.133 0 0 9.42 KB
#4767 RunWaf(args=NestedMap (10)) netcoreapp3.1 19.6μs 6.97ns 27ns 0.127 0 0 9.42 KB
#4767 RunWaf(args=NestedMap (10)) net472 28.5μs 12.6ns 43.6ns 1.49 0 0 9.48 KB
#4767 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 77.4μs 42.2ns 164ns 0.193 0 0 15.77 KB
#4767 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 84.9μs 72.9ns 282ns 0.215 0 0 15.72 KB
#4767 RunWafWithAttack(args=Neste(...)tack) [22]) net472 99.2μs 239ns 924ns 2.54 0 0 16.04 KB
#4767 RunWaf(args=NestedMap (100)) net6.0 23.8μs 9.36ns 33.8ns 0.274 0 0 19.66 KB
#4767 RunWaf(args=NestedMap (100)) netcoreapp3.1 37.9μs 17.4ns 65.1ns 0.284 0 0 20.42 KB
#4767 RunWaf(args=NestedMap (100)) net472 54.5μs 41.7ns 150ns 3.27 0.0273 0 20.63 KB
#4767 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 91μs 31.3ns 121ns 0.363 0 0 26.01 KB
#4767 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 109μs 74.8ns 280ns 0.325 0 0 26.72 KB
#4767 RunWafWithAttack(args=Neste(...)tack) [23]) net472 132μs 136ns 528ns 4.26 0.0655 0 27.19 KB
#4767 RunWaf(args=NestedMap (20)) net6.0 24.1μs 9.65ns 37.4ns 0.264 0 0 19.4 KB
#4767 RunWaf(args=NestedMap (20)) netcoreapp3.1 37.7μs 9.65ns 36.1ns 0.264 0 0 19.84 KB
#4767 RunWaf(args=NestedMap (20)) net472 54.4μs 75.9ns 294ns 3.16 0.054 0 20.04 KB
#4767 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 92.6μs 121ns 469ns 0.325 0 0 25.74 KB
#4767 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 107μs 41.8ns 145ns 0.326 0 0 26.14 KB
#4767 RunWafWithAttack(args=Neste(...)tack) [22]) net472 129μs 131ns 507ns 4.19 0.0644 0 26.6 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 170μs 160ns 618ns 0.17 0 0 18.04 KB
master SendRequest netcoreapp3.1 187μs 201ns 780ns 0.188 0 0 20.2 KB
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#4767 SendRequest net6.0 170μs 170ns 660ns 0.256 0 0 18.04 KB
#4767 SendRequest netcoreapp3.1 191μs 430ns 1.66μs 0.187 0 0 20.2 KB
#4767 SendRequest net472 3.88E‑05ns 3.88E‑05ns 0.00015ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 566μs 385ns 1.49μs 0.558 0 0 41.77 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 666μs 858ns 3.32μs 0.332 0 0 41.81 KB
master WriteAndFlushEnrichedTraces net472 845μs 3.24μs 12.5μs 8.33 2.5 0.417 53.26 KB
#4767 WriteAndFlushEnrichedTraces net6.0 545μs 2.73μs 11.6μs 0.551 0 0 41.81 KB
#4767 WriteAndFlushEnrichedTraces netcoreapp3.1 663μs 605ns 2.34μs 0.334 0 0 41.64 KB
#4767 WriteAndFlushEnrichedTraces net472 856μs 3.89μs 15.5μs 8.39 2.52 0.419 53.23 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.07μs 0.778ns 3.01ns 0.0107 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.3μs 1.33ns 5.13ns 0.0104 0 0 768 B
master ExecuteNonQuery net472 1.67μs 0.439ns 1.64ns 0.116 0 0 730 B
#4767 ExecuteNonQuery net6.0 997ns 0.925ns 3.58ns 0.0105 0 0 768 B
#4767 ExecuteNonQuery netcoreapp3.1 1.25μs 1.11ns 3.84ns 0.00987 0 0 768 B
#4767 ExecuteNonQuery net472 1.7μs 0.579ns 2.24ns 0.115 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.07μs 0.449ns 1.68ns 0.0129 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.41μs 1.22ns 4.73ns 0.0126 0 0 936 B
master CallElasticsearch net472 2.39μs 1.22ns 4.74ns 0.151 0 0 955 B
master CallElasticsearchAsync net6.0 1.22μs 0.915ns 3.43ns 0.0128 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.47μs 2.28ns 8.83ns 0.0132 0 0 984 B
master CallElasticsearchAsync net472 2.36μs 1.3ns 5.05ns 0.16 0 0 1.01 KB
#4767 CallElasticsearch net6.0 1.19μs 0.444ns 1.66ns 0.0131 0 0 936 B
#4767 CallElasticsearch netcoreapp3.1 1.27μs 0.354ns 1.33ns 0.0127 0 0 936 B
#4767 CallElasticsearch net472 2.33μs 0.65ns 2.52ns 0.152 0 0 955 B
#4767 CallElasticsearchAsync net6.0 1.28μs 0.734ns 2.75ns 0.0125 0 0 912 B
#4767 CallElasticsearchAsync netcoreapp3.1 1.48μs 0.45ns 1.68ns 0.0133 0 0 984 B
#4767 CallElasticsearchAsync net472 2.43μs 0.482ns 1.8ns 0.16 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.13μs 1.36ns 5.1ns 0.013 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.47μs 0.533ns 1.99ns 0.0124 0 0 912 B
master ExecuteAsync net472 1.77μs 0.747ns 2.8ns 0.138 0 0 875 B
#4767 ExecuteAsync net6.0 1.22μs 1.62ns 6.28ns 0.0131 0 0 912 B
#4767 ExecuteAsync netcoreapp3.1 1.44μs 0.748ns 2.7ns 0.0125 0 0 912 B
#4767 ExecuteAsync net472 1.76μs 0.264ns 0.99ns 0.138 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 3.78μs 1.46ns 5.67ns 0.0263 0 0 1.9 KB
master SendAsync netcoreapp3.1 4.4μs 3.23ns 12.5ns 0.0329 0 0 2.43 KB
master SendAsync net472 7.03μs 2.21ns 8.26ns 0.473 0 0 2.99 KB
#4767 SendAsync net6.0 3.72μs 2.47ns 9.58ns 0.026 0 0 1.9 KB
#4767 SendAsync netcoreapp3.1 4.43μs 0.749ns 2.7ns 0.0313 0 0 2.43 KB
#4767 SendAsync net472 7.11μs 3.55ns 13.7ns 0.473 0 0 2.99 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.37μs 0.349ns 1.31ns 0.022 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 1.97μs 0.827ns 3.09ns 0.0217 0 0 1.57 KB
master EnrichedLog net472 2.16μs 2.05ns 7.94ns 0.236 0 0 1.49 KB
#4767 EnrichedLog net6.0 1.42μs 0.851ns 3.18ns 0.022 0 0 1.57 KB
#4767 EnrichedLog netcoreapp3.1 1.95μs 2.03ns 7.59ns 0.0213 0 0 1.57 KB
#4767 EnrichedLog net472 2.32μs 2.67ns 9.98ns 0.236 0 0 1.49 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 111μs 103ns 399ns 0.0552 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 119μs 151ns 584ns 0.0589 0 0 4.21 KB
master EnrichedLog net472 150μs 139ns 537ns 0.674 0.225 0 4.38 KB
#4767 EnrichedLog net6.0 113μs 160ns 619ns 0.0557 0 0 4.21 KB
#4767 EnrichedLog netcoreapp3.1 118μs 215ns 831ns 0 0 0 4.21 KB
#4767 EnrichedLog net472 147μs 111ns 431ns 0.66 0.22 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.02μs 0.867ns 3.24ns 0.0287 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 3.89μs 1.01ns 3.77ns 0.0292 0 0 2.13 KB
master EnrichedLog net472 4.5μs 1.21ns 4.51ns 0.307 0 0 1.93 KB
#4767 EnrichedLog net6.0 2.77μs 0.865ns 3.35ns 0.0291 0 0 2.13 KB
#4767 EnrichedLog netcoreapp3.1 3.78μs 1.19ns 4.46ns 0.0285 0 0 2.13 KB
#4767 EnrichedLog net472 4.57μs 2.18ns 8.44ns 0.305 0 0 1.93 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.27μs 0.727ns 2.72ns 0.0152 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.6μs 0.704ns 2.54ns 0.0152 0 0 1.1 KB
master SendReceive net472 2.01μs 1.94ns 7ns 0.177 0 0 1.12 KB
#4767 SendReceive net6.0 1.27μs 4.63ns 17.9ns 0.0157 0 0 1.1 KB
#4767 SendReceive netcoreapp3.1 1.59μs 0.672ns 2.6ns 0.0149 0 0 1.1 KB
#4767 SendReceive net472 1.91μs 0.937ns 3.38ns 0.177 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.51μs 0.606ns 2.35ns 0.0213 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.76μs 2.73ns 10.6ns 0.0202 0 0 1.58 KB
master EnrichedLog net472 4.03μs 1.26ns 4.73ns 0.31 0 0 1.96 KB
#4767 EnrichedLog net6.0 2.57μs 1.86ns 7.22ns 0.0217 0 0 1.53 KB
#4767 EnrichedLog netcoreapp3.1 3.73μs 1.21ns 4.7ns 0.0205 0 0 1.58 KB
#4767 EnrichedLog net472 4μs 1.39ns 5.21ns 0.311 0 0 1.96 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4767

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.148 671.38 770.67
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 1.139 774.32 881.75

Faster 🎉 in #4767

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 1.310 652.11 497.77

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 476ns 0.23ns 0.86ns 0.00747 0 0 536 B
master StartFinishSpan netcoreapp3.1 652ns 0.221ns 0.826ns 0.00717 0 0 536 B
master StartFinishSpan net472 699ns 0.137ns 0.529ns 0.0853 0 0 538 B
master StartFinishScope net6.0 486ns 0.276ns 1.07ns 0.00929 0 0 656 B
master StartFinishScope netcoreapp3.1 672ns 0.24ns 0.866ns 0.00877 0 0 656 B
master StartFinishScope net472 775ns 0.236ns 0.915ns 0.0982 0 0 618 B
#4767 StartFinishSpan net6.0 459ns 0.321ns 1.2ns 0.00737 0 0 536 B
#4767 StartFinishSpan netcoreapp3.1 498ns 0.15ns 0.56ns 0.00727 0 0 536 B
#4767 StartFinishSpan net472 707ns 0.178ns 0.667ns 0.0852 0 0 538 B
#4767 StartFinishScope net6.0 471ns 0.118ns 0.442ns 0.00912 0 0 656 B
#4767 StartFinishScope netcoreapp3.1 770ns 0.265ns 1.02ns 0.00886 0 0 656 B
#4767 StartFinishScope net472 881ns 0.417ns 1.61ns 0.098 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4767

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 1.130 963.34 1,088.33

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 560ns 0.165ns 0.639ns 0.00922 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 859ns 0.26ns 1.01ns 0.00895 0 0 656 B
master RunOnMethodBegin net472 963ns 0.32ns 1.24ns 0.0982 0 0 618 B
#4767 RunOnMethodBegin net6.0 533ns 0.541ns 2.1ns 0.00931 0 0 656 B
#4767 RunOnMethodBegin netcoreapp3.1 804ns 0.167ns 0.624ns 0.00891 0 0 656 B
#4767 RunOnMethodBegin net472 1.09μs 0.261ns 1.01ns 0.0978 0 0 618 B

@zacharycmontoya zacharycmontoya force-pushed the zach/elasticsearch/refactor-1 branch from fca6399 to e4845ee Compare October 26, 2023 16:28
@andrewlock
Copy link
Member

andrewlock commented Oct 26, 2023 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4767) - mean (71ms)  : 62, 80
     .   : milestone, 71,
    master - mean (72ms)  : 62, 82
     .   : milestone, 72,

    section CallTarget+Inlining+NGEN
    This PR (4767) - mean (993ms)  : 971, 1015
     .   : milestone, 993,
    master - mean (1,000ms)  : 982, 1018
     .   : milestone, 1000,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4767) - mean (106ms)  : 102, 109
     .   : milestone, 106,
    master - mean (106ms)  : 103, 109
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (4767) - mean (685ms)  : 666, 704
     .   : milestone, 685,
    master - mean (691ms)  : 670, 713
     .   : milestone, 691,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4767) - mean (89ms)  : 86, 93
     .   : milestone, 89,
    master - mean (90ms)  : 88, 92
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (4767) - mean (654ms)  : 626, 683
     .   : milestone, 654,
    master - mean (655ms)  : 634, 677
     .   : milestone, 655,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4767) - mean (188ms)  : 185, 191
     .   : milestone, 188,
    master - mean (189ms)  : 186, 191
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (4767) - mean (1,140ms)  : 1118, 1162
     .   : milestone, 1140,
    master - mean (1,153ms)  : 1132, 1175
     .   : milestone, 1153,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4767) - mean (272ms)  : 268, 276
     .   : milestone, 272,
    master - mean (273ms)  : 269, 277
     .   : milestone, 273,

    section CallTarget+Inlining+NGEN
    This PR (4767) - mean (1,097ms)  : 1072, 1121
     .   : milestone, 1097,
    master - mean (1,103ms)  : 1077, 1129
     .   : milestone, 1103,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4767) - mean (262ms)  : 259, 265
     .   : milestone, 262,
    master - mean (263ms)  : 260, 267
     .   : milestone, 263,

    section CallTarget+Inlining+NGEN
    This PR (4767) - mean (1,064ms)  : 1037, 1090
     .   : milestone, 1064,
    master - mean (1,072ms)  : 1044, 1100
     .   : milestone, 1072,
Loading

@andrewlock
Copy link
Member

andrewlock commented Oct 26, 2023 •
edited
Loading

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4767) (10.624M)   : 0, 10623596
    master (10.858M)   : 0, 10858226
    benchmarks/2.38.0 (11.867M)   : 0, 11866956
    benchmarks/2.9.0 (11.240M)   : 0, 11240287

    section Automatic
    This PR (4767) (7.356M)   : 0, 7356191
    master (7.460M)   : 0, 7460403
    benchmarks/2.38.0 (8.176M)   : 0, 8175636
    benchmarks/2.9.0 (7.993M)   : 0, 7992575

    section Trace stats
    This PR (4767) (7.650M)   : 0, 7650447
    master (7.731M)   : 0, 7731452
    benchmarks/2.38.0 (8.450M)   : 0, 8450370

    section Manual
    This PR (4767) (9.419M)   : 0, 9418778
    master (9.543M)   : 0, 9542867
    benchmarks/2.38.0 (10.334M)   : 0, 10334368

    section Manual + Automatic
    This PR (4767) (7.073M)   : 0, 7073127
    master (6.981M)   : 0, 6981252
    benchmarks/2.38.0 (7.750M)   : 0, 7750484

    section Version Conflict
    This PR (4767) (6.510M)   : 0, 6510046
    master (6.546M)   : 0, 6546236
    benchmarks/2.38.0 (7.137M)   : 0, 7136691
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4767) (9.804M)   : 0, 9804346
    master (9.578M)   : 0, 9578480
    benchmarks/2.38.0 (9.548M)   : 0, 9548121
    benchmarks/2.9.0 (9.726M)   : 0, 9725615

    section Automatic
    This PR (4767) (6.663M)   : 0, 6662658
    master (6.752M)   : 0, 6752023
    benchmarks/2.38.0 (6.747M)   : 0, 6747227

    section Trace stats
    This PR (4767) (6.848M)   : 0, 6847774
    master (7.056M)   : 0, 7055624
    benchmarks/2.38.0 (6.815M)   : 0, 6814846

    section Manual
    This PR (4767) (8.388M)   : 0, 8388174
    master (8.307M)   : 0, 8306680
    benchmarks/2.38.0 (8.263M)   : 0, 8263131

    section Manual + Automatic
    This PR (4767) (6.395M)   : 0, 6394821
    master (6.217M)   : 0, 6216711
    benchmarks/2.38.0 (6.275M)   : 0, 6275411

    section Version Conflict
    This PR (4767) (5.720M)   : 0, 5719789
    master (5.921M)   : 0, 5921235
    benchmarks/2.38.0 (5.670M)   : 0, 5669744
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4767) (9.523M)   : 0, 9523407
    master (9.485M)   : 0, 9485335
    benchmarks/2.38.0 (9.007M)   : 0, 9007066
    benchmarks/2.9.0 (9.713M)   : 0, 9713176

    section Automatic
    This PR (4767) (6.727M)   : 0, 6726896
    master (6.717M)   : 0, 6716658
    benchmarks/2.38.0 (6.423M)   : 0, 6422756
    benchmarks/2.9.0 (6.972M)   : 0, 6971870

    section Trace stats
    This PR (4767) (7.027M)   : 0, 7026789
    master (6.967M)   : 0, 6966783
    benchmarks/2.38.0 (6.721M)   : 0, 6720838

    section Manual
    This PR (4767) (8.419M)   : 0, 8418633
    master (8.322M)   : 0, 8321923
    benchmarks/2.38.0 (7.977M)   : 0, 7977494

    section Manual + Automatic
    This PR (4767) (6.484M)   : 0, 6483998
    master (6.475M)   : 0, 6474564
    benchmarks/2.38.0 (6.349M)   : 0, 6349066

    section Version Conflict
    This PR (4767) (5.974M)   : 0, 5974203
    master (5.862M)   : 0, 5862141
    benchmarks/2.38.0 (5.682M)   : 0, 5681654
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.688M)   : 0, 7687660
    benchmarks/2.38.0 (7.578M)   : 0, 7577544
    benchmarks/2.9.0 (8.065M)   : 0, 8064881

    section No attack
    master (2.239M)   : 0, 2239244
    benchmarks/2.38.0 (2.184M)   : 0, 2184279
    benchmarks/2.9.0 (3.354M)   : 0, 3354146

    section Attack
    master (1.779M)   : 0, 1778952
    benchmarks/2.38.0 (1.711M)   : 0, 1710542
    benchmarks/2.9.0 (2.662M)   : 0, 2661593

    section Blocking
    master (3.553M)   : 0, 3553184
    benchmarks/2.38.0 (3.486M)   : 0, 3486124

    section IAST default
    master (6.995M)   : 0, 6995161

    section IAST full
    master (6.364M)   : 0, 6363731

    section Base vuln
    master (0.982M)   : 0, 982038

    section IAST vuln
    master (0.930M)   : 0, 929607
Loading

andrewlock
andrewlock approved these changes Oct 27, 2023
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks! I didn't scrutinize the snapshots too closely, but they LGTM. The only potential issue may be flakiness around sort-order. I think we'll need to update the span order to sort by resource name and elasticsearch.url tag

tracer/test/snapshots/Elasticsearch5Tests.SubmitsTraces_SchemaV0.verified.txt Outdated Show resolved Hide resolved
pablomartinezbernardo
pablomartinezbernardo approved these changes Oct 31, 2023
View reviewed changes
Copy link
Contributor

@pablomartinezbernardo pablomartinezbernardo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@zacharycmontoya zacharycmontoya force-pushed the zach/elasticsearch/refactor-1 branch from 7eed1be to 5c045df Compare October 31, 2023 23:06
andrewlock
andrewlock approved these changes Nov 1, 2023
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 🤖

Quick thought - did you a do a "full package versions" run for these?

@zacharycmontoya
Copy link
Collaborator Author

🚫 🤖

Quick thought - did you a do a "full package versions" run for these?

I ran a separate "full package versions" job for each of the Elasticsearch test applications and the linux integration tests passed 👍🏼

@zacharycmontoya zacharycmontoya merged commit 272ac31 into master Nov 1, 2023
@zacharycmontoya zacharycmontoya deleted the zach/elasticsearch/refactor-1 branch November 1, 2023 19:02
@github-actions github-actions bot added this to the vNext milestone Nov 1, 2023
@andrewlock andrewlock added area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) area:tests unit tests, integration tests area:integrations labels Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pablomartinezbernardo pablomartinezbernardo pablomartinezbernardo approved these changes

@andrewlock andrewlock andrewlock approved these changes

@lucaspimentel lucaspimentel lucaspimentel approved these changes

Assignees
No one assigned
Labels
area:integrations area:tests unit tests, integration tests area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations)
Projects
None yet
Milestone
2.41.0
Development

Successfully merging this pull request may close these issues.
4 participants
@zacharycmontoya @andrewlock @lucaspimentel @pablomartinezbernardo