Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ASM] Skip integration tests using Microsoft.Data.Sqlite on netcoreapp3.0 #6342

Merged
merged 1 commit into from
Nov 26, 2024
Merged

[ASM] Skip integration tests using Microsoft.Data.Sqlite on netcoreapp3.0 #6342

merged 1 commit into from
Nov 26, 2024

Conversation

e-n-0
Copy link
Member

@e-n-0 e-n-0 commented Nov 25, 2024 •
edited
Loading

Summary of changes

Skip integration tests using Microsoft.Data.Sqlite on netcoreapp3.0.

Reason for change

On the CI, the tests using the sqlite databse using Microsoft.Data.Sqlite on netcoreapp3.0 were failing.

More info

The reason behind these fails is that the alpine container on netcoreapp3.0 don't find the location of the sqlite binaries (maybe an error with musl). Forcefully running the test with --runtime linux-musl-x64 kind of fix it, but we won't do that and just skip it for that PR.

@github-actions github-actions bot added area:tests unit tests, integration tests area:asm labels Nov 25, 2024
@andrewlock
Copy link
Member

andrewlock commented Nov 25, 2024 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6342) - mean (77ms)  : 66, 89
     .   : milestone, 77,
    master - mean (72ms)  : 63, 80
     .   : milestone, 72,

    section CallTarget+Inlining+NGEN
    This PR (6342) - mean (1,023ms)  : 984, 1062
     .   : milestone, 1023,
    master - mean (990ms)  : 964, 1016
     .   : milestone, 990,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6342) - mean (115ms)  : 112, 117
     .   : milestone, 115,
    master - mean (108ms)  : 105, 110
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6342) - mean (703ms)  : 688, 718
     .   : milestone, 703,
    master - mean (682ms)  : 667, 698
     .   : milestone, 682,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6342) - mean (96ms)  : 93, 99
     .   : milestone, 96,
    master - mean (91ms)  : 89, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6342) - mean (653ms)  : 632, 674
     .   : milestone, 653,
    master - mean (637ms)  : 623, 651
     .   : milestone, 637,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6342) - mean (193ms)  : 188, 197
     .   : milestone, 193,
    master - mean (191ms)  : 187, 196
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (6342) - mean (1,100ms)  : 1073, 1128
     .   : milestone, 1100,
    master - mean (1,096ms)  : 1060, 1132
     .   : milestone, 1096,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6342) - mean (277ms)  : 272, 281
     .   : milestone, 277,
    master - mean (276ms)  : 272, 280
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (6342) - mean (884ms)  : 848, 919
     .   : milestone, 884,
    master - mean (873ms)  : 844, 903
     .   : milestone, 873,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6342) - mean (266ms)  : 261, 270
     .   : milestone, 266,
    master - mean (266ms)  : 262, 269
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (6342) - mean (851ms)  : 816, 887
     .   : milestone, 851,
    master - mean (862ms)  : 829, 895
     .   : milestone, 862,
Loading

@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Nov 25, 2024 •
edited
Loading

Datadog Report

Branch report: flavien/iast/fix-ci-sqlite-core3
Commit report: eeb569f
Test service: dd-trace-dotnet

✅ 0 Failed, 449329 Passed, 2753 Skipped, 20h 47m 57.69s Total Time
❄️ 1 New Flaky

New Flaky Tests (1)

  • SeparatesLogsIntoBatches - Datadog.Trace.Tests.Telemetry.Collectors.RedactedErrorLogCollectorTests - Last Failure

    Expand for error 
     Expected logs.Sum(x => x.Count) to be 556, but found 547 (difference of -9).

@andrewlock
Copy link
Member

andrewlock commented Nov 25, 2024 •
edited
Loading

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6342) (11.082M)   : 0, 11081898
    master (11.216M)   : 0, 11215877
    benchmarks/2.9.0 (11.033M)   : 0, 11032866

    section Automatic
    This PR (6342) (7.251M)   : 0, 7251088
    master (7.356M)   : 0, 7355647
    benchmarks/2.9.0 (7.786M)   : 0, 7785853

    section Trace stats
    master (7.653M)   : 0, 7652612

    section Manual
    master (11.177M)   : 0, 11177132

    section Manual + Automatic
    This PR (6342) (6.748M)   : 0, 6747609
    master (6.722M)   : 0, 6722247

    section DD_TRACE_ENABLED=0
    master (10.246M)   : 0, 10246375
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6342) (9.685M)   : 0, 9685108
    master (9.495M)   : 0, 9495372
    benchmarks/2.9.0 (9.495M)   : 0, 9494821

    section Automatic
    This PR (6342) (6.315M)   : 0, 6314526
    master (6.445M)   : 0, 6445315

    section Trace stats
    master (6.617M)   : 0, 6617151

    section Manual
    master (9.564M)   : 0, 9564169

    section Manual + Automatic
    This PR (6342) (6.015M)   : 0, 6014978
    master (5.943M)   : 0, 5943103

    section DD_TRACE_ENABLED=0
    master (8.993M)   : 0, 8992941
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6342) (8.978M)   : 0, 8978010
    master (8.980M)   : 0, 8980408
    benchmarks/2.9.0 (10.020M)   : 0, 10019592

    section Automatic
    This PR (6342) (5.734M)   : 0, 5734086
    master (5.865M)   : 0, 5865070
    benchmarks/2.9.0 (7.255M)   : 0, 7255257

    section Trace stats
    master (6.446M)   : 0, 6446474

    section Manual
    master (9.151M)   : 0, 9151083

    section Manual + Automatic
    This PR (6342) (5.588M)   : 0, 5588115
    master (5.595M)   : 0, 5594798

    section DD_TRACE_ENABLED=0
    master (8.375M)   : 0, 8375216
Loading

@andrewlock
Copy link
Member

andrewlock commented Nov 25, 2024 •
edited
Loading

Benchmarks Report for appsec 🐌

Benchmarks for #6342 compared to master:

  • 2 benchmarks are slower, with geometric mean 1.490
  • 2 benchmarks have fewer allocations
  • 2 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6342

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 1.190 203.60 242.22

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 219μs 132ns 513ns 2.5 0 0 176.82 KB
master AllCycleSimpleBody netcoreapp3.1 322μs 444ns 1.6μs 2.4 0 0 184.26 KB
master AllCycleSimpleBody net472 280μs 329ns 1.27μs 33.2 2.57 0 210.04 KB
master AllCycleMoreComplexBody net6.0 222μs 124ns 462ns 2.55 0 0 180.33 KB
master AllCycleMoreComplexBody netcoreapp3.1 323μs 242ns 907ns 2.45 0 0 187.68 KB
master AllCycleMoreComplexBody net472 287μs 328ns 1.23μs 33.8 2.57 0 213.56 KB
master ObjectExtractorSimpleBody net6.0 148ns 0.0921ns 0.345ns 0.00392 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 204ns 0.156ns 0.603ns 0.00371 0 0 272 B
master ObjectExtractorSimpleBody net472 215ns 0.285ns 1.07ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.04μs 1.78ns 6.66ns 0.053 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.06μs 2.04ns 7.62ns 0.0508 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 4.58μs 4.66ns 18ns 0.603 0.00667 0 3.8 KB
#6342 AllCycleSimpleBody net6.0 224μs 221ns 854ns 2.46 0 0 176.82 KB
#6342 AllCycleSimpleBody netcoreapp3.1 323μs 249ns 930ns 2.4 0 0 184.27 KB
#6342 AllCycleSimpleBody net472 283μs 187ns 724ns 33.4 2.55 0 210.05 KB
#6342 AllCycleMoreComplexBody net6.0 227μs 167ns 625ns 2.5 0 0 180.33 KB
#6342 AllCycleMoreComplexBody netcoreapp3.1 329μs 342ns 1.32μs 2.47 0 0 187.68 KB
#6342 AllCycleMoreComplexBody net472 305μs 251ns 938ns 33.9 2.6 0 213.56 KB
#6342 ObjectExtractorSimpleBody net6.0 144ns 0.16ns 0.621ns 0.00394 0 0 280 B
#6342 ObjectExtractorSimpleBody netcoreapp3.1 242ns 0.18ns 0.696ns 0.0037 0 0 272 B
#6342 ObjectExtractorSimpleBody net472 213ns 0.215ns 0.834ns 0.0446 0 0 281 B
#6342 ObjectExtractorMoreComplexBody net6.0 3.06μs 1.12ns 4.19ns 0.0524 0 0 3.78 KB
#6342 ObjectExtractorMoreComplexBody netcoreapp3.1 3.92μs 3.35ns 12.1ns 0.051 0 0 3.69 KB
#6342 ObjectExtractorMoreComplexBody net472 4.83μs 6.68ns 25.9ns 0.602 0.00477 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 38.4μs 41.9ns 162ns 0.46 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 53.9μs 32.9ns 128ns 0.43 0 0 32.4 KB
master EncodeArgs net472 67.8μs 38.3ns 143ns 5.14 0.0677 0 32.5 KB
master EncodeLegacyArgs net6.0 78.1μs 26.7ns 99.7ns 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 104μs 85.5ns 308ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 157μs 141ns 547ns 0.313 0 0 2.15 KB
#6342 EncodeArgs net6.0 37.6μs 28.6ns 111ns 0.446 0 0 32.4 KB
#6342 EncodeArgs netcoreapp3.1 54μs 26.3ns 102ns 0.43 0 0 32.4 KB
#6342 EncodeArgs net472 67.7μs 62.7ns 235ns 5.15 0.0678 0 32.5 KB
#6342 EncodeLegacyArgs net6.0 73.7μs 133ns 513ns 0.0362 0 0 2.14 KB
#6342 EncodeLegacyArgs netcoreapp3.1 106μs 245ns 918ns 0 0 0 2.14 KB
#6342 EncodeLegacyArgs net472 157μs 104ns 390ns 0.313 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 185μs 193ns 748ns 0 0 0 2.44 KB
master RunWafRealisticBenchmark netcoreapp3.1 195μs 274ns 1.06μs 0 0 0 2.39 KB
master RunWafRealisticBenchmark net472 210μs 84.1ns 326ns 0.314 0 0 2.46 KB
master RunWafRealisticBenchmarkWithAttack net6.0 123μs 111ns 429ns 0 0 0 1.47 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 130μs 181ns 702ns 0 0 0 1.46 KB
master RunWafRealisticBenchmarkWithAttack net472 139μs 103ns 398ns 0.209 0 0 1.49 KB
#6342 RunWafRealisticBenchmark net6.0 184μs 104ns 403ns 0 0 0 2.44 KB
#6342 RunWafRealisticBenchmark netcoreapp3.1 196μs 176ns 657ns 0 0 0 2.39 KB
#6342 RunWafRealisticBenchmark net472 209μs 184ns 714ns 0.314 0 0 2.46 KB
#6342 RunWafRealisticBenchmarkWithAttack net6.0 123μs 83.2ns 322ns 0 0 0 1.47 KB
#6342 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 131μs 117ns 455ns 0 0 0 1.46 KB
#6342 RunWafRealisticBenchmarkWithAttack net472 140μs 69.8ns 270ns 0.21 0 0 1.48 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #6342

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 1.866 342,000.00 638,150.00 bimodal

More allocations ⚠️ in #6342

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 252.74 KB 325.31 KB 72.57 KB 28.71%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 254.36 KB 263.26 KB 8.9 KB 3.50%

Fewer allocations 🎉 in #6342

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 278.53 KB 275.68 KB -2.85 KB -1.02%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 62.23 KB 57.34 KB -4.89 KB -7.85%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 59.7μs 704ns 7μs 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 58.1μs 672ns 6.62μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 38μs 169ns 633ns 0 0 0 62.23 KB
master StringConcatAspectBenchmark net6.0 313μs 1.7μs 10.5μs 0 0 0 254.36 KB
master StringConcatAspectBenchmark netcoreapp3.1 348μs 2μs 15.4μs 0 0 0 252.74 KB
master StringConcatAspectBenchmark net472 282μs 5.25μs 51.2μs 0 0 0 278.53 KB
#6342 StringConcatBenchmark net6.0 57.4μs 564ns 5.41μs 0 0 0 43.44 KB
#6342 StringConcatBenchmark netcoreapp3.1 62.6μs 836ns 8.31μs 0 0 0 42.64 KB
#6342 StringConcatBenchmark net472 37.9μs 167ns 625ns 0 0 0 57.34 KB
#6342 StringConcatAspectBenchmark net6.0 307μs 1.6μs 10.1μs 0 0 0 263.26 KB
#6342 StringConcatAspectBenchmark netcoreapp3.1 638μs 3.87μs 37.6μs 0 0 0 325.31 KB
#6342 StringConcatAspectBenchmark net472 301μs 7μs 68.6μs 0 0 0 275.68 KB

@andrewlock
Copy link
Member

andrewlock commented Nov 25, 2024 •
edited
Loading

Benchmarks Report for tracer 🐌

Benchmarks for #6342 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.141
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.97μs 45.4ns 346ns 0.0156 0.0078 0 5.62 KB
master StartStopWithChild netcoreapp3.1 9.98μs 53ns 347ns 0.0142 0.00475 0 5.8 KB
master StartStopWithChild net472 16.2μs 38.1ns 148ns 1.04 0.309 0.103 6.22 KB
#6342 StartStopWithChild net6.0 7.74μs 40ns 226ns 0.0156 0.00778 0 5.6 KB
#6342 StartStopWithChild netcoreapp3.1 9.94μs 54.4ns 308ns 0.0201 0.0101 0 5.8 KB
#6342 StartStopWithChild net472 16.2μs 43.5ns 169ns 1.04 0.299 0.0971 6.21 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 491μs 235ns 880ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 667μs 298ns 1.15μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 855μs 589ns 2.2μs 0.425 0 0 3.3 KB
#6342 WriteAndFlushEnrichedTraces net6.0 512μs 479ns 1.86μs 0 0 0 2.7 KB
#6342 WriteAndFlushEnrichedTraces netcoreapp3.1 659μs 602ns 2.33μs 0 0 0 2.7 KB
#6342 WriteAndFlushEnrichedTraces net472 842μs 443ns 1.72μs 0.419 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 154μs 878ns 7.13μs 0.147 0 0 14.47 KB
master SendRequest netcoreapp3.1 166μs 975ns 9.35μs 0.168 0 0 17.27 KB
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#6342 SendRequest net6.0 154μs 1.16μs 11.6μs 0.142 0 0 14.47 KB
#6342 SendRequest netcoreapp3.1 175μs 1.17μs 11.5μs 0.177 0 0 17.27 KB
#6342 SendRequest net472 0.00352ns 0.00135ns 0.00525ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 588μs 3.28μs 21.3μs 0.566 0 0 41.66 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 692μs 3.8μs 21.8μs 0.338 0 0 41.9 KB
master WriteAndFlushEnrichedTraces net472 863μs 4.48μs 21.5μs 8.33 2.5 0.417 53.3 KB
#6342 WriteAndFlushEnrichedTraces net6.0 567μs 2.81μs 11.6μs 0.556 0 0 41.66 KB
#6342 WriteAndFlushEnrichedTraces netcoreapp3.1 708μs 4.05μs 39.7μs 0.388 0 0 41.82 KB
#6342 WriteAndFlushEnrichedTraces net472 869μs 3.33μs 12.9μs 8.45 2.53 0.422 53.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.34μs 0.956ns 3.7ns 0.0141 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.79μs 1.07ns 4.14ns 0.0134 0 0 1.02 KB
master ExecuteNonQuery net472 2.14μs 1.9ns 7.37ns 0.156 0.00108 0 987 B
#6342 ExecuteNonQuery net6.0 1.36μs 1.1ns 3.97ns 0.0143 0 0 1.02 KB
#6342 ExecuteNonQuery netcoreapp3.1 1.73μs 1.3ns 5.05ns 0.0139 0 0 1.02 KB
#6342 ExecuteNonQuery net472 2.12μs 2.42ns 9.37ns 0.156 0.00106 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.16μs 0.422ns 1.52ns 0.0134 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.57μs 1.38ns 5.18ns 0.0133 0 0 976 B
master CallElasticsearch net472 2.6μs 1.99ns 7.7ns 0.158 0 0 995 B
master CallElasticsearchAsync net6.0 1.28μs 0.477ns 1.85ns 0.0133 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.66μs 0.798ns 2.98ns 0.014 0 0 1.02 KB
master CallElasticsearchAsync net472 2.68μs 2.46ns 9.53ns 0.166 0 0 1.05 KB
#6342 CallElasticsearch net6.0 1.13μs 2.06ns 7.97ns 0.0138 0 0 976 B
#6342 CallElasticsearch netcoreapp3.1 1.49μs 1.1ns 4.25ns 0.0126 0 0 976 B
#6342 CallElasticsearch net472 2.56μs 2.6ns 9.75ns 0.158 0 0 995 B
#6342 CallElasticsearchAsync net6.0 1.24μs 0.539ns 2.02ns 0.013 0 0 952 B
#6342 CallElasticsearchAsync netcoreapp3.1 1.7μs 1.01ns 3.93ns 0.0136 0 0 1.02 KB
#6342 CallElasticsearchAsync net472 2.7μs 2.11ns 8.17ns 0.166 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.28μs 0.401ns 1.45ns 0.0135 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.64μs 0.548ns 2.05ns 0.013 0 0 952 B
master ExecuteAsync net472 1.86μs 1.01ns 3.91ns 0.145 0 0 915 B
#6342 ExecuteAsync net6.0 1.22μs 0.624ns 2.33ns 0.0135 0 0 952 B
#6342 ExecuteAsync netcoreapp3.1 1.6μs 2.02ns 7.82ns 0.0128 0 0 952 B
#6342 ExecuteAsync net472 1.89μs 0.611ns 2.29ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.28μs 2.91ns 10.9ns 0.032 0 0 2.31 KB
master SendAsync netcoreapp3.1 5.31μs 2.28ns 8.85ns 0.0372 0 0 2.85 KB
master SendAsync net472 7.24μs 1.79ns 6.93ns 0.493 0 0 3.12 KB
#6342 SendAsync net6.0 4.37μs 2.31ns 8.63ns 0.0311 0 0 2.31 KB
#6342 SendAsync netcoreapp3.1 5.31μs 1.43ns 5.54ns 0.0373 0 0 2.85 KB
#6342 SendAsync net472 7.31μs 1.57ns 6.06ns 0.493 0 0 3.12 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.45μs 0.734ns 2.75ns 0.0226 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.2μs 1.66ns 6.19ns 0.0218 0 0 1.64 KB
master EnrichedLog net472 2.66μs 0.828ns 3.1ns 0.249 0 0 1.57 KB
#6342 EnrichedLog net6.0 1.54μs 1.11ns 4.14ns 0.0232 0 0 1.64 KB
#6342 EnrichedLog netcoreapp3.1 2.25μs 0.61ns 2.11ns 0.0217 0 0 1.64 KB
#6342 EnrichedLog net472 2.45μs 0.938ns 3.51ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 117μs 54.4ns 211ns 0 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 120μs 70.4ns 263ns 0 0 0 4.28 KB
master EnrichedLog net472 152μs 118ns 455ns 0.688 0.229 0 4.46 KB
#6342 EnrichedLog net6.0 121μs 234ns 907ns 0.0607 0 0 4.28 KB
#6342 EnrichedLog netcoreapp3.1 122μs 153ns 594ns 0.0608 0 0 4.28 KB
#6342 EnrichedLog net472 152μs 217ns 839ns 0.681 0.227 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.91μs 1.19ns 4.61ns 0.0307 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.14μs 1.66ns 5.97ns 0.0289 0 0 2.2 KB
master EnrichedLog net472 4.84μs 1.3ns 4.88ns 0.319 0 0 2.02 KB
#6342 EnrichedLog net6.0 3.03μs 0.786ns 3.04ns 0.0303 0 0 2.2 KB
#6342 EnrichedLog netcoreapp3.1 4.05μs 0.934ns 3.62ns 0.0304 0 0 2.2 KB
#6342 EnrichedLog net472 4.87μs 1.47ns 5.48ns 0.321 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.42μs 0.764ns 2.96ns 0.0164 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.7μs 0.913ns 3.54ns 0.0155 0 0 1.14 KB
master SendReceive net472 2.03μs 0.746ns 2.79ns 0.183 0 0 1.16 KB
#6342 SendReceive net6.0 1.32μs 0.844ns 3.27ns 0.0158 0 0 1.14 KB
#6342 SendReceive netcoreapp3.1 1.78μs 0.738ns 2.86ns 0.0151 0 0 1.14 KB
#6342 SendReceive net472 2.12μs 1.39ns 5.37ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.74μs 1.07ns 4.15ns 0.0219 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 4.01μs 0.981ns 3.67ns 0.022 0 0 1.65 KB
master EnrichedLog net472 4.38μs 4.01ns 15.5ns 0.323 0 0 2.04 KB
#6342 EnrichedLog net6.0 2.89μs 0.87ns 3.37ns 0.0224 0 0 1.6 KB
#6342 EnrichedLog netcoreapp3.1 3.9μs 4.05ns 15.7ns 0.0217 0 0 1.65 KB
#6342 EnrichedLog net472 4.32μs 3.85ns 14.9ns 0.322 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6342

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.141 559.38 490.08

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 403ns 0.652ns 2.53ns 0.00807 0 0 576 B
master StartFinishSpan netcoreapp3.1 551ns 0.695ns 2.69ns 0.00779 0 0 576 B
master StartFinishSpan net472 690ns 1.81ns 7.02ns 0.0915 0 0 578 B
master StartFinishScope net6.0 559ns 0.821ns 3.18ns 0.00971 0 0 696 B
master StartFinishScope netcoreapp3.1 734ns 1.69ns 6.55ns 0.00946 0 0 696 B
master StartFinishScope net472 938ns 2.23ns 8.63ns 0.104 0 0 658 B
#6342 StartFinishSpan net6.0 409ns 0.385ns 1.49ns 0.00817 0 0 576 B
#6342 StartFinishSpan netcoreapp3.1 587ns 0.891ns 3.45ns 0.00764 0 0 576 B
#6342 StartFinishSpan net472 683ns 1.25ns 4.82ns 0.0915 0 0 578 B
#6342 StartFinishScope net6.0 490ns 0.418ns 1.62ns 0.00978 0 0 696 B
#6342 StartFinishScope netcoreapp3.1 708ns 0.738ns 2.86ns 0.00913 0 0 696 B
#6342 StartFinishScope net472 852ns 0.897ns 3.47ns 0.105 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 678ns 0.842ns 3.26ns 0.00974 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 880ns 1.28ns 4.97ns 0.00948 0 0 696 B
master RunOnMethodBegin net472 1.13μs 2.05ns 7.92ns 0.104 0 0 658 B
#6342 RunOnMethodBegin net6.0 662ns 1.07ns 4.15ns 0.00961 0 0 696 B
#6342 RunOnMethodBegin netcoreapp3.1 859ns 1.74ns 6.76ns 0.009 0 0 696 B
#6342 RunOnMethodBegin net472 1.16μs 1.29ns 4.67ns 0.104 0 0 658 B

@e-n-0 e-n-0 force-pushed the flavien/iast/fix-ci-sqlite-core3 branch from dae096a to 451dadc Compare November 25, 2024 16:27
@e-n-0 e-n-0 changed the title [ASM] Fix sqlite binaries for netcoreapp3.0 [ASM] Skip integrations tests using Microsoft.Data.Sqlite on netcoreapp3.0 Nov 25, 2024
@e-n-0 e-n-0 changed the title [ASM] Skip integrations tests using Microsoft.Data.Sqlite on netcoreapp3.0 [ASM] Skip integration tests using Microsoft.Data.Sqlite on netcoreapp3.0 Nov 25, 2024
@e-n-0 e-n-0 force-pushed the flavien/iast/fix-ci-sqlite-core3 branch from 451dadc to eeb569f Compare November 25, 2024 16:30
@e-n-0 e-n-0 marked this pull request as ready for review November 25, 2024 16:30
@e-n-0 e-n-0 requested a review from a team as a code owner November 25, 2024 16:30
@e-n-0 e-n-0 self-assigned this Nov 25, 2024
andrewlock
andrewlock approved these changes Nov 25, 2024
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

bouwkast
bouwkast approved these changes Nov 25, 2024
View reviewed changes
Copy link
Contributor

@bouwkast bouwkast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

robertpi
robertpi approved these changes Nov 26, 2024
View reviewed changes
Copy link
Member

@robertpi robertpi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Easy to review :)

daniel-romano-DD
daniel-romano-DD approved these changes Nov 26, 2024
View reviewed changes
Copy link
Contributor

@daniel-romano-DD daniel-romano-DD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TY

@e-n-0 e-n-0 merged commit b15c82c into master Nov 26, 2024
78 checks passed
@e-n-0 e-n-0 deleted the flavien/iast/fix-ci-sqlite-core3 branch November 26, 2024 10:36
@github-actions github-actions bot added this to the vNext-v3 milestone Nov 26, 2024
veerbia pushed a commit that referenced this pull request Dec 16, 2024
@e-n-0 @veerbia
[ASM] Skip integration tests Microsoft.Data.Sqlite on net 3.0 (#6342)
d6e8ba1 
## Summary of changes

Skip integration tests using `Microsoft.Data.Sqlite` on `netcoreapp3.0`.

## Reason for change

On the CI, the tests using the sqlite databse using
`Microsoft.Data.Sqlite` on `netcoreapp3.0` were failing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daniel-romano-DD daniel-romano-DD daniel-romano-DD approved these changes

@NachoEchevarria NachoEchevarria NachoEchevarria approved these changes

@andrewlock andrewlock andrewlock approved these changes

@bouwkast bouwkast bouwkast approved these changes

@robertpi robertpi robertpi approved these changes

Assignees

@e-n-0 e-n-0

Labels
area:asm area:tests unit tests, integration tests
Projects
None yet
Milestone
3.7.0
Development

Successfully merging this pull request may close these issues.
6 participants
@e-n-0 @andrewlock @robertpi @bouwkast @NachoEchevarria @daniel-romano-DD