[CI Visibility] - Enable AutomaticDecompression on CI Visibility http clients #5043

tonyredondo · 2024-01-11T10:42:53Z

Summary of changes

This PR enables the AutomaticDecompression flag on CI Visibility http clients.

Reason for change

When using Intelligent Test Runner the backend send us a lot of data like commits shas or the test list to be skipped. Recently the backend endpoints added support to return compressed payloads, and we want to start taking advantage of this.

Implementation details

Just enable the flag on a custom HttpClientHandler.

This PR only enables the feature on NETCOREAPP frameworks for now, most of the transaction are made by the dotnet tool (only supported on .NET Core and greater). Maybe in the future we will enable it for .NET Framework, we will see.

datadog-ddstaging · 2024-01-11T11:03:16Z

Datadog Report

Branch report: tony/ci-visibility-enable-automatic-decompression
Commit report: d81a71e
Test service: dd-trace-dotnet

✅ 0 Failed, 300605 Passed, 1014 Skipped, 57m 2.05s Wall Time
❄️ 33 New Flaky

New Flaky Tests (33)

SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.AgentMalfunctionTests - Last Failure
Expand for error
```
 Expected exit code: 0, actual exit code: -1073741819.
```
HttpClient_SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.HttpMessageHandlerTests - Last Failure
Expand for error
```
 Expected exit code: 0, actual exit code: -1073741819.
```
HttpClient_SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.HttpMessageHandlerTests - Last Failure
Expand for error
```
 Expected exit code: 0, actual exit code: -1073741819.
```

SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OwinWebApi2TestsCallTarget - Last Failure

Expand for error

 An error occurred while sending the request.
 ---- System.Net.WebException : Unable to connect to the remote server
 -------- System.Net.Sockets.SocketException : No connection could be made because the target machine actively refused it 127.0.0.1:52906

SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OwinWebApi2TestsCallTarget - Last Failure

Expand for error

 An error occurred while sending the request.
 ---- System.Net.WebException : Unable to connect to the remote server
 -------- System.Net.Sockets.SocketException : No connection could be made because the target machine actively refused it 127.0.0.1:52906

andrewlock · 2024-01-11T11:14:46Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5043) - mean (74ms)  : 62, 86
     .   : milestone, 74,
    master - mean (71ms)  : 60, 81
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (5043) - mean (1,037ms)  : 1011, 1063
     .   : milestone, 1037,
    master - mean (1,024ms)  : 1002, 1047
     .   : milestone, 1024,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5043) - mean (105ms)  : 100, 110
     .   : milestone, 105,
    master - mean (105ms)  : 99, 111
     .   : milestone, 105,

    section CallTarget+Inlining+NGEN
    This PR (5043) - mean (739ms)  : 723, 755
     .   : milestone, 739,
    master - mean (726ms)  : 704, 748
     .   : milestone, 726,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5043) - mean (91ms)  : 87, 95
     .   : milestone, 91,
    master - mean (88ms)  : 83, 92
     .   : milestone, 88,

    section CallTarget+Inlining+NGEN
    This PR (5043) - mean (698ms)  : 679, 718
     .   : milestone, 698,
    master - mean (689ms)  : 663, 715
     .   : milestone, 689,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5043) - mean (188ms)  : 186, 190
     .   : milestone, 188,
    master - mean (188ms)  : 185, 190
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (5043) - mean (1,141ms)  : 1122, 1160
     .   : milestone, 1141,
    master - mean (1,142ms)  : 1122, 1163
     .   : milestone, 1142,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5043) - mean (272ms)  : 268, 276
     .   : milestone, 272,
    master - mean (271ms)  : 267, 275
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (5043) - mean (1,102ms)  : 1076, 1128
     .   : milestone, 1102,
    master - mean (1,102ms)  : 1080, 1124
     .   : milestone, 1102,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5043) - mean (262ms)  : 259, 265
     .   : milestone, 262,
    master - mean (260ms)  : 258, 263
     .   : milestone, 260,

    section CallTarget+Inlining+NGEN
    This PR (5043) - mean (1,067ms)  : 1045, 1088
     .   : milestone, 1067,
    master - mean (1,063ms)  : 1045, 1080
     .   : milestone, 1063,

tracer/src/Datadog.Trace/Ci/CIVisibility.cs

andrewlock · 2024-01-11T21:32:16Z

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5043) (11.247M)   : 0, 11246609
    master (10.866M)   : 0, 10866082
    benchmarks/2.9.0 (11.571M)   : 0, 11571320

    section Automatic
    This PR (5043) (7.615M)   : 0, 7614560
    master (7.540M)   : 0, 7540346
    benchmarks/2.9.0 (8.323M)   : 0, 8322791

    section Trace stats
    This PR (5043) (8.016M)   : 0, 8015804
    master (7.871M)   : 0, 7871047

    section Manual
    This PR (5043) (9.713M)   : 0, 9712655
    master (9.592M)   : 0, 9592317

    section Manual + Automatic
    This PR (5043) (7.209M)   : 0, 7209470
    master (7.111M)   : 0, 7111355

    section Version Conflict
    This PR (5043) (6.525M)   : 0, 6525454
    master (6.500M)   : 0, 6500033

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5043) (9.582M)   : 0, 9582477
    master (9.626M)   : 0, 9626140
    benchmarks/2.9.0 (9.660M)   : 0, 9659654

    section Automatic
    This PR (5043) (6.556M)   : 0, 6555967
    master (6.598M)   : 0, 6597508

    section Trace stats
    This PR (5043) (7.007M)   : 0, 7007329
    master (6.903M)   : 0, 6903022

    section Manual
    This PR (5043) (8.251M)   : 0, 8250783
    master (8.229M)   : 0, 8229482

    section Manual + Automatic
    This PR (5043) (6.112M)   : 0, 6111747
    master (5.942M)   : 0, 5941926

    section Version Conflict
    This PR (5043) (5.744M)   : 0, 5744240
    master (5.798M)   : 0, 5797646

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5043) (10.160M)   : 0, 10159593
    master (10.043M)   : 0, 10043350
    benchmarks/2.9.0 (9.963M)   : 0, 9963205

    section Automatic
    This PR (5043) (7.307M)   : 0, 7307380
    master (7.177M)   : 0, 7176524
    benchmarks/2.9.0 (7.483M)   : 0, 7483414

    section Trace stats
    This PR (5043) (7.729M)   : 0, 7728510
    master (7.524M)   : 0, 7523623

    section Manual
    This PR (5043) (9.232M)   : 0, 9232192
    master (8.868M)   : 0, 8868075

    section Manual + Automatic
    This PR (5043) (6.986M)   : 0, 6986076
    master (6.874M)   : 0, 6873629

    section Version Conflict
    This PR (5043) (6.440M)   : 0, 6440253
    master (6.235M)   : 0, 6235474

gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.393M)   : 0, 7392821
    benchmarks/2.9.0 (7.887M)   : 0, 7887276

    section No attack
    master (1.755M)   : 0, 1754573
    benchmarks/2.9.0 (3.249M)   : 0, 3249034

    section Attack
    master (1.418M)   : 0, 1417862
    benchmarks/2.9.0 (2.565M)   : 0, 2565044

    section Blocking
    master (3.126M)   : 0, 3126330

    section IAST default
    master (6.441M)   : 0, 6440706

    section IAST full
    master (5.740M)   : 0, 5739720

    section Base vuln
    master (0.955M)   : 0, 955487

    section IAST vuln
    master (0.871M)   : 0, 870710

Enable AutomaticDecompression on CI Visibility http clients

d81a71e

tonyredondo requested review from a team as code owners January 11, 2024 10:42

github-actions bot added the area:ci-visibility label Jan 11, 2024

tonyredondo self-assigned this Jan 11, 2024

anmarchenko approved these changes Jan 11, 2024

View reviewed changes

andrewlock approved these changes Jan 11, 2024

View reviewed changes

tracer/src/Datadog.Trace/Ci/CIVisibility.cs Show resolved Hide resolved

tonyredondo merged commit a978693 into master Jan 11, 2024
52 of 55 checks passed

tonyredondo deleted the tony/ci-visibility-enable-automatic-decompression branch January 11, 2024 22:31

github-actions bot added this to the vNext milestone Jan 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CI Visibility] - Enable AutomaticDecompression on CI Visibility http clients #5043

[CI Visibility] - Enable AutomaticDecompression on CI Visibility http clients #5043

tonyredondo commented Jan 11, 2024

datadog-ddstaging bot commented Jan 11, 2024 •

edited

Loading

andrewlock commented Jan 11, 2024

andrewlock commented Jan 11, 2024

[CI Visibility] - Enable AutomaticDecompression on CI Visibility http clients #5043

[CI Visibility] - Enable AutomaticDecompression on CI Visibility http clients #5043

Conversation

tonyredondo commented Jan 11, 2024

Summary of changes

Reason for change

Implementation details

datadog-ddstaging bot commented Jan 11, 2024 • edited Loading

Datadog Report

New Flaky Tests (33)

andrewlock commented Jan 11, 2024

Execution-Time Benchmarks Report ⏱️

andrewlock commented Jan 11, 2024

Throughput/Crank Report:zap:

datadog-ddstaging bot commented Jan 11, 2024 •

edited

Loading