-
Notifications
You must be signed in to change notification settings - Fork 143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update the Rel Env docker-image workflow #4448
Conversation
Datadog ReportBranch report: ❄️ New Flaky Tests (2)
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4448) - mean (3,011ms) : 2914, 3107
. : milestone, 3011,
master - mean (3,014ms) : 2932, 3095
. : milestone, 3014,
section CallTarget+Inlining+NGEN
This PR (4448) - mean (3,826ms) : 3752, 3899
. : milestone, 3826,
master - mean (3,820ms) : 3729, 3911
. : milestone, 3820,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4448) - mean (3,117ms) : 2978, 3256
. : milestone, 3117,
master - mean (3,141ms) : 3046, 3236
. : milestone, 3141,
section CallTarget+Inlining+NGEN
This PR (4448) - mean (3,604ms) : 3562, 3645
. : milestone, 3604,
master - mean (3,602ms) : 3527, 3678
. : milestone, 3602,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4448) - mean (3,096ms) : 2980, 3212
. : milestone, 3096,
master - mean (3,118ms) : 3007, 3229
. : milestone, 3118,
section CallTarget+Inlining+NGEN
This PR (4448) - mean (3,581ms) : 3524, 3637
. : milestone, 3581,
master - mean (3,587ms) : 3505, 3669
. : milestone, 3587,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4448) - mean (194ms) : 191, 196
. : milestone, 194,
master - mean (191ms) : 185, 196
. : milestone, 191,
section CallTarget+Inlining+NGEN
This PR (4448) - mean (1,125ms) : 1093, 1158
. : milestone, 1125,
master - mean (1,115ms) : 1087, 1143
. : milestone, 1115,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4448) - mean (374ms) : 366, 381
. : milestone, 374,
master - mean (373ms) : 367, 379
. : milestone, 373,
section CallTarget+Inlining+NGEN
This PR (4448) - mean (1,180ms) : 1146, 1214
. : milestone, 1180,
master - mean (1,179ms) : 1152, 1207
. : milestone, 1179,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4448) - mean (361ms) : 357, 366
. : milestone, 361,
master - mean (361ms) : 355, 367
. : milestone, 361,
section CallTarget+Inlining+NGEN
This PR (4448) - mean (1,135ms) : 1109, 1161
. : milestone, 1135,
master - mean (1,135ms) : 1108, 1162
. : milestone, 1135,
|
Benchmarks Report 🐌Benchmarks for #4448 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AppSecBodyBenchmark - Slower
|
Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net472 | 1.294 | 147.08 | 190.36 |
Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 | 1.178 | 202.94 | 172.25 | |
Benchmarks.Trace.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody‑net472 | 1.162 | 4,729.36 | 4,070.73 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | AllCycleSimpleBody |
net6.0 | 39.4μs | 67.6ns | 262ns | 0.0195 | 0 | 0 | 1.65 KB |
master | AllCycleSimpleBody |
netcoreapp3.1 | 40.9μs | 104ns | 387ns | 0.0204 | 0 | 0 | 1.63 KB |
master | AllCycleSimpleBody |
net472 | 42.3μs | 203ns | 788ns | 0.253 | 0 | 0 | 1.69 KB |
master | AllCycleMoreComplexBody |
net6.0 | 222μs | 77ns | 298ns | 0.112 | 0 | 0 | 9.22 KB |
master | AllCycleMoreComplexBody |
netcoreapp3.1 | 229μs | 262ns | 1.01μs | 0.115 | 0 | 0 | 9.12 KB |
master | AllCycleMoreComplexBody |
net472 | 239μs | 53.9ns | 202ns | 1.43 | 0 | 0 | 9.28 KB |
master | ObjectExtractorSimpleBody |
net6.0 | 120ns | 0.0281ns | 0.105ns | 0.00395 | 0 | 0 | 280 B |
master | ObjectExtractorSimpleBody |
netcoreapp3.1 | 203ns | 0.0702ns | 0.263ns | 0.00369 | 0 | 0 | 272 B |
master | ObjectExtractorSimpleBody |
net472 | 147ns | 0.0525ns | 0.197ns | 0.0446 | 0 | 0 | 281 B |
master | ObjectExtractorMoreComplexBody |
net6.0 | 3.02μs | 1.13ns | 4.23ns | 0.0542 | 0 | 0 | 3.88 KB |
master | ObjectExtractorMoreComplexBody |
netcoreapp3.1 | 4.09μs | 1.33ns | 5.17ns | 0.0512 | 0 | 0 | 3.78 KB |
master | ObjectExtractorMoreComplexBody |
net472 | 4.72μs | 15.9ns | 61.5ns | 0.618 | 0.00705 | 0 | 3.89 KB |
#4448 | AllCycleSimpleBody |
net6.0 | 38.7μs | 16.7ns | 64.7ns | 0.0195 | 0 | 0 | 1.65 KB |
#4448 | AllCycleSimpleBody |
netcoreapp3.1 | 41.5μs | 101ns | 391ns | 0.0209 | 0 | 0 | 1.63 KB |
#4448 | AllCycleSimpleBody |
net472 | 42.4μs | 59.8ns | 231ns | 0.267 | 0 | 0 | 1.69 KB |
#4448 | AllCycleMoreComplexBody |
net6.0 | 220μs | 58.4ns | 219ns | 0.11 | 0 | 0 | 9.22 KB |
#4448 | AllCycleMoreComplexBody |
netcoreapp3.1 | 230μs | 160ns | 599ns | 0.117 | 0 | 0 | 9.12 KB |
#4448 | AllCycleMoreComplexBody |
net472 | 238μs | 65.8ns | 246ns | 1.42 | 0 | 0 | 9.28 KB |
#4448 | ObjectExtractorSimpleBody |
net6.0 | 120ns | 0.0373ns | 0.139ns | 0.00397 | 0 | 0 | 280 B |
#4448 | ObjectExtractorSimpleBody |
netcoreapp3.1 | 172ns | 0.133ns | 0.496ns | 0.00374 | 0 | 0 | 272 B |
#4448 | ObjectExtractorSimpleBody |
net472 | 186ns | 1.06ns | 7.71ns | 0.0446 | 0 | 0 | 281 B |
#4448 | ObjectExtractorMoreComplexBody |
net6.0 | 3.04μs | 3.59ns | 12.9ns | 0.0543 | 0 | 0 | 3.88 KB |
#4448 | ObjectExtractorMoreComplexBody |
netcoreapp3.1 | 4.06μs | 1.36ns | 5.27ns | 0.0509 | 0 | 0 | 3.78 KB |
#4448 | ObjectExtractorMoreComplexBody |
net472 | 4.07μs | 1.42ns | 5.51ns | 0.618 | 0.00612 | 0 | 3.89 KB |
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | SendRequest |
net6.0 | 168μs | 74.4ns | 279ns | 0.253 | 0 | 0 | 18.08 KB |
master | SendRequest |
netcoreapp3.1 | 188μs | 360ns | 1.39μs | 0.189 | 0 | 0 | 20.14 KB |
master | SendRequest |
net472 | 0.000356ns | 0.000172ns | 0.000642ns | 0 | 0 | 0 | 0 b |
#4448 | SendRequest |
net6.0 | 171μs | 117ns | 454ns | 0.171 | 0 | 0 | 18.08 KB |
#4448 | SendRequest |
netcoreapp3.1 | 186μs | 255ns | 988ns | 0.275 | 0 | 0 | 20.14 KB |
#4448 | SendRequest |
net472 | 0.000198ns | 7.9E‑05ns | 0.000285ns | 0 | 0 | 0 | 0 b |
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️
More allocations ⚠️ in #4448
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0
41.46 KB
41.76 KB
301 B
0.73%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1
41.65 KB
41.95 KB
301 B
0.72%
Benchmark | Base Allocated | Diff Allocated | Change | Change % |
---|---|---|---|---|
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 | 41.46 KB | 41.76 KB | 301 B | 0.73% |
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 | 41.65 KB | 41.95 KB | 301 B | 0.72% |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | WriteAndFlushEnrichedTraces |
net6.0 | 519μs | 334ns | 1.29μs | 0.527 | 0 | 0 | 41.46 KB |
master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 635μs | 1.19μs | 4.62μs | 0.317 | 0 | 0 | 41.65 KB |
master | WriteAndFlushEnrichedTraces |
net472 | 773μs | 3.13μs | 12.1μs | 8.15 | 2.33 | 0.388 | 53.22 KB |
#4448 | WriteAndFlushEnrichedTraces |
net6.0 | 517μs | 1.57μs | 5.66μs | 0.504 | 0 | 0 | 41.76 KB |
#4448 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 631μs | 1.07μs | 4.16μs | 0.326 | 0 | 0 | 41.95 KB |
#4448 | WriteAndFlushEnrichedTraces |
net472 | 777μs | 3.21μs | 12μs | 8.44 | 2.68 | 0.383 | 53.23 KB |
Benchmarks.Trace.DbCommandBenchmark - Faster 🎉 Same allocations ✔️
Faster 🎉 in #4448
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0
1.137
1,015.74
893.54
Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0 | 1.137 | 1,015.74 | 893.54 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | ExecuteNonQuery |
net6.0 | 1.02μs | 0.385ns | 1.44ns | 0.0105 | 0 | 0 | 768 B |
master | ExecuteNonQuery |
netcoreapp3.1 | 1.22μs | 1.69ns | 6.33ns | 0.0104 | 0 | 0 | 768 B |
master | ExecuteNonQuery |
net472 | 1.38μs | 0.333ns | 1.2ns | 0.116 | 0 | 0 | 730 B |
#4448 | ExecuteNonQuery |
net6.0 | 894ns | 0.287ns | 1.11ns | 0.0107 | 0 | 0 | 768 B |
#4448 | ExecuteNonQuery |
netcoreapp3.1 | 1.23μs | 0.492ns | 1.9ns | 0.0106 | 0 | 0 | 768 B |
#4448 | ExecuteNonQuery |
net472 | 1.45μs | 0.509ns | 1.9ns | 0.116 | 0 | 0 | 730 B |
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | CallElasticsearch |
net6.0 | 1.15μs | 8.75ns | 81.1ns | 0.0139 | 0 | 0 | 992 B |
master | CallElasticsearch |
netcoreapp3.1 | 1.28μs | 0.278ns | 1.04ns | 0.0135 | 0 | 0 | 992 B |
master | CallElasticsearch |
net472 | 2.1μs | 1.26ns | 4.56ns | 0.158 | 0.00106 | 0 | 1 KB |
master | CallElasticsearchAsync |
net6.0 | 1.2μs | 0.294ns | 1.1ns | 0.0133 | 0 | 0 | 968 B |
master | CallElasticsearchAsync |
netcoreapp3.1 | 1.4μs | 0.558ns | 2.01ns | 0.014 | 0 | 0 | 1.04 KB |
master | CallElasticsearchAsync |
net472 | 2.28μs | 0.652ns | 2.44ns | 0.167 | 0.00114 | 0 | 1.06 KB |
#4448 | CallElasticsearch |
net6.0 | 1.13μs | 0.449ns | 1.74ns | 0.0141 | 0 | 0 | 992 B |
#4448 | CallElasticsearch |
netcoreapp3.1 | 1.25μs | 0.734ns | 2.75ns | 0.0131 | 0 | 0 | 992 B |
#4448 | CallElasticsearch |
net472 | 2.09μs | 0.643ns | 2.49ns | 0.159 | 0.00105 | 0 | 1 KB |
#4448 | CallElasticsearchAsync |
net6.0 | 1.1μs | 0.555ns | 2.08ns | 0.0138 | 0 | 0 | 968 B |
#4448 | CallElasticsearchAsync |
netcoreapp3.1 | 1.43μs | 0.855ns | 3.2ns | 0.0137 | 0 | 0 | 1.04 KB |
#4448 | CallElasticsearchAsync |
net472 | 2.09μs | 0.755ns | 2.92ns | 0.168 | 0.00104 | 0 | 1.06 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | ExecuteAsync |
net6.0 | 1.06μs | 1.2ns | 4.49ns | 0.0126 | 0 | 0 | 912 B |
master | ExecuteAsync |
netcoreapp3.1 | 1.41μs | 0.457ns | 1.65ns | 0.0127 | 0 | 0 | 912 B |
master | ExecuteAsync |
net472 | 1.55μs | 0.565ns | 2.19ns | 0.139 | 0.000775 | 0 | 875 B |
#4448 | ExecuteAsync |
net6.0 | 1.17μs | 1.21ns | 4.19ns | 0.013 | 0 | 0 | 912 B |
#4448 | ExecuteAsync |
netcoreapp3.1 | 1.45μs | 2.61ns | 9.76ns | 0.0122 | 0 | 0 | 912 B |
#4448 | ExecuteAsync |
net472 | 1.6μs | 0.392ns | 1.41ns | 0.138 | 0 | 0 | 875 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | SendAsync |
net6.0 | 3.53μs | 1.01ns | 3.8ns | 0.0266 | 0 | 0 | 1.94 KB |
master | SendAsync |
netcoreapp3.1 | 4.29μs | 11.8ns | 45.8ns | 0.034 | 0 | 0 | 2.48 KB |
master | SendAsync |
net472 | 6.99μs | 10.9ns | 42.2ns | 0.481 | 0 | 0 | 3.05 KB |
#4448 | SendAsync |
net6.0 | 3.73μs | 7.94ns | 29.7ns | 0.0275 | 0 | 0 | 1.94 KB |
#4448 | SendAsync |
netcoreapp3.1 | 4.29μs | 2.06ns | 7.72ns | 0.0323 | 0 | 0 | 2.48 KB |
#4448 | SendAsync |
net472 | 6.82μs | 1.84ns | 6.88ns | 0.484 | 0 | 0 | 3.05 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 1.23μs | 0.521ns | 1.95ns | 0.0229 | 0 | 0 | 1.62 KB |
master | EnrichedLog |
netcoreapp3.1 | 1.83μs | 0.641ns | 2.48ns | 0.0219 | 0 | 0 | 1.62 KB |
master | EnrichedLog |
net472 | 2.28μs | 1.56ns | 6.02ns | 0.245 | 0 | 0 | 1.54 KB |
#4448 | EnrichedLog |
net6.0 | 1.33μs | 0.542ns | 2.03ns | 0.0226 | 0 | 0 | 1.62 KB |
#4448 | EnrichedLog |
netcoreapp3.1 | 1.85μs | 0.907ns | 3.39ns | 0.0221 | 0 | 0 | 1.62 KB |
#4448 | EnrichedLog |
net472 | 2.26μs | 1.6ns | 6.19ns | 0.244 | 0 | 0 | 1.54 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 113μs | 87ns | 326ns | 0.057 | 0 | 0 | 4.21 KB |
master | EnrichedLog |
netcoreapp3.1 | 118μs | 182ns | 704ns | 0.0584 | 0 | 0 | 4.21 KB |
master | EnrichedLog |
net472 | 146μs | 113ns | 423ns | 0.655 | 0.218 | 0 | 4.38 KB |
#4448 | EnrichedLog |
net6.0 | 112μs | 168ns | 650ns | 0.056 | 0 | 0 | 4.21 KB |
#4448 | EnrichedLog |
netcoreapp3.1 | 117μs | 113ns | 409ns | 0 | 0 | 0 | 4.21 KB |
#4448 | EnrichedLog |
net472 | 153μs | 750ns | 3.09μs | 0.666 | 0.222 | 0 | 4.38 KB |
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 2.88μs | 0.917ns | 3.55ns | 0.0302 | 0 | 0 | 2.18 KB |
master | EnrichedLog |
netcoreapp3.1 | 3.82μs | 1.19ns | 4.47ns | 0.0287 | 0 | 0 | 2.18 KB |
master | EnrichedLog |
net472 | 4.4μs | 3.13ns | 12.1ns | 0.315 | 0 | 0 | 1.99 KB |
#4448 | EnrichedLog |
net6.0 | 2.9μs | 12.9ns | 46.6ns | 0.03 | 0 | 0 | 2.18 KB |
#4448 | EnrichedLog |
netcoreapp3.1 | 3.79μs | 1.37ns | 4.76ns | 0.0287 | 0 | 0 | 2.18 KB |
#4448 | EnrichedLog |
net472 | 4.69μs | 3.27ns | 12.2ns | 0.315 | 0 | 0 | 1.99 KB |
Benchmarks.Trace.RedisBenchmark - Faster 🎉 Same allocations ✔️
Faster 🎉 in #4448
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.RedisBenchmark.SendReceive‑net6.0
1.118
1,250.97
1,119.26
Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.RedisBenchmark.SendReceive‑net6.0 | 1.118 | 1,250.97 | 1,119.26 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | SendReceive |
net6.0 | 1.25μs | 0.559ns | 2.09ns | 0.0163 | 0 | 0 | 1.16 KB |
master | SendReceive |
netcoreapp3.1 | 1.51μs | 0.679ns | 2.54ns | 0.0151 | 0 | 0 | 1.16 KB |
master | SendReceive |
net472 | 1.73μs | 0.951ns | 3.3ns | 0.184 | 0 | 0 | 1.16 KB |
#4448 | SendReceive |
net6.0 | 1.12μs | 1.98ns | 8.62ns | 0.0163 | 0 | 0 | 1.16 KB |
#4448 | SendReceive |
netcoreapp3.1 | 1.5μs | 1.03ns | 3.84ns | 0.0157 | 0 | 0 | 1.16 KB |
#4448 | SendReceive |
net472 | 1.83μs | 1.6ns | 5.99ns | 0.185 | 0 | 0 | 1.16 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 2.47μs | 0.665ns | 2.58ns | 0.0209 | 0 | 0 | 1.53 KB |
master | EnrichedLog |
netcoreapp3.1 | 3.47μs | 1.4ns | 5.44ns | 0.021 | 0 | 0 | 1.58 KB |
master | EnrichedLog |
net472 | 4.04μs | 1.73ns | 6.7ns | 0.311 | 0 | 0 | 1.96 KB |
#4448 | EnrichedLog |
net6.0 | 2.69μs | 0.824ns | 3.19ns | 0.0215 | 0 | 0 | 1.53 KB |
#4448 | EnrichedLog |
netcoreapp3.1 | 3.57μs | 1.23ns | 4.75ns | 0.0216 | 0 | 0 | 1.58 KB |
#4448 | EnrichedLog |
net472 | 4.02μs | 1.17ns | 4.54ns | 0.31 | 0 | 0 | 1.96 KB |
Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️
Faster 🎉 in #4448
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0
1.167
457.23
391.64
Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 | 1.167 | 457.23 | 391.64 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | StartFinishSpan |
net6.0 | 457ns | 0.192ns | 0.719ns | 0.00759 | 0 | 0 | 536 B |
master | StartFinishSpan |
netcoreapp3.1 | 568ns | 0.464ns | 1.8ns | 0.00718 | 0 | 0 | 536 B |
master | StartFinishSpan |
net472 | 576ns | 0.142ns | 0.551ns | 0.0852 | 0 | 0 | 538 B |
master | StartFinishScope |
net6.0 | 459ns | 0.119ns | 0.446ns | 0.00915 | 0 | 0 | 656 B |
master | StartFinishScope |
netcoreapp3.1 | 653ns | 0.116ns | 0.417ns | 0.00884 | 0 | 0 | 656 B |
master | StartFinishScope |
net472 | 787ns | 1.62ns | 6.06ns | 0.0979 | 0 | 0 | 618 B |
#4448 | StartFinishSpan |
net6.0 | 392ns | 0.162ns | 0.628ns | 0.00758 | 0 | 0 | 536 B |
#4448 | StartFinishSpan |
netcoreapp3.1 | 574ns | 0.183ns | 0.659ns | 0.0072 | 0 | 0 | 536 B |
#4448 | StartFinishSpan |
net472 | 556ns | 0.174ns | 0.672ns | 0.0852 | 0 | 0 | 538 B |
#4448 | StartFinishScope |
net6.0 | 450ns | 0.18ns | 0.674ns | 0.00907 | 0 | 0 | 656 B |
#4448 | StartFinishScope |
netcoreapp3.1 | 651ns | 0.587ns | 2.2ns | 0.00885 | 0 | 0 | 656 B |
#4448 | StartFinishScope |
net472 | 746ns | 0.309ns | 1.2ns | 0.0981 | 0 | 0 | 618 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Faster 🎉 Same allocations ✔️
Faster 🎉 in #4448
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0
1.151
601.40
522.65
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1
1.130
781.18
691.06
Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 | 1.151 | 601.40 | 522.65 | |
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 | 1.130 | 781.18 | 691.06 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | RunOnMethodBegin |
net6.0 | 602ns | 0.208ns | 0.749ns | 0.00937 | 0 | 0 | 656 B |
master | RunOnMethodBegin |
netcoreapp3.1 | 781ns | 0.372ns | 1.39ns | 0.00862 | 0 | 0 | 656 B |
master | RunOnMethodBegin |
net472 | 925ns | 0.179ns | 0.694ns | 0.0979 | 0 | 0 | 618 B |
#4448 | RunOnMethodBegin |
net6.0 | 523ns | 0.154ns | 0.575ns | 0.00922 | 0 | 0 | 656 B |
#4448 | RunOnMethodBegin |
netcoreapp3.1 | 691ns | 0.279ns | 0.968ns | 0.0087 | 0 | 0 | 656 B |
#4448 | RunOnMethodBegin |
net472 | 971ns | 0.156ns | 0.563ns | 0.0979 | 0 | 0 | 618 B |
f1c6c30
to
087c0af
Compare
Throughput/Crank Report:zap:Throughput results for AspNetCoreSimpleController comparing the following branches/commits: Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red. Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards! gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4448) (11.133M) : 0, 11132835
master (11.149M) : 0, 11149466
benchmarks/2.34.0 (11.172M) : 0, 11172386
benchmarks/2.9.0 (11.226M) : 0, 11225900
section Automatic
This PR (4448) (7.811M) : 0, 7811282
master (7.882M) : 0, 7882084
benchmarks/2.34.0 (7.859M) : 0, 7858718
benchmarks/2.9.0 (8.128M) : 0, 8127670
section Trace stats
master (7.826M) : 0, 7826063
benchmarks/2.34.0 (7.905M) : 0, 7904802
section Manual
This PR (4448) (9.929M) : 0, 9928894
master (9.820M) : 0, 9819660
benchmarks/2.34.0 (9.986M) : 0, 9986137
section Manual + Automatic
This PR (4448) (7.622M) : 0, 7621770
master (7.435M) : 0, 7435018
benchmarks/2.34.0 (7.570M) : 0, 7570487
section Version Conflict
master (6.641M) : 0, 6640824
benchmarks/2.34.0 (6.749M) : 0, 6749094
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4448) (9.665M) : 0, 9665094
master (9.617M) : 0, 9617498
benchmarks/2.34.0 (9.689M) : 0, 9689253
benchmarks/2.9.0 (9.659M) : 0, 9658728
section Automatic
This PR (4448) (6.699M) : 0, 6699445
master (6.703M) : 0, 6702641
benchmarks/2.34.0 (6.670M) : 0, 6670396
section Trace stats
master (6.666M) : 0, 6665747
benchmarks/2.34.0 (6.743M) : 0, 6743225
section Manual
This PR (4448) (8.616M) : 0, 8616243
master (8.521M) : 0, 8520521
benchmarks/2.34.0 (8.614M) : 0, 8614349
section Manual + Automatic
This PR (4448) (6.667M) : 0, 6667397
master (6.551M) : 0, 6551069
benchmarks/2.34.0 (6.532M) : 0, 6532340
section Version Conflict
master (5.875M) : 0, 5875078
benchmarks/2.34.0 (5.854M) : 0, 5853936
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4448) (10.190M) : 0, 10190247
master (10.155M) : 0, 10155240
benchmarks/2.34.0 (10.187M) : 0, 10187020
benchmarks/2.9.0 (10.240M) : 0, 10239951
section Automatic
This PR (4448) (7.639M) : 0, 7638574
master (7.588M) : 0, 7588027
benchmarks/2.34.0 (7.402M) : 0, 7401872
benchmarks/2.9.0 (7.435M) : 0, 7435116
section Trace stats
master (7.344M) : 0, 7343611
benchmarks/2.34.0 (7.426M) : 0, 7426314
section Manual
This PR (4448) (9.224M) : 0, 9224216
master (9.276M) : 0, 9275658
benchmarks/2.34.0 (9.173M) : 0, 9173438
section Manual + Automatic
This PR (4448) (7.019M) : 0, 7018744
master (7.213M) : 0, 7213113
benchmarks/2.34.0 (7.218M) : 0, 7218183
section Version Conflict
master (6.567M) : 0, 6566571
benchmarks/2.34.0 (6.597M) : 0, 6597265
gantt
title Throughput Linux x64 (ASM) (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4448) (7.476M) : 0, 7476175
master (7.422M) : 0, 7422015
benchmarks/2.34.0 (7.612M) : 0, 7611651
benchmarks/2.9.0 (7.840M) : 0, 7839825
section No attack
This PR (4448) (2.135M) : 0, 2134542
master (2.167M) : 0, 2167375
benchmarks/2.34.0 (2.156M) : 0, 2156076
benchmarks/2.9.0 (3.181M) : 0, 3180750
section Attack
This PR (4448) (1.828M) : 0, 1828104
master (1.868M) : 0, 1868231
benchmarks/2.9.0 (2.474M) : 0, 2474287
section Blocking
This PR (4448) (3.716M) : 0, 3715570
master (3.849M) : 0, 3849329
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM with one comment on the trigger on the main pipeline.
I assume we should add @robertomonteromiguel as a reviewer as he's the one who introduced this so that he's aware of the changes if future refactos are needed
- Extract a GitHub action that removes a bit of the complexity from the workflows - Rename `is_snapshot` to `is_release_version` to make it more explicit about what's happening - Remove the creating of the rel env release images out of the create_draft_release workflow - Create a new workflow to create the images after the release is merged
8511b87
to
d917c5e
Compare
d917c5e
to
712e40e
Compare
Summary of changes
create_draft_release
workflowis_snapshot
tois_release_version
to make it more explicit about what's happeningReason for change
The existing trigger (silently) failed in the
create_draft_release
workflow. The implications of this weren't clear at the time, partly as the naming wasn't clear that it was related to the rel-env (and therefor low impact for the release).By moving to a separate workflow it moves it out of the critical path.
Implementation details
Test coverage
TODO