[ASM] Remove debug mode for unit tests #4728

anna-git · 2023-10-13T19:31:38Z

Summary of changes

Accidentally added debug mode for unit tests so remove it

Reason for change

can potentially cause waf timeouts during tests

Implementation details

Test coverage

Other details

datadog-ddstaging · 2023-10-13T19:56:00Z

Datadog Report

Branch report: anna/asm/remove-debug-mode-unittests
Commit report: e52538e

✅ dd-trace-dotnet: 0 Failed, 0 New Flaky, 307536 Passed, 1155 Skipped, 51m 48.43s Wall Time

andrewlock · 2023-10-13T19:58:09Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4728) - mean (69ms)  : 65, 74
     .   : milestone, 69,
    master - mean (75ms)  : 57, 92
     .   : milestone, 75,

    section CallTarget+Inlining+NGEN
    This PR (4728) - mean (1,030ms)  : 1010, 1050
     .   : milestone, 1030,
    master - mean (1,030ms)  : 1013, 1046
     .   : milestone, 1030,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4728) - mean (106ms)  : 102, 110
     .   : milestone, 106,
    master - mean (105ms)  : 103, 108
     .   : milestone, 105,

    section CallTarget+Inlining+NGEN
    This PR (4728) - mean (735ms)  : 717, 753
     .   : milestone, 735,
    master - mean (742ms)  : 727, 757
     .   : milestone, 742,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4728) - mean (89ms)  : 87, 91
     .   : milestone, 89,
    master - mean (90ms)  : 86, 94
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (4728) - mean (699ms)  : 682, 716
     .   : milestone, 699,
    master - mean (706ms)  : 687, 726
     .   : milestone, 706,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4728) - mean (188ms)  : 185, 190
     .   : milestone, 188,
    master - mean (187ms)  : 184, 190
     .   : milestone, 187,

    section CallTarget+Inlining+NGEN
    This PR (4728) - mean (1,134ms)  : 1113, 1156
     .   : milestone, 1134,
    master - mean (1,132ms)  : 1106, 1159
     .   : milestone, 1132,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4728) - mean (271ms)  : 267, 275
     .   : milestone, 271,
    master - mean (271ms)  : 268, 274
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (4728) - mean (1,105ms)  : 1088, 1122
     .   : milestone, 1105,
    master - mean (1,101ms)  : 1077, 1126
     .   : milestone, 1101,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4728) - mean (260ms)  : 257, 264
     .   : milestone, 260,
    master - mean (260ms)  : 257, 263
     .   : milestone, 260,

    section CallTarget+Inlining+NGEN
    This PR (4728) - mean (1,068ms)  : 1047, 1088
     .   : milestone, 1068,
    master - mean (1,059ms)  : 1033, 1086
     .   : milestone, 1059,

andrewlock · 2023-10-13T21:05:57Z

Benchmarks Report 🐌

Benchmarks for #4728 compared to master:

1 benchmarks are faster, with geometric mean 1.150
2 benchmarks are slower, with geometric mean 1.153
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.24μs	45ns	255ns	0.0206	0.00822	0	7.29 KB
master	`StartStopWithChild`	netcoreapp3.1	10.2μs	55.6ns	315ns	0.0254	0.0102	0	7.39 KB
master	`StartStopWithChild`	net472	15.5μs	60.3ns	226ns	1.29	0.326	0.101	7.66 KB
#4728	`StartStopWithChild`	net6.0	8.18μs	43.1ns	255ns	0.0254	0.00847	0	7.28 KB
#4728	`StartStopWithChild`	netcoreapp3.1	10.2μs	52.7ns	269ns	0.0251	0.0151	0	7.39 KB
#4728	`StartStopWithChild`	net472	15.5μs	53.4ns	207ns	1.28	0.31	0.101	7.66 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	472μs	239ns	927ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	622μs	149ns	558ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	799μs	282ns	1.09μs	0.398	3.3 KB
#4728	`WriteAndFlushEnrichedTraces`	net6.0	467μs	432ns	1.67μs	0	2.7 KB
#4728	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	632μs	254ns	983ns	0	2.7 KB
#4728	`WriteAndFlushEnrichedTraces`	net472	803μs	284ns	1.06μs	0.401	3.3 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	47.7μs	42.4ns	164ns	0.0215	0	2.03 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	47.1μs	111ns	430ns	0.0232	0	2.01 KB
master	`AllCycleSimpleBody`	net472	47.2μs	43ns	155ns	0.325	0	2.08 KB
master	`AllCycleMoreComplexBody`	net6.0	230μs	184ns	713ns	0.114	0	8.63 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	236μs	159ns	594ns	0	0	8.52 KB
master	`AllCycleMoreComplexBody`	net472	240μs	106ns	398ns	1.31	0	8.7 KB
master	`ObjectExtractorSimpleBody`	net6.0	128ns	0.109ns	0.408ns	0.00392	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	177ns	0.137ns	0.513ns	0.00369	0	272 B
master	`ObjectExtractorSimpleBody`	net472	147ns	0.156ns	0.605ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3.19μs	2.36ns	8.83ns	0.0548	0	3.88 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.15μs	2.34ns	8.11ns	0.0498	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	net472	4.31μs	5.34ns	20.7ns	0.618	0.00648	3.89 KB
#4728	`AllCycleSimpleBody`	net6.0	43.5μs	44.8ns	168ns	0.0217	0	2.03 KB
#4728	`AllCycleSimpleBody`	netcoreapp3.1	45.9μs	37.1ns	139ns	0.0228	0	2.01 KB
#4728	`AllCycleSimpleBody`	net472	47.8μs	38.8ns	145ns	0.328	0	2.08 KB
#4728	`AllCycleMoreComplexBody`	net6.0	230μs	156ns	605ns	0	0	8.63 KB
#4728	`AllCycleMoreComplexBody`	netcoreapp3.1	236μs	181ns	699ns	0.117	0	8.52 KB
#4728	`AllCycleMoreComplexBody`	net472	242μs	1.02μs	6.28μs	1.31	0	8.7 KB
#4728	`ObjectExtractorSimpleBody`	net6.0	122ns	0.098ns	0.38ns	0.00393	0	280 B
#4728	`ObjectExtractorSimpleBody`	netcoreapp3.1	179ns	0.138ns	0.534ns	0.00378	0	272 B
#4728	`ObjectExtractorSimpleBody`	net472	145ns	0.156ns	0.585ns	0.0446	0	281 B
#4728	`ObjectExtractorMoreComplexBody`	net6.0	3.1μs	1.37ns	4.93ns	0.0542	0	3.88 KB
#4728	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.34μs	3.48ns	13.5ns	0.0499	0	3.78 KB
#4728	`ObjectExtractorMoreComplexBody`	net472	4.26μs	6.71ns	25.1ns	0.618	0.00637	3.89 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`RunWaf(args=NestedMap (10))`	net6.0	12.7μs	3.25ns	12.6ns	0.128	0	9.42 KB
master	`RunWaf(args=NestedMap (10))`	netcoreapp3.1	19.6μs	5.17ns	18.6ns	0.127	0	9.42 KB
master	`RunWaf(args=NestedMap (10))`	net472	28.3μs	25.6ns	99.2ns	1.49	0	9.48 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	77.4μs	77ns	298ns	0.192	0	15.77 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	87.5μs	38.5ns	144ns	0.175	0	15.72 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	99.1μs	177ns	686ns	2.51	0	16.04 KB
master	`RunWaf(args=NestedMap (100))`	net6.0	24.6μs	8.12ns	30.4ns	0.269	0	19.66 KB
master	`RunWaf(args=NestedMap (100))`	netcoreapp3.1	38.2μs	16.9ns	63.3ns	0.267	0	20.42 KB
master	`RunWaf(args=NestedMap (100))`	net472	55.3μs	33.4ns	129ns	3.26	0.0278	20.63 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	net6.0	92.8μs	39.4ns	153ns	0.323	0	26.01 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	netcoreapp3.1	109μs	292ns	1.13μs	0.325	0	26.72 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	net472	133μs	267ns	1.03μs	4.26	0.0655	27.19 KB
master	`RunWaf(args=NestedMap (20))`	net6.0	24.4μs	8.08ns	31.3ns	0.266	0	19.4 KB
master	`RunWaf(args=NestedMap (20))`	netcoreapp3.1	37.8μs	10.1ns	37.7ns	0.264	0	19.84 KB
master	`RunWaf(args=NestedMap (20))`	net472	54.8μs	29ns	108ns	3.16	0.055	20.04 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	92μs	39.2ns	152ns	0.364	0	25.74 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	109μs	165ns	639ns	0.327	0	26.14 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	131μs	234ns	907ns	4.22	0.065	26.6 KB
#4728	`RunWaf(args=NestedMap (10))`	net6.0	12.6μs	4.26ns	15.9ns	0.132	0	9.42 KB
#4728	`RunWaf(args=NestedMap (10))`	netcoreapp3.1	19.5μs	5.16ns	20ns	0.127	0	9.42 KB
#4728	`RunWaf(args=NestedMap (10))`	net472	28.1μs	10.9ns	42.1ns	1.5	0	9.48 KB
#4728	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	77.7μs	48.5ns	188ns	0.196	0	15.77 KB
#4728	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	88.2μs	125ns	467ns	0.176	0	15.72 KB
#4728	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	99μs	277ns	1.07μs	2.52	0	16.04 KB
#4728	`RunWaf(args=NestedMap (100))`	net6.0	26.5μs	13.4ns	50.2ns	0.265	0	19.66 KB
#4728	`RunWaf(args=NestedMap (100))`	netcoreapp3.1	38μs	15.4ns	57.6ns	0.266	0	20.42 KB
#4728	`RunWaf(args=NestedMap (100))`	net472	53.8μs	23.6ns	81.8ns	3.26	0.027	20.63 KB
#4728	`RunWafWithAttack(args=Neste(...)tack) [23])`	net6.0	92.5μs	128ns	494ns	0.371	0	26.01 KB
#4728	`RunWafWithAttack(args=Neste(...)tack) [23])`	netcoreapp3.1	109μs	245ns	950ns	0.329	0	26.72 KB
#4728	`RunWafWithAttack(args=Neste(...)tack) [23])`	net472	134μs	48.9ns	183ns	4.28	0.0669	27.19 KB
#4728	`RunWaf(args=NestedMap (20))`	net6.0	26.1μs	6.38ns	23.9ns	0.275	0	19.4 KB
#4728	`RunWaf(args=NestedMap (20))`	netcoreapp3.1	37.8μs	15.9ns	61.6ns	0.265	0	19.84 KB
#4728	`RunWaf(args=NestedMap (20))`	net472	53.7μs	15.8ns	59ns	3.17	0.0538	20.04 KB
#4728	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	92.2μs	25ns	86.5ns	0.368	0	25.74 KB
#4728	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	110μs	160ns	621ns	0.328	0	26.14 KB
#4728	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	134μs	69.7ns	261ns	4.16	0.0661	26.6 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	169μs	95.6ns	370ns	0.252	18.09 KB
master	`SendRequest`	netcoreapp3.1	190μs	205ns	767ns	0.19	20.25 KB
master	`SendRequest`	net472	0.000947ns	0.000317ns	0.00123ns	0	0 b
#4728	`SendRequest`	net6.0	169μs	237ns	918ns	0.254	18.09 KB
#4728	`SendRequest`	netcoreapp3.1	188μs	317ns	1.23μs	0.185	20.25 KB
#4728	`SendRequest`	net472	0.00247ns	0.000442ns	0.00171ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #4728

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.54 KB	41.85 KB	310 B	0.75%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	552μs	303ns	1.09μs	0.553	0	0	41.54 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	668μs	975ns	3.78μs	0.334	0	0	41.85 KB
master	`WriteAndFlushEnrichedTraces`	net472	828μs	2.78μs	10.8μs	8.08	2.55	0.425	53.25 KB
#4728	`WriteAndFlushEnrichedTraces`	net6.0	576μs	835ns	3.23μs	0.573	0	0	41.85 KB
#4728	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	664μs	1.16μs	4.48μs	0.336	0	0	41.8 KB
#4728	`WriteAndFlushEnrichedTraces`	net472	846μs	3.84μs	14.9μs	8.39	2.52	0.419	53.24 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.02μs	1.09ns	4.2ns	0.0107	768 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.24μs	0.498ns	1.86ns	0.0106	768 B
master	`ExecuteNonQuery`	net472	1.65μs	0.836ns	3.01ns	0.116	730 B
#4728	`ExecuteNonQuery`	net6.0	1.04μs	0.572ns	2.22ns	0.011	768 B
#4728	`ExecuteNonQuery`	netcoreapp3.1	1.36μs	0.689ns	2.67ns	0.01	768 B
#4728	`ExecuteNonQuery`	net472	1.79μs	1.9ns	7.1ns	0.115	730 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.14μs	0.454ns	1.7ns	0.0137	992 B
master	`CallElasticsearch`	netcoreapp3.1	1.43μs	0.935ns	3.37ns	0.0135	992 B
master	`CallElasticsearch`	net472	2.27μs	1.2ns	4.5ns	0.159	1 KB
master	`CallElasticsearchAsync`	net6.0	1.25μs	0.457ns	1.65ns	0.0136	968 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.59μs	0.625ns	2.42ns	0.0142	1.04 KB
master	`CallElasticsearchAsync`	net472	2.59μs	1.11ns	4.14ns	0.168	1.06 KB
#4728	`CallElasticsearch`	net6.0	1.26μs	0.59ns	2.21ns	0.0138	992 B
#4728	`CallElasticsearch`	netcoreapp3.1	1.55μs	5.5ns	21.3ns	0.0129	992 B
#4728	`CallElasticsearch`	net472	2.27μs	1.48ns	5.55ns	0.159	1 KB
#4728	`CallElasticsearchAsync`	net6.0	1.27μs	0.987ns	3.69ns	0.0134	968 B
#4728	`CallElasticsearchAsync`	netcoreapp3.1	1.63μs	1.06ns	3.68ns	0.0137	1.04 KB
#4728	`CallElasticsearchAsync`	net472	2.47μs	2.37ns	8.86ns	0.168	1.06 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.17μs	0.621ns	2.41ns	0.0129	912 B
master	`ExecuteAsync`	netcoreapp3.1	1.56μs	1.23ns	4.77ns	0.0124	912 B
master	`ExecuteAsync`	net472	1.74μs	1.01ns	3.92ns	0.138	875 B
#4728	`ExecuteAsync`	net6.0	1.28μs	0.413ns	1.6ns	0.0127	912 B
#4728	`ExecuteAsync`	netcoreapp3.1	1.55μs	0.703ns	2.63ns	0.0123	912 B
#4728	`ExecuteAsync`	net472	1.77μs	1.06ns	3.96ns	0.139	875 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	3.86μs	2.07ns	8.02ns	0.0268	1.94 KB
master	`SendAsync`	netcoreapp3.1	4.42μs	1.35ns	5.06ns	0.0331	2.48 KB
master	`SendAsync`	net472	7.2μs	3.98ns	14.3ns	0.481	3.05 KB
#4728	`SendAsync`	net6.0	3.87μs	1.42ns	5.5ns	0.027	1.94 KB
#4728	`SendAsync`	netcoreapp3.1	4.46μs	1.74ns	6.5ns	0.0334	2.48 KB
#4728	`SendAsync`	net472	7.19μs	2.05ns	7.96ns	0.484	3.05 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.46μs	7.6ns	36.5ns	0.0226	1.62 KB
master	`EnrichedLog`	netcoreapp3.1	2.03μs	1.61ns	6.23ns	0.0221	1.62 KB
master	`EnrichedLog`	net472	2.28μs	2.4ns	8.99ns	0.245	1.54 KB
#4728	`EnrichedLog`	net6.0	1.28μs	0.624ns	2.33ns	0.0232	1.62 KB
#4728	`EnrichedLog`	netcoreapp3.1	1.86μs	1.05ns	4.07ns	0.0219	1.62 KB
#4728	`EnrichedLog`	net472	2.33μs	1.03ns	3.72ns	0.245	1.54 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	111μs	105ns	407ns	0.0556	0	4.21 KB
master	`EnrichedLog`	netcoreapp3.1	118μs	107ns	401ns	0	0	4.21 KB
master	`EnrichedLog`	net472	147μs	201ns	780ns	0.654	0.218	4.38 KB
#4728	`EnrichedLog`	net6.0	111μs	78.5ns	304ns	0.0556	0	4.21 KB
#4728	`EnrichedLog`	netcoreapp3.1	118μs	265ns	1.03μs	0	0	4.21 KB
#4728	`EnrichedLog`	net472	147μs	99.7ns	386ns	0.666	0.222	4.38 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.85μs	1.3ns	4.87ns	0.0299	2.18 KB
master	`EnrichedLog`	netcoreapp3.1	3.91μs	1.1ns	3.96ns	0.0296	2.18 KB
master	`EnrichedLog`	net472	4.58μs	1.04ns	4.02ns	0.315	1.99 KB
#4728	`EnrichedLog`	net6.0	2.81μs	0.909ns	3.4ns	0.031	2.18 KB
#4728	`EnrichedLog`	netcoreapp3.1	4.05μs	10.6ns	41ns	0.0278	2.18 KB
#4728	`EnrichedLog`	net472	4.51μs	1.31ns	5.08ns	0.314	1.99 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.29μs	0.772ns	2.89ns	0.0161	1.16 KB
master	`SendReceive`	netcoreapp3.1	1.64μs	0.644ns	2.49ns	0.0156	1.16 KB
master	`SendReceive`	net472	2.05μs	2.03ns	7.86ns	0.184	1.16 KB
#4728	`SendReceive`	net6.0	1.31μs	1.27ns	4.77ns	0.0164	1.16 KB
#4728	`SendReceive`	netcoreapp3.1	1.58μs	0.364ns	1.36ns	0.0158	1.16 KB
#4728	`SendReceive`	net472	2.05μs	3.34ns	12.9ns	0.185	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.63μs	1.3ns	4.87ns	0.021	1.53 KB
master	`EnrichedLog`	netcoreapp3.1	3.53μs	0.897ns	3.23ns	0.0217	1.58 KB
master	`EnrichedLog`	net472	3.99μs	1.47ns	5.3ns	0.311	1.96 KB
#4728	`EnrichedLog`	net6.0	2.7μs	1.29ns	4.99ns	0.0213	1.53 KB
#4728	`EnrichedLog`	netcoreapp3.1	3.75μs	4.57ns	17.7ns	0.0206	1.58 KB
#4728	`EnrichedLog`	net472	4.03μs	1.95ns	7.29ns	0.311	1.96 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4728

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.189	393.38	467.89
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0	1.117	472.89	528.41

Faster 🎉 in #4728

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1	1.150	746.47	649.33

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	393ns	0.168ns	0.63ns	0.00744	536 B
master	`StartFinishSpan`	netcoreapp3.1	641ns	2.11ns	8.19ns	0.00743	536 B
master	`StartFinishSpan`	net472	686ns	0.196ns	0.761ns	0.0854	538 B
master	`StartFinishScope`	net6.0	473ns	0.277ns	1.07ns	0.00914	656 B
master	`StartFinishScope`	netcoreapp3.1	746ns	0.451ns	1.69ns	0.00895	656 B
master	`StartFinishScope`	net472	832ns	0.401ns	1.5ns	0.0981	618 B
#4728	`StartFinishSpan`	net6.0	468ns	0.171ns	0.64ns	0.00758	536 B
#4728	`StartFinishSpan`	netcoreapp3.1	580ns	0.428ns	1.66ns	0.00703	536 B
#4728	`StartFinishSpan`	net472	713ns	0.471ns	1.83ns	0.0853	538 B
#4728	`StartFinishScope`	net6.0	528ns	0.513ns	1.85ns	0.00907	656 B
#4728	`StartFinishScope`	netcoreapp3.1	649ns	0.194ns	0.7ns	0.00872	656 B
#4728	`StartFinishScope`	net472	800ns	0.863ns	3.34ns	0.0978	618 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	613ns	0.22ns	0.85ns	0.00914	656 B
master	`RunOnMethodBegin`	netcoreapp3.1	767ns	1.57ns	6.06ns	0.00882	656 B
master	`RunOnMethodBegin`	net472	916ns	0.774ns	3ns	0.0979	618 B
#4728	`RunOnMethodBegin`	net6.0	628ns	0.192ns	0.719ns	0.00918	656 B
#4728	`RunOnMethodBegin`	netcoreapp3.1	822ns	0.323ns	1.25ns	0.00888	656 B
#4728	`RunOnMethodBegin`	net472	924ns	0.578ns	2.16ns	0.098	618 B

Remove debug mode

e52538e

anna-git marked this pull request as ready for review October 13, 2023 19:31

anna-git requested a review from a team as a code owner October 13, 2023 19:31

github-actions bot added area:tests unit tests, integration tests area:asm labels Oct 13, 2023

anna-git requested a review from a team as a code owner October 13, 2023 21:39

anna-git force-pushed the anna/asm/remove-debug-mode-unittests branch from 1cd770d to e52538e Compare October 13, 2023 21:41

andrewlock approved these changes Oct 16, 2023

View reviewed changes

NachoEchevarria approved these changes Oct 16, 2023

View reviewed changes

anna-git merged commit 09efe0a into master Oct 17, 2023

anna-git deleted the anna/asm/remove-debug-mode-unittests branch October 17, 2023 09:02

github-actions bot added this to the vNext milestone Oct 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ASM] Remove debug mode for unit tests #4728

[ASM] Remove debug mode for unit tests #4728

anna-git commented Oct 13, 2023

datadog-ddstaging bot commented Oct 13, 2023 •

edited

Loading

andrewlock commented Oct 13, 2023 •

edited

Loading

andrewlock commented Oct 13, 2023 •

edited

Loading

Raw results

Raw results

Raw results

Raw results

Raw results

More allocations ⚠️ in #4728

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #4728

Faster 🎉 in #4728

Raw results

Raw results

[ASM] Remove debug mode for unit tests #4728

[ASM] Remove debug mode for unit tests #4728

Conversation

anna-git commented Oct 13, 2023

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Oct 13, 2023 • edited Loading

Datadog Report

andrewlock commented Oct 13, 2023 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented Oct 13, 2023 • edited Loading

Benchmarks Report 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

More allocations ⚠️ in #4728

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #4728

Faster 🎉 in #4728

Raw results

Raw results

datadog-ddstaging bot commented Oct 13, 2023 •

edited

Loading

andrewlock commented Oct 13, 2023 •

edited

Loading

andrewlock commented Oct 13, 2023 •

edited

Loading