AutoGenerator bug fixes #5029

andrewlock · 2024-01-08T14:10:39Z

Summary of changes

Fix: ensure we include the full type name in the [InstrumentMethod] attribute
Fix: ensure generics are defined correctly (previously incorrectly had a space in the name)
nit: Remove _ from typename
Use [] C#12 (as suggested in IDE by default now)

Reason for change

The first 2 are bugs, the other 2 are just quality of life fixes

Implementation details

Pretty self-explanatory. Tested locally with .\tracer\build.ps1 RunInstrumentationGenerator and looks good to me

Test coverage

Manual testing

[InstrumentMethod(
    AssemblyName = "Datadog.Trace.Custom",
-   TypeName = "ITest",
+   TypeName = "Datadog.Trace.Ci.ITest",
    MethodName = "SetTraits",
    ReturnTypeName = ClrNames.Void,
-   ParameterTypeNames = new []{ "System.Collections.Generic.Dictionary`2[System.String, System.Collections.Generic.List`1[System.String]]"},
+   ParameterTypeNames = ["System.Collections.Generic.Dictionary`2[System.String,System.Collections.Generic.List`1[System.String]]"],
    MinimumVersion = "3.0.0",
    MaximumVersion = "3.*.*",
    IntegrationName = IntegrationName)]
[Browsable(false)]
[EditorBrowsable(EditorBrowsableState.Never)]
-public class ITest_SetTraitsIntegration
+public class ITestSetTraitsIntegration

Other details

Thanks @tonyredondo 😄

…e them!

tonyredondo

Thanks for the fixes

andrewlock · 2024-01-08T15:04:14Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5029) - mean (71ms)  : 63, 79
     .   : milestone, 71,
    master - mean (71ms)  : 63, 79
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (5029) - mean (1,028ms)  : 1001, 1054
     .   : milestone, 1028,
    master - mean (1,018ms)  : 994, 1042
     .   : milestone, 1018,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5029) - mean (106ms)  : 102, 109
     .   : milestone, 106,
    master - mean (105ms)  : 101, 109
     .   : milestone, 105,

    section CallTarget+Inlining+NGEN
    This PR (5029) - mean (731ms)  : 705, 757
     .   : milestone, 731,
    master - mean (723ms)  : 702, 745
     .   : milestone, 723,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5029) - mean (90ms)  : 88, 92
     .   : milestone, 90,
    master - mean (89ms)  : 85, 93
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (5029) - mean (694ms)  : 671, 717
     .   : milestone, 694,
    master - mean (690ms)  : 665, 715
     .   : milestone, 690,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5029) - mean (189ms)  : 187, 191
     .   : milestone, 189,
    master - mean (189ms)  : 186, 191
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (5029) - mean (1,132ms)  : 1107, 1158
     .   : milestone, 1132,
    master - mean (1,132ms)  : 1107, 1156
     .   : milestone, 1132,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5029) - mean (273ms)  : 269, 277
     .   : milestone, 273,
    master - mean (273ms)  : 269, 276
     .   : milestone, 273,

    section CallTarget+Inlining+NGEN
    This PR (5029) - mean (1,087ms)  : 1066, 1107
     .   : milestone, 1087,
    master - mean (1,080ms)  : 1050, 1109
     .   : milestone, 1080,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5029) - mean (262ms)  : 258, 266
     .   : milestone, 262,
    master - mean (262ms)  : 259, 265
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (5029) - mean (1,055ms)  : 1034, 1077
     .   : milestone, 1055,
    master - mean (1,048ms)  : 1019, 1077
     .   : milestone, 1048,

datadog-ddstaging · 2024-01-08T15:12:38Z

Datadog Report

Branch report: andrew/fix-auto-generator-bugs
Commit report: 19219fd
Test service: dd-trace-dotnet

✅ 0 Failed, 305692 Passed, 1000 Skipped, 40m 33.69s Wall Time
❄️ 1 New Flaky

New Flaky Tests (1)

SubmitsTraces - Datadog.Trace.Security.IntegrationTests.Iast.DeduplicationTests - Last Failure
Expand for error
```
 Expected exit code: 0, actual exit code: -1073741819.
```

daniel-romano-DD

Nice :)

andrewlock · 2024-01-09T03:07:52Z

Benchmarks Report 🐌

Benchmarks for #5029 compared to master:

1 benchmarks are faster, with geometric mean 1.359
3 benchmarks are slower, with geometric mean 1.151
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.7μs	46.8ns	269ns	0.0164	0.0082	0	7.48 KB
master	`StartStopWithChild`	netcoreapp3.1	10.7μs	57.1ns	352ns	0.026	0.0104	0	7.57 KB
master	`StartStopWithChild`	net472	17.1μs	45.9ns	178ns	1.32	0.332	0.111	7.95 KB
#5029	`StartStopWithChild`	net6.0	8.82μs	49.2ns	307ns	0.0257	0.0128	0	7.48 KB
#5029	`StartStopWithChild`	netcoreapp3.1	11μs	61.3ns	393ns	0.0279	0.0112	0	7.58 KB
#5029	`StartStopWithChild`	net472	17.1μs	51.6ns	200ns	1.34	0.344	0.117	7.95 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	451μs	135ns	505ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	645μs	249ns	965ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	786μs	288ns	1.08μs	0.391	3.3 KB
#5029	`WriteAndFlushEnrichedTraces`	net6.0	471μs	224ns	808ns	0	2.7 KB
#5029	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	623μs	377ns	1.46μs	0	2.7 KB
#5029	`WriteAndFlushEnrichedTraces`	net472	787μs	384ns	1.49μs	0.393	3.3 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5029

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net472	1.359	224.68	165.30

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	38.4μs	9.28ns	36ns	0.0194	0	1.77 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	41.7μs	89.4ns	346ns	0.0207	0	1.74 KB
master	`AllCycleSimpleBody`	net472	44.5μs	16.2ns	60.7ns	0.267	0	1.81 KB
master	`AllCycleMoreComplexBody`	net6.0	200μs	56.9ns	213ns	0.101	0	9.25 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	213μs	209ns	809ns	0.105	0	9.14 KB
master	`AllCycleMoreComplexBody`	net472	226μs	69ns	258ns	1.47	0	9.32 KB
master	`ObjectExtractorSimpleBody`	net6.0	140ns	0.0514ns	0.192ns	0.00395	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	218ns	0.317ns	1.23ns	0.00371	0	272 B
master	`ObjectExtractorSimpleBody`	net472	225ns	0.196ns	0.735ns	0.0445	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	2.94μs	1.34ns	5.21ns	0.0529	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.88μs	2.28ns	8.83ns	0.0503	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	3.73μs	4.44ns	17.2ns	0.603	0.00562	3.8 KB
#5029	`AllCycleSimpleBody`	net6.0	38.6μs	14.4ns	55.9ns	0.0191	0	1.77 KB
#5029	`AllCycleSimpleBody`	netcoreapp3.1	41.9μs	29.9ns	116ns	0.0209	0	1.74 KB
#5029	`AllCycleSimpleBody`	net472	43.8μs	14.2ns	53.1ns	0.286	0	1.81 KB
#5029	`AllCycleMoreComplexBody`	net6.0	198μs	71.3ns	247ns	0.0995	0	9.25 KB
#5029	`AllCycleMoreComplexBody`	netcoreapp3.1	211μs	138ns	533ns	0.106	0	9.14 KB
#5029	`AllCycleMoreComplexBody`	net472	225μs	84.1ns	326ns	1.46	0	9.32 KB
#5029	`ObjectExtractorSimpleBody`	net6.0	139ns	0.0468ns	0.175ns	0.00394	0	280 B
#5029	`ObjectExtractorSimpleBody`	netcoreapp3.1	200ns	0.0814ns	0.315ns	0.00376	0	272 B
#5029	`ObjectExtractorSimpleBody`	net472	165ns	0.213ns	0.798ns	0.0446	0	281 B
#5029	`ObjectExtractorMoreComplexBody`	net6.0	3μs	1.73ns	6.46ns	0.0525	0	3.78 KB
#5029	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.89μs	1.62ns	6.29ns	0.0504	0	3.69 KB
#5029	`ObjectExtractorMoreComplexBody`	net472	3.79μs	3.3ns	12.8ns	0.602	0.0057	3.8 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5029

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWaf(args: NestedMap (10))‑net6.0	1.152	50,163.68	57,802.74

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`RunWaf(args=NestedMap (10))`	net6.0	50.2μs	40.7ns	204ns	0.226	0	16.06 KB
master	`RunWaf(args=NestedMap (10))`	netcoreapp3.1	65.7μs	32.7ns	122ns	0.206	0	16.06 KB
master	`RunWaf(args=NestedMap (10))`	net472	97μs	343ns	1.33μs	2.53	0.0955	16.14 KB
master	`RunWafTwice(args=NestedMap (10))`	net6.0	58.4μs	287ns	1.25μs	0.236	0	16.6 KB
master	`RunWafTwice(args=NestedMap (10))`	netcoreapp3.1	73.4μs	410ns	2.78μs	0.209	0	16.58 KB
master	`RunWafTwice(args=NestedMap (10))`	net472	103μs	51.5ns	199ns	2.63	0.103	16.69 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	113μs	48.3ns	181ns	0.282	0	22.41 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	128μs	602ns	2.41μs	0.251	0	22.36 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	159μs	112ns	419ns	3.58	0.159	22.7 KB
master	`RunWaf(args=NestedMap (100))`	net6.0	105μs	611ns	5.33μs	0.443	0	32.76 KB
master	`RunWaf(args=NestedMap (100))`	netcoreapp3.1	132μs	622ns	2.57μs	0.413	0	33.33 KB
master	`RunWaf(args=NestedMap (100))`	net472	188μs	207ns	801ns	5.32	0.373	33.67 KB
master	`RunWafTwice(args=NestedMap (100))`	net6.0	105μs	307ns	1.19μs	0.461	0	33.3 KB
master	`RunWafTwice(args=NestedMap (100))`	netcoreapp3.1	135μs	667ns	2.75μs	0.418	0	33.86 KB
master	`RunWafTwice(args=NestedMap (100))`	net472	199μs	714ns	2.76μs	5.36	0.39	34.22 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	net6.0	164μs	657ns	2.54μs	0.557	0	39.1 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	netcoreapp3.1	200μs	715ns	2.77μs	0.484	0	39.63 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	net472	265μs	200ns	774ns	6.28	0.512	40.23 KB
master	`RunWaf(args=NestedMap (20))`	net6.0	99.3μs	352ns	1.27μs	0.416	0	32.18 KB
master	`RunWaf(args=NestedMap (20))`	netcoreapp3.1	132μs	486ns	1.88μs	0.4	0	32.3 KB
master	`RunWaf(args=NestedMap (20))`	net472	190μs	994ns	4.97μs	5.12	0.372	32.63 KB
master	`RunWafTwice(args=NestedMap (20))`	net6.0	105μs	226ns	876ns	0.45	0	32.72 KB
master	`RunWafTwice(args=NestedMap (20))`	netcoreapp3.1	138μs	758ns	3.12μs	0.426	0	32.82 KB
master	`RunWafTwice(args=NestedMap (20))`	net472	197μs	733ns	2.84μs	5.25	0.389	33.19 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	156μs	105ns	444ns	0.493	0	38.53 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	199μs	882ns	3.41μs	0.476	0	38.6 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	263μs	278ns	1.04μs	6.11	0.509	39.2 KB
#5029	`RunWaf(args=NestedMap (10))`	net6.0	57.8μs	19.9ns	74.3ns	0.217	0	16.06 KB
#5029	`RunWaf(args=NestedMap (10))`	netcoreapp3.1	70.3μs	401ns	3.03μs	0.196	0	16.06 KB
#5029	`RunWaf(args=NestedMap (10))`	net472	95.2μs	38.6ns	139ns	2.52	0.095	16.14 KB
#5029	`RunWafTwice(args=NestedMap (10))`	net6.0	58.9μs	327ns	2.02μs	0.212	0	16.6 KB
#5029	`RunWafTwice(args=NestedMap (10))`	netcoreapp3.1	71.8μs	339ns	1.4μs	0.206	0	16.58 KB
#5029	`RunWafTwice(args=NestedMap (10))`	net472	104μs	64.4ns	249ns	2.6	0.104	16.69 KB
#5029	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	115μs	65.5ns	245ns	0.287	0	22.41 KB
#5029	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	130μs	677ns	3.31μs	0.255	0	22.36 KB
#5029	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	160μs	97.3ns	377ns	3.55	0.161	22.7 KB
#5029	`RunWaf(args=NestedMap (100))`	net6.0	106μs	579ns	3.52μs	0.434	0	32.76 KB
#5029	`RunWaf(args=NestedMap (100))`	netcoreapp3.1	130μs	540ns	2.09μs	0.396	0	33.33 KB
#5029	`RunWaf(args=NestedMap (100))`	net472	194μs	485ns	1.88μs	5.31	0.373	33.67 KB
#5029	`RunWafTwice(args=NestedMap (100))`	net6.0	100μs	24.5ns	94.9ns	0.452	0	33.3 KB
#5029	`RunWafTwice(args=NestedMap (100))`	netcoreapp3.1	140μs	764ns	4.39μs	0.422	0	33.86 KB
#5029	`RunWafTwice(args=NestedMap (100))`	net472	195μs	98.4ns	381ns	5.36	0.39	34.22 KB
#5029	`RunWafWithAttack(args=Neste(...)tack) [23])`	net6.0	158μs	65.5ns	286ns	0.503	0	39.1 KB
#5029	`RunWafWithAttack(args=Neste(...)tack) [23])`	netcoreapp3.1	200μs	1.03μs	4.74μs	0.506	0	39.63 KB
#5029	`RunWafWithAttack(args=Neste(...)tack) [23])`	net472	257μs	240ns	931ns	6.39	0.511	40.23 KB
#5029	`RunWaf(args=NestedMap (20))`	net6.0	98.9μs	371ns	1.7μs	0.462	0	32.18 KB
#5029	`RunWaf(args=NestedMap (20))`	netcoreapp3.1	134μs	735ns	4.29μs	0.405	0	32.3 KB
#5029	`RunWaf(args=NestedMap (20))`	net472	189μs	890ns	3.56μs	5.19	0.37	32.63 KB
#5029	`RunWafTwice(args=NestedMap (20))`	net6.0	109μs	52.4ns	182ns	0.431	0	32.72 KB
#5029	`RunWafTwice(args=NestedMap (20))`	netcoreapp3.1	138μs	521ns	2.02μs	0.404	0	32.82 KB
#5029	`RunWafTwice(args=NestedMap (20))`	net472	193μs	255ns	987ns	5.22	0.386	33.19 KB
#5029	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	169μs	448ns	1.73μs	0.55	0	38.53 KB
#5029	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	198μs	290ns	1.08μs	0.497	0	38.6 KB
#5029	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	255μs	140ns	544ns	6.12	0.51	39.2 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	170μs	125ns	484ns	0.168	18.25 KB
master	`SendRequest`	netcoreapp3.1	192μs	270ns	1.05μs	0.192	20.41 KB
master	`SendRequest`	net472	0.000279ns	0.000172ns	0.000644ns	0	0 b
#5029	`SendRequest`	net6.0	169μs	116ns	433ns	0.251	18.25 KB
#5029	`SendRequest`	netcoreapp3.1	195μs	331ns	1.28μs	0.192	20.41 KB
#5029	`SendRequest`	net472	4.97E‑05ns	4.97E‑05ns	0.000186ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	544μs	367ns	1.27μs	0.539	0	0	41.52 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	652μs	526ns	2.04μs	0.327	0	0	41.75 KB
master	`WriteAndFlushEnrichedTraces`	net472	839μs	3.71μs	13.9μs	8.33	2.5	0.417	53.23 KB
#5029	`WriteAndFlushEnrichedTraces`	net6.0	541μs	701ns	2.72μs	0.546	0	0	41.57 KB
#5029	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	645μs	1.03μs	4μs	0.319	0	0	41.56 KB
#5029	`WriteAndFlushEnrichedTraces`	net472	871μs	4.19μs	16.2μs	8.36	2.64	0.44	53.21 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.03μs	0.57ns	2.13ns	0.0108	768 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.43μs	0.602ns	2.33ns	0.0102	768 B
master	`ExecuteNonQuery`	net472	1.81μs	0.698ns	2.7ns	0.115	730 B
#5029	`ExecuteNonQuery`	net6.0	1.14μs	0.492ns	1.91ns	0.0108	768 B
#5029	`ExecuteNonQuery`	netcoreapp3.1	1.48μs	0.358ns	1.34ns	0.0104	768 B
#5029	`ExecuteNonQuery`	net472	1.81μs	3.35ns	13ns	0.115	730 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.26μs	0.603ns	2.34ns	0.0133	936 B
master	`CallElasticsearch`	netcoreapp3.1	1.58μs	1.07ns	3.71ns	0.0127	936 B
master	`CallElasticsearch`	net472	2.53μs	1.41ns	5.46ns	0.151	955 B
master	`CallElasticsearchAsync`	net6.0	1.2μs	0.365ns	1.41ns	0.0125	912 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.56μs	1.21ns	4.51ns	0.0127	984 B
master	`CallElasticsearchAsync`	net472	2.72μs	0.767ns	2.87ns	0.159	1.01 KB
#5029	`CallElasticsearch`	net6.0	1.19μs	0.731ns	2.83ns	0.0132	936 B
#5029	`CallElasticsearch`	netcoreapp3.1	1.45μs	1.44ns	5.59ns	0.0124	936 B
#5029	`CallElasticsearch`	net472	2.53μs	0.712ns	2.66ns	0.151	955 B
#5029	`CallElasticsearchAsync`	net6.0	1.2μs	0.822ns	3.07ns	0.0124	912 B
#5029	`CallElasticsearchAsync`	netcoreapp3.1	1.6μs	1.35ns	5.23ns	0.0136	984 B
#5029	`CallElasticsearchAsync`	net472	2.72μs	1.08ns	3.88ns	0.16	1.01 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.28μs	0.961ns	3.6ns	0.0129	912 B
master	`ExecuteAsync`	netcoreapp3.1	1.57μs	3.39ns	13.6ns	0.0119	912 B
master	`ExecuteAsync`	net472	1.78μs	1.34ns	5.18ns	0.139	875 B
#5029	`ExecuteAsync`	net6.0	1.36μs	1.29ns	4.98ns	0.0126	912 B
#5029	`ExecuteAsync`	netcoreapp3.1	1.63μs	1.52ns	5.88ns	0.0123	912 B
#5029	`ExecuteAsync`	net472	1.82μs	1.06ns	4.1ns	0.139	875 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.08μs	1.1ns	4.11ns	0.0285	2.1 KB
master	`SendAsync`	netcoreapp3.1	4.97μs	2ns	7.76ns	0.0347	2.63 KB
master	`SendAsync`	net472	7.8μs	5.23ns	20.2ns	0.522	3.31 KB
#5029	`SendAsync`	net6.0	4.11μs	5.38ns	20.8ns	0.0288	2.1 KB
#5029	`SendAsync`	netcoreapp3.1	5.03μs	2.2ns	8.53ns	0.0353	2.63 KB
#5029	`SendAsync`	net472	7.74μs	3.03ns	11.3ns	0.525	3.31 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5029

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472	57.78 KB	60.06 KB	2.28 KB	3.95%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	net6.0	50.6μs	230ns	892ns	43.44 KB
master	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	netcoreapp3.1	52.6μs	256ns	1.06μs	42.64 KB
master	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	net472	38.4μs	197ns	923ns	57.78 KB
master	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	net6.0	81.2μs	2.34μs	23.4μs	43.29 KB
master	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	netcoreapp3.1	108μs	3.49μs	34.7μs	42.64 KB
master	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	net472	62.8μs	207ns	745ns	57.34 KB
#5029	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	net6.0	58.9μs	688ns	6.88μs	43.44 KB
#5029	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	netcoreapp3.1	53μs	243ns	911ns	42.64 KB
#5029	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	net472	38.1μs	116ns	436ns	60.06 KB
#5029	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	net6.0	83.7μs	1.61μs	16.1μs	43.29 KB
#5029	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	netcoreapp3.1	108μs	3.27μs	32.4μs	42.64 KB
#5029	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	net472	62.8μs	175ns	655ns	57.34 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.45μs	0.445ns	1.66ns	0.0219	1.57 KB
master	`EnrichedLog`	netcoreapp3.1	2.21μs	1.32ns	5.11ns	0.0209	1.57 KB
master	`EnrichedLog`	net472	2.61μs	1.07ns	3.86ns	0.238	1.5 KB
#5029	`EnrichedLog`	net6.0	1.38μs	0.415ns	1.44ns	0.0219	1.57 KB
#5029	`EnrichedLog`	netcoreapp3.1	2.15μs	0.774ns	3ns	0.0206	1.57 KB
#5029	`EnrichedLog`	net472	2.54μs	2.55ns	9.54ns	0.238	1.5 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	115μs	115ns	428ns	0	0	4.21 KB
master	`EnrichedLog`	netcoreapp3.1	120μs	205ns	768ns	0.0595	0	4.21 KB
master	`EnrichedLog`	net472	147μs	55.1ns	206ns	0.661	0.22	4.39 KB
#5029	`EnrichedLog`	net6.0	115μs	92.3ns	357ns	0.0572	0	4.21 KB
#5029	`EnrichedLog`	netcoreapp3.1	119μs	259ns	1μs	0.0588	0	4.21 KB
#5029	`EnrichedLog`	net472	149μs	239ns	925ns	0.673	0.224	4.39 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.06μs	1.09ns	4.22ns	0.0306	2.13 KB
master	`EnrichedLog`	netcoreapp3.1	4.12μs	1.4ns	5.42ns	0.0288	2.13 KB
master	`EnrichedLog`	net472	4.72μs	1.97ns	7.64ns	0.307	1.95 KB
#5029	`EnrichedLog`	net6.0	2.94μs	1.26ns	4.87ns	0.0295	2.13 KB
#5029	`EnrichedLog`	netcoreapp3.1	4.13μs	2.89ns	11.2ns	0.0289	2.13 KB
#5029	`EnrichedLog`	net472	4.84μs	1.42ns	4.92ns	0.307	1.95 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.31μs	0.787ns	3.05ns	0.0157	1.1 KB
master	`SendReceive`	netcoreapp3.1	1.8μs	0.998ns	3.6ns	0.0144	1.1 KB
master	`SendReceive`	net472	2.06μs	1.17ns	4.38ns	0.177	1.12 KB
#5029	`SendReceive`	net6.0	1.34μs	0.794ns	3.08ns	0.0154	1.1 KB
#5029	`SendReceive`	netcoreapp3.1	1.74μs	0.387ns	1.45ns	0.0147	1.1 KB
#5029	`SendReceive`	net472	2.08μs	2.38ns	9.23ns	0.177	1.12 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.76μs	1.06ns	4.12ns	0.0207	1.53 KB
master	`EnrichedLog`	netcoreapp3.1	4.13μs	1.82ns	7.05ns	0.0206	1.58 KB
master	`EnrichedLog`	net472	4.37μs	2.44ns	9.46ns	0.312	1.97 KB
#5029	`EnrichedLog`	net6.0	2.77μs	0.934ns	3.37ns	0.0205	1.53 KB
#5029	`EnrichedLog`	netcoreapp3.1	3.83μs	1.64ns	6.34ns	0.021	1.58 KB
#5029	`EnrichedLog`	net472	4.28μs	2.98ns	11.5ns	0.312	1.97 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5029

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472	1.156	681.83	788.03
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1	1.146	664.41	761.51

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	561ns	0.578ns	2.08ns	0.00759	536 B
master	`StartFinishSpan`	netcoreapp3.1	662ns	2.2ns	8.22ns	0.00713	536 B
master	`StartFinishSpan`	net472	682ns	0.557ns	2.16ns	0.0854	538 B
master	`StartFinishScope`	net6.0	579ns	1.53ns	5.92ns	0.00908	656 B
master	`StartFinishScope`	netcoreapp3.1	815ns	0.394ns	1.53ns	0.00863	656 B
master	`StartFinishScope`	net472	928ns	0.266ns	1.03ns	0.098	618 B
#5029	`StartFinishSpan`	net6.0	514ns	0.0943ns	0.365ns	0.00742	536 B
#5029	`StartFinishSpan`	netcoreapp3.1	762ns	0.125ns	0.468ns	0.00733	536 B
#5029	`StartFinishSpan`	net472	788ns	0.827ns	3.2ns	0.0853	538 B
#5029	`StartFinishScope`	net6.0	556ns	0.42ns	1.63ns	0.00912	656 B
#5029	`StartFinishScope`	netcoreapp3.1	807ns	0.518ns	2.01ns	0.0089	656 B
#5029	`StartFinishScope`	net472	1.03μs	1.97ns	7.62ns	0.0977	618 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	654ns	0.308ns	1.15ns	0.00922	656 B
master	`RunOnMethodBegin`	netcoreapp3.1	911ns	3.71ns	14.4ns	0.00884	656 B
master	`RunOnMethodBegin`	net472	1.06μs	0.613ns	2.37ns	0.0979	618 B
#5029	`RunOnMethodBegin`	net6.0	640ns	0.345ns	1.34ns	0.00909	656 B
#5029	`RunOnMethodBegin`	netcoreapp3.1	944ns	0.524ns	2.03ns	0.009	656 B
#5029	`RunOnMethodBegin`	net472	993ns	0.314ns	1.17ns	0.0977	618 B

andrewlock added 4 commits January 8, 2024 13:52

Fix: ensure we include the full type name in the TypeName property

9851172

Fix incorrectly adding space in generic parameters

2dacfdc

nit: don't add _ in generated class name as Rider tells us to remov…

f6969e2

…e them!

C#12 FTW

19219fd

andrewlock added the area:tools label Jan 8, 2024

andrewlock requested a review from a team as a code owner January 8, 2024 14:10

tonyredondo approved these changes Jan 8, 2024

View reviewed changes

daniel-romano-DD approved these changes Jan 8, 2024

View reviewed changes

kevingosse approved these changes Jan 11, 2024

View reviewed changes

andrewlock merged commit 7e55d08 into master Jan 11, 2024
53 of 55 checks passed

andrewlock deleted the andrew/fix-auto-generator-bugs branch January 11, 2024 13:16

github-actions bot added this to the vNext milestone Jan 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AutoGenerator bug fixes #5029

AutoGenerator bug fixes #5029

andrewlock commented Jan 8, 2024

tonyredondo left a comment

andrewlock commented Jan 8, 2024

datadog-ddstaging bot commented Jan 8, 2024 •

edited

Loading

daniel-romano-DD left a comment

andrewlock commented Jan 9, 2024

Raw results

Raw results

Faster 🎉 in #5029

Raw results

Slower ⚠️ in #5029

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

More allocations ⚠️ in #5029

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #5029

Raw results

Raw results

AutoGenerator bug fixes #5029

AutoGenerator bug fixes #5029

Conversation

andrewlock commented Jan 8, 2024

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

tonyredondo left a comment

Choose a reason for hiding this comment

andrewlock commented Jan 8, 2024

Execution-Time Benchmarks Report ⏱️

datadog-ddstaging bot commented Jan 8, 2024 • edited Loading

Datadog Report

New Flaky Tests (1)

daniel-romano-DD left a comment

Choose a reason for hiding this comment

andrewlock commented Jan 9, 2024

Benchmarks Report 🐌

Benchmark details

Raw results

Raw results

Faster 🎉 in #5029

Raw results

Slower ⚠️ in #5029

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

More allocations ⚠️ in #5029

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #5029

Raw results

Raw results

datadog-ddstaging bot commented Jan 8, 2024 •

edited

Loading