fix UpdateVendoredCode on mac #6490

vandonr · 2024-12-31T11:04:57Z

Summary of changes

force the name of the honeypot template because it's different on mac and on windows

Reason for change

the command was broken with the following error:

System.ArgumentNullException: Value cannot be null. (Parameter 'stream')
   at System.IO.StreamReader..ctor(Stream stream, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize, Boolean leaveOpen)
   at Honeypot.DependabotFileManager.GetHoneyPotProjTemplate() in /Users/raphael.vandon/go/src/github.com/DataDog/dd-trace-dotnet/tracer/build/_build/Honeypot/DependabotFileManager.cs:line 120
   at Honeypot.DependabotFileManager.UpdateVendors(AbsolutePath honeypotProject) in /Users/raphael.vandon/go/src/github.com/DataDog/dd-trace-dotnet/tracer/build/_build/Honeypot/DependabotFileManager.cs:line 31

I discovered almost by chance that the resource had the "wrong" name in the dll when built on my mac. Forcing the name doesn't change anything on windows, and unifies the behavior on mac.

Implementation details

Test coverage

Other details

andrewlock

Thanks! I still don't understand why though 😅

andrewlock · 2024-12-31T11:37:43Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6490) - mean (69ms)  : 66, 71
     .   : milestone, 69,
    master - mean (69ms)  : 64, 73
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6490) - mean (979ms)  : 955, 1003
     .   : milestone, 979,
    master - mean (976ms)  : 946, 1006
     .   : milestone, 976,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6490) - mean (107ms)  : 105, 109
     .   : milestone, 107,
    master - mean (107ms)  : 105, 109
     .   : milestone, 107,

    section CallTarget+Inlining+NGEN
    This PR (6490) - mean (675ms)  : 661, 688
     .   : milestone, 675,
    master - mean (677ms)  : 659, 694
     .   : milestone, 677,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6490) - mean (91ms)  : 89, 93
     .   : milestone, 91,
    master - mean (91ms)  : 89, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6490) - mean (632ms)  : 614, 650
     .   : milestone, 632,
    master - mean (629ms)  : 614, 643
     .   : milestone, 629,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6490) - mean (195ms)  : 190, 200
     .   : milestone, 195,
    master - mean (193ms)  : 188, 199
     .   : milestone, 193,

    section CallTarget+Inlining+NGEN
    This PR (6490) - mean (1,101ms)  : 1071, 1130
     .   : milestone, 1101,
    master - mean (1,099ms)  : 1070, 1129
     .   : milestone, 1099,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6490) - mean (279ms)  : 275, 284
     .   : milestone, 279,
    master - mean (279ms)  : 273, 284
     .   : milestone, 279,

    section CallTarget+Inlining+NGEN
    This PR (6490) - mean (870ms)  : 835, 905
     .   : milestone, 870,
    master - mean (874ms)  : 844, 903
     .   : milestone, 874,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6490) - mean (267ms)  : 262, 271
     .   : milestone, 267,
    master - mean (267ms)  : 261, 273
     .   : milestone, 267,

    section CallTarget+Inlining+NGEN
    This PR (6490) - mean (852ms)  : 821, 884
     .   : milestone, 852,
    master - mean (846ms)  : 819, 873
     .   : milestone, 846,

datadog-ddstaging · 2024-12-31T11:37:45Z

Datadog Report

Branch report: vandonr/macfix
Commit report: aacce3d
Test service: dd-trace-dotnet

✅ 0 Failed, 229670 Passed, 2100 Skipped, 18h 22m 9.42s Total Time

andrewlock · 2024-12-31T12:27:42Z

Benchmarks Report for tracer 🐌

Benchmarks for #6490 compared to master:

2 benchmarks are faster, with geometric mean 1.198
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.2μs	46.6ns	339ns	0.0118	0.00393	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.2μs	45.4ns	176ns	0.0313	0.0157	0.00522	5.8 KB
master	`StartStopWithChild`	net472	16.5μs	56.5ns	219ns	1.04	0.302	0.098	6.2 KB
#6490	`StartStopWithChild`	net6.0	8.07μs	45.5ns	295ns	0.0152	0.0076	0	5.6 KB
#6490	`StartStopWithChild`	netcoreapp3.1	9.99μs	55.3ns	341ns	0.0192	0.00959	0	5.8 KB
#6490	`StartStopWithChild`	net472	16.4μs	60.2ns	233ns	1.04	0.305	0.099	6.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	475μs	362ns	1.4μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	636μs	418ns	1.56μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	855μs	533ns	2.06μs	0.425	3.3 KB
#6490	`WriteAndFlushEnrichedTraces`	net6.0	479μs	154ns	576ns	0	2.7 KB
#6490	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	647μs	300ns	1.12μs	0	2.7 KB
#6490	`WriteAndFlushEnrichedTraces`	net472	853μs	641ns	2.4μs	0.431	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	162μs	1.47μs	14.4μs	0.164	14.47 KB
master	`SendRequest`	netcoreapp3.1	172μs	1μs	9.77μs	0.159	17.27 KB
master	`SendRequest`	net472	0.00297ns	0.00093ns	0.00348ns	0	0 b
#6490	`SendRequest`	net6.0	149μs	840ns	6.06μs	0.143	14.47 KB
#6490	`SendRequest`	netcoreapp3.1	172μs	1.07μs	10.5μs	0.166	17.27 KB
#6490	`SendRequest`	net472	9.03E‑05ns	9.03E‑05ns	0.000326ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	551μs	2.79μs	12.8μs	0.553	0	0	41.66 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	682μs	3.86μs	28.4μs	0.327	0	0	41.83 KB
master	`WriteAndFlushEnrichedTraces`	net472	841μs	2.93μs	11.3μs	8.17	2.45	0.408	53.31 KB
#6490	`WriteAndFlushEnrichedTraces`	net6.0	579μs	3.26μs	26.9μs	0.573	0	0	41.45 KB
#6490	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	675μs	3.18μs	11.9μs	0.336	0	0	41.9 KB
#6490	`WriteAndFlushEnrichedTraces`	net472	847μs	3.38μs	13.1μs	8.33	2.5	0.417	53.28 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.33μs	1.34ns	5.2ns	0.014	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.84μs	1.16ns	4.49ns	0.0139	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.12μs	2.57ns	9.95ns	0.156	0.00106	987 B
#6490	`ExecuteNonQuery`	net6.0	1.34μs	1.2ns	4.65ns	0.0142	0	1.02 KB
#6490	`ExecuteNonQuery`	netcoreapp3.1	1.83μs	2.51ns	9.72ns	0.0136	0	1.02 KB
#6490	`ExecuteNonQuery`	net472	2.06μs	2ns	7.74ns	0.156	0.00101	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.17μs	1.67ns	6.26ns	0.0135	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.59μs	0.706ns	2.74ns	0.0128	976 B
master	`CallElasticsearch`	net472	2.54μs	0.707ns	2.55ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.3μs	1.42ns	5.32ns	0.0129	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.63μs	1.98ns	7.41ns	0.014	1.02 KB
master	`CallElasticsearchAsync`	net472	2.54μs	0.793ns	2.97ns	0.166	1.05 KB
#6490	`CallElasticsearch`	net6.0	1.18μs	1.14ns	4.27ns	0.0136	976 B
#6490	`CallElasticsearch`	netcoreapp3.1	1.61μs	0.537ns	2.01ns	0.0128	976 B
#6490	`CallElasticsearch`	net472	2.61μs	0.896ns	3.23ns	0.158	995 B
#6490	`CallElasticsearchAsync`	net6.0	1.38μs	0.673ns	2.52ns	0.0137	952 B
#6490	`CallElasticsearchAsync`	netcoreapp3.1	1.67μs	1.23ns	4.62ns	0.0142	1.02 KB
#6490	`CallElasticsearchAsync`	net472	2.55μs	1.47ns	5.71ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.29μs	0.612ns	2.37ns	0.0135	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.66μs	0.905ns	3.5ns	0.0132	952 B
master	`ExecuteAsync`	net472	1.84μs	0.438ns	1.64ns	0.145	915 B
#6490	`ExecuteAsync`	net6.0	1.24μs	0.515ns	1.86ns	0.0139	952 B
#6490	`ExecuteAsync`	netcoreapp3.1	1.61μs	0.493ns	1.84ns	0.0131	952 B
#6490	`ExecuteAsync`	net472	1.87μs	0.64ns	2.48ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.37μs	1.59ns	5.95ns	0.0327	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.42μs	1.45ns	5.62ns	0.0379	2.85 KB
master	`SendAsync`	net472	7.31μs	1.48ns	5.75ns	0.495	3.12 KB
#6490	`SendAsync`	net6.0	4.35μs	1.51ns	5.45ns	0.0326	2.31 KB
#6490	`SendAsync`	netcoreapp3.1	5.34μs	2.16ns	8.35ns	0.0375	2.85 KB
#6490	`SendAsync`	net472	7.37μs	2.36ns	9.14ns	0.494	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.5μs	0.833ns	3.12ns	0.0232	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.29μs	1.62ns	6.06ns	0.0218	1.64 KB
master	`EnrichedLog`	net472	2.62μs	1.43ns	5.54ns	0.249	1.57 KB
#6490	`EnrichedLog`	net6.0	1.53μs	0.778ns	3.01ns	0.023	1.64 KB
#6490	`EnrichedLog`	netcoreapp3.1	2.28μs	0.881ns	3.17ns	0.0216	1.64 KB
#6490	`EnrichedLog`	net472	2.74μs	2.25ns	8.44ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	118μs	271ns	1.05μs	0.0594	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	122μs	156ns	604ns	0	0	4.28 KB
master	`EnrichedLog`	net472	151μs	211ns	816ns	0.679	0.226	4.46 KB
#6490	`EnrichedLog`	net6.0	116μs	279ns	1.08μs	0.0574	0	4.28 KB
#6490	`EnrichedLog`	netcoreapp3.1	123μs	142ns	548ns	0	0	4.28 KB
#6490	`EnrichedLog`	net472	153μs	179ns	692ns	0.684	0.228	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.95μs	2.48ns	9.27ns	0.0311	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.07μs	3.37ns	13ns	0.0305	2.2 KB
master	`EnrichedLog`	net472	4.91μs	0.977ns	3.79ns	0.32	2.02 KB
#6490	`EnrichedLog`	net6.0	3.07μs	1.11ns	4.02ns	0.0308	2.2 KB
#6490	`EnrichedLog`	netcoreapp3.1	4.2μs	1.14ns	4.41ns	0.0294	2.2 KB
#6490	`EnrichedLog`	net472	4.96μs	1.23ns	4.76ns	0.319	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.4μs	0.819ns	3.06ns	0.0161	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.81μs	1.16ns	4.48ns	0.0155	1.14 KB
master	`SendReceive`	net472	2.09μs	1.97ns	7.61ns	0.183	1.16 KB
#6490	`SendReceive`	net6.0	1.41μs	0.679ns	2.63ns	0.0161	1.14 KB
#6490	`SendReceive`	netcoreapp3.1	1.79μs	0.756ns	2.93ns	0.0153	1.14 KB
#6490	`SendReceive`	net472	2.07μs	1.6ns	6.2ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.79μs	0.673ns	2.52ns	0.0223	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.92μs	1.39ns	5.02ns	0.0215	1.65 KB
master	`EnrichedLog`	net472	4.38μs	2.32ns	8.69ns	0.323	2.04 KB
#6490	`EnrichedLog`	net6.0	2.61μs	1.16ns	4.51ns	0.0222	1.6 KB
#6490	`EnrichedLog`	netcoreapp3.1	3.73μs	3.21ns	12.4ns	0.0224	1.65 KB
#6490	`EnrichedLog`	net472	4.38μs	1.74ns	6.73ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6490

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1	1.185	804.22	678.77	bimodal

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	404ns	0.252ns	0.942ns	0.00815	576 B
master	`StartFinishSpan`	netcoreapp3.1	543ns	0.273ns	1.06ns	0.00789	576 B
master	`StartFinishSpan`	net472	662ns	0.46ns	1.78ns	0.0917	578 B
master	`StartFinishScope`	net6.0	530ns	2.9ns	15.9ns	0.00978	696 B
master	`StartFinishScope`	netcoreapp3.1	804ns	1.43ns	5.54ns	0.00926	696 B
master	`StartFinishScope`	net472	832ns	0.577ns	2.23ns	0.104	658 B
#6490	`StartFinishSpan`	net6.0	404ns	0.218ns	0.845ns	0.00816	576 B
#6490	`StartFinishSpan`	netcoreapp3.1	561ns	0.391ns	1.52ns	0.00789	576 B
#6490	`StartFinishSpan`	net472	712ns	0.466ns	1.81ns	0.0915	578 B
#6490	`StartFinishScope`	net6.0	551ns	0.309ns	1.16ns	0.00962	696 B
#6490	`StartFinishScope`	netcoreapp3.1	691ns	3.46ns	15.5ns	0.00953	696 B
#6490	`StartFinishScope`	net472	829ns	1.4ns	5.42ns	0.105	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6490

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.212	714.08	589.39

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	714ns	0.47ns	1.82ns	0.00953	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	911ns	0.601ns	2.33ns	0.00915	696 B
master	`RunOnMethodBegin`	net472	1.05μs	0.872ns	3.38ns	0.104	658 B
#6490	`RunOnMethodBegin`	net6.0	589ns	0.534ns	2.07ns	0.00975	696 B
#6490	`RunOnMethodBegin`	netcoreapp3.1	947ns	1.25ns	4.85ns	0.00923	696 B
#6490	`RunOnMethodBegin`	net472	1.05μs	0.695ns	2.69ns	0.104	658 B

fix UpdateVendoredCode on mac

aacce3d

vandonr requested a review from a team as a code owner December 31, 2024 11:04

github-actions bot added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Dec 31, 2024

andrewlock approved these changes Dec 31, 2024

View reviewed changes

bouwkast approved these changes Dec 31, 2024

View reviewed changes

vandonr merged commit 47d26de into master Dec 31, 2024
99 of 102 checks passed

vandonr deleted the vandonr/macfix branch December 31, 2024 13:41

github-actions bot added this to the vNext-v3 milestone Dec 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix UpdateVendoredCode on mac #6490

fix UpdateVendoredCode on mac #6490

vandonr commented Dec 31, 2024

andrewlock left a comment

andrewlock commented Dec 31, 2024

datadog-ddstaging bot commented Dec 31, 2024 •

edited

Loading

andrewlock commented Dec 31, 2024

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6490

Raw results

Faster 🎉 in #6490

Raw results

fix UpdateVendoredCode on mac #6490

fix UpdateVendoredCode on mac #6490

Conversation

vandonr commented Dec 31, 2024

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

andrewlock left a comment

Choose a reason for hiding this comment

andrewlock commented Dec 31, 2024

Execution-Time Benchmarks Report ⏱️

datadog-ddstaging bot commented Dec 31, 2024 • edited Loading

Datadog Report

andrewlock commented Dec 31, 2024

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6490

Raw results

Faster 🎉 in #6490

Raw results

datadog-ddstaging bot commented Dec 31, 2024 •

edited

Loading