Add verification step to create_draft_release to check SSI one-pipeline succeeded

[andrewlock]

Summary of changes

Adds a stage to create_draft_release that verifies the gitlab stages all passed before starting the release

Reason for change

Before we merged #5818 we were testing the lib injection images prior to publishing, but that PR removed the checks (we're building/testing everything in the one pipeline instead)

This PR, explicitly checks that everything passed in GitLab before starting the release.

Implementation details

Pulls all the statuses for the commit, and makes sure the GitLab ones all have a passing status.

Pros:

• If/when the one-pipeline expands, we automatically check for success.
• One pipeline can change (add/remove checks) and we automatically make sure they all passed before doing a release.

Cons:

• The one pipeline could break at some point without being a real issue, which would block our releases. So added an override we can use in case of emergency. It should only be used when the reporting is "false" though, because the one pipeline creates and publishes the SSI artifacts.
• We don't explicitly require any of the checks. This gives us flexibility (they can add or remove checks) but we won't know if we're actually checking them all (for example)
• If the GitHub/GitLab link isn't working (due to issues either side), we might not be notified about failures

Test coverage

Tested locally to confirm the VerifyReleaseReadiness works as expected. e.g. run

.\tracer\build.ps1 VerifyReleaseReadiness -CommitSha 89bf7b321e4b0643e62a54c9b3497fc19ac3615d -GITHUB_TOKEN <your token>

and it passes, but ec735117ceaa963d5033f83ed80daaa88f970867 fails (for example)

Other details

Should be back-ported to 2.x once we have the one pipeline there

[andrewlock]

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

• This PR (5865)
• master

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

• Welch test with statistical test for significance of 5%
• Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5865) - mean (73ms)  : 65, 82
     .   : milestone, 73,
    master - mean (72ms)  : 64, 80
     .   : milestone, 72,

    section CallTarget+Inlining+NGEN
    This PR (5865) - mean (1,067ms)  : 1048, 1085
     .   : milestone, 1067,
    master - mean (1,065ms)  : 1044, 1085
     .   : milestone, 1065,

Loading

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5865) - mean (109ms)  : 105, 113
     .   : milestone, 109,
    master - mean (108ms)  : 104, 112
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (5865) - mean (750ms)  : 729, 772
     .   : milestone, 750,
    master - mean (745ms)  : 721, 770
     .   : milestone, 745,

Loading

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5865) - mean (93ms)  : 90, 96
     .   : milestone, 93,
    master - mean (91ms)  : 88, 94
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (5865) - mean (704ms)  : 683, 726
     .   : milestone, 704,
    master - mean (705ms)  : 688, 722
     .   : milestone, 705,

Loading

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5865) - mean (192ms)  : 189, 194
     .   : milestone, 192,
    master - mean (192ms)  : 189, 196
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (5865) - mean (1,171ms)  : 1148, 1194
     .   : milestone, 1171,
    master - mean (1,172ms)  : 1139, 1205
     .   : milestone, 1172,

Loading

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5865) - mean (276ms)  : 272, 279
     .   : milestone, 276,
    master - mean (276ms)  : 271, 281
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (5865) - mean (923ms)  : 893, 952
     .   : milestone, 923,
    master - mean (923ms)  : 896, 949
     .   : milestone, 923,

Loading

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5865) - mean (266ms)  : 262, 269
     .   : milestone, 266,
    master - mean (266ms)  : 263, 270
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (5865) - mean (908ms)  : 882, 933
     .   : milestone, 908,
    master - mean (906ms)  : 882, 930
     .   : milestone, 906,

Loading

[datadog-ddstaging]

Datadog Report

Branch report: andrew/ci/add-checks-to-create-release
Commit report: 7ff035c
Test service: dd-trace-dotnet

✅ 0 Failed, 345586 Passed, 1762 Skipped, 14h 54m 7.02s Total Time

[andrewlock]

Benchmarks Report for tracer 🐌

Benchmarks for #5865 compared to master:

• 5 benchmarks are faster, with geometric mean 1.206
• 1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

• Mann–Whitney U test with statistical test for significance of 5%
• Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	StartStopWithChild	net6.0	7.64μs	42.5ns	272ns	0.0161	0.00803	0	5.43 KB
master	StartStopWithChild	netcoreapp3.1	9.76μs	55.6ns	412ns	0.014	0.00468	0	5.62 KB
master	StartStopWithChild	net472	16.1μs	52.8ns	204ns	1.02	0.307	0.0939	6.07 KB
#5865	StartStopWithChild	net6.0	7.72μs	44ns	341ns	0.0117	0.00391	0	5.43 KB
#5865	StartStopWithChild	netcoreapp3.1	10.1μs	53.5ns	273ns	0.0146	0.00975	0	5.62 KB
#5865	StartStopWithChild	net472	16μs	34.7ns	135ns	1.02	0.308	0.0947	6.07 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	WriteAndFlushEnrichedTraces	net6.0	472μs	475ns	1.78μs	0	0	0	2.7 KB
master	WriteAndFlushEnrichedTraces	netcoreapp3.1	630μs	567ns	2.12μs	0	0	0	2.7 KB
master	WriteAndFlushEnrichedTraces	net472	830μs	756ns	2.73μs	0.417	0	0	3.3 KB
#5865	WriteAndFlushEnrichedTraces	net6.0	465μs	456ns	1.77μs	0	0	0	2.7 KB
#5865	WriteAndFlushEnrichedTraces	netcoreapp3.1	645μs	486ns	1.88μs	0	0	0	2.7 KB
#5865	WriteAndFlushEnrichedTraces	net472	825μs	719ns	2.69μs	0.411	0	0	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	SendRequest	net6.0	211μs	1.64μs	16.2μs	0.223	0	0	18.45 KB
master	SendRequest	netcoreapp3.1	245μs	2.12μs	21μs	0.25	0	0	20.61 KB
master	SendRequest	net472	0.000939ns	0.000576ns	0.00223ns	0	0	0	0 b
#5865	SendRequest	net6.0	212μs	1.3μs	12.7μs	0.2	0	0	18.45 KB
#5865	SendRequest	netcoreapp3.1	227μs	1.72μs	17μs	0.226	0	0	20.61 KB
#5865	SendRequest	net472	0.000661ns	0.000456ns	0.00171ns	0	0	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #5865

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1	41.78 KB	41.56 KB	-220 B	-0.53%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	WriteAndFlushEnrichedTraces	net6.0	584μs	3.38μs	27.9μs	0.306	0	0	41.65 KB
master	WriteAndFlushEnrichedTraces	netcoreapp3.1	684μs	3.5μs	16μs	0.343	0	0	41.78 KB
master	WriteAndFlushEnrichedTraces	net472	898μs	4.29μs	18.2μs	8.68	2.6	0.434	53.34 KB
#5865	WriteAndFlushEnrichedTraces	net6.0	615μs	3.62μs	35.4μs	0.641	0	0	41.68 KB
#5865	WriteAndFlushEnrichedTraces	netcoreapp3.1	759μs	4.33μs	30.9μs	0.359	0	0	41.56 KB
#5865	WriteAndFlushEnrichedTraces	net472	893μs	4.4μs	18.7μs	8.19	2.59	0.431	53.28 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	ExecuteNonQuery	net6.0	1.25μs	2.35ns	9.09ns	0.0143	0	0	1.02 KB
master	ExecuteNonQuery	netcoreapp3.1	1.82μs	1.51ns	5.65ns	0.0138	0	0	1.02 KB
master	ExecuteNonQuery	net472	2.1μs	2.31ns	8.64ns	0.157	0	0	987 B
#5865	ExecuteNonQuery	net6.0	1.27μs	0.976ns	3.78ns	0.0147	0	0	1.02 KB
#5865	ExecuteNonQuery	netcoreapp3.1	1.67μs	2.01ns	7.8ns	0.014	0	0	1.02 KB
#5865	ExecuteNonQuery	net472	2.06μs	2.71ns	10.5ns	0.156	0	0	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	CallElasticsearch	net6.0	1.2μs	0.681ns	2.64ns	0.0138	0	0	976 B
master	CallElasticsearch	netcoreapp3.1	1.51μs	0.507ns	1.96ns	0.0128	0	0	976 B
master	CallElasticsearch	net472	2.46μs	1.81ns	7.03ns	0.157	0	0	995 B
master	CallElasticsearchAsync	net6.0	1.24μs	1.12ns	4.19ns	0.013	0	0	952 B
master	CallElasticsearchAsync	netcoreapp3.1	1.66μs	3.17ns	12.3ns	0.0141	0	0	1.02 KB
master	CallElasticsearchAsync	net472	2.65μs	1.32ns	5.11ns	0.166	0	0	1.05 KB
#5865	CallElasticsearch	net6.0	1.23μs	0.514ns	1.92ns	0.0137	0	0	976 B
#5865	CallElasticsearch	netcoreapp3.1	1.51μs	1.74ns	6.53ns	0.0134	0	0	976 B
#5865	CallElasticsearch	net472	2.57μs	1.99ns	7.71ns	0.158	0	0	995 B
#5865	CallElasticsearchAsync	net6.0	1.34μs	0.954ns	3.69ns	0.0134	0	0	952 B
#5865	CallElasticsearchAsync	netcoreapp3.1	1.73μs	0.862ns	3.34ns	0.014	0	0	1.02 KB
#5865	CallElasticsearchAsync	net472	2.65μs	1.79ns	6.94ns	0.166	0	0	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	ExecuteAsync	net6.0	1.25μs	1.53ns	5.92ns	0.0137	0	0	952 B
master	ExecuteAsync	netcoreapp3.1	1.63μs	1.23ns	4.76ns	0.0122	0	0	952 B
master	ExecuteAsync	net472	1.75μs	1.13ns	4.39ns	0.145	0	0	915 B
#5865	ExecuteAsync	net6.0	1.22μs	0.955ns	3.7ns	0.0133	0	0	952 B
#5865	ExecuteAsync	netcoreapp3.1	1.6μs	2.37ns	9.18ns	0.0128	0	0	952 B
#5865	ExecuteAsync	net472	1.81μs	0.844ns	3.27ns	0.145	0	0	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	SendAsync	net6.0	4.23μs	2.23ns	8.35ns	0.0297	0	0	2.22 KB
master	SendAsync	netcoreapp3.1	5.15μs	2.83ns	11ns	0.0361	0	0	2.76 KB
master	SendAsync	net472	7.7μs	3.12ns	11.7ns	0.5	0	0	3.15 KB
#5865	SendAsync	net6.0	4.16μs	4.17ns	15.6ns	0.0308	0	0	2.22 KB
#5865	SendAsync	netcoreapp3.1	5.04μs	2.41ns	9.34ns	0.0353	0	0	2.76 KB
#5865	SendAsync	net472	7.74μs	2.96ns	11.5ns	0.499	0	0	3.15 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	EnrichedLog	net6.0	1.58μs	0.93ns	3.48ns	0.0228	0	0	1.64 KB
master	EnrichedLog	netcoreapp3.1	2.07μs	0.687ns	2.48ns	0.0229	0	0	1.64 KB
master	EnrichedLog	net472	2.74μs	3.68ns	14.3ns	0.249	0	0	1.57 KB
#5865	EnrichedLog	net6.0	1.64μs	4.38ns	16.4ns	0.023	0	0	1.64 KB
#5865	EnrichedLog	netcoreapp3.1	2.1μs	0.723ns	2.61ns	0.0221	0	0	1.64 KB
#5865	EnrichedLog	net472	2.64μs	1.01ns	3.63ns	0.249	0	0	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	EnrichedLog	net6.0	114μs	209ns	808ns	0.0571	0	0	4.28 KB
master	EnrichedLog	netcoreapp3.1	120μs	99.2ns	358ns	0	0	0	4.28 KB
master	EnrichedLog	net472	149μs	114ns	440ns	0.668	0.223	0	4.46 KB
#5865	EnrichedLog	net6.0	117μs	262ns	1.01μs	0.0588	0	0	4.28 KB
#5865	EnrichedLog	netcoreapp3.1	119μs	422ns	1.64μs	0.0593	0	0	4.28 KB
#5865	EnrichedLog	net472	148μs	531ns	2.06μs	0.658	0.219	0	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	EnrichedLog	net6.0	2.96μs	0.98ns	3.67ns	0.0311	0	0	2.2 KB
master	EnrichedLog	netcoreapp3.1	4.11μs	1.56ns	6.06ns	0.0288	0	0	2.2 KB
master	EnrichedLog	net472	4.74μs	2.05ns	7.67ns	0.319	0	0	2.02 KB
#5865	EnrichedLog	net6.0	2.89μs	1ns	3.88ns	0.0304	0	0	2.2 KB
#5865	EnrichedLog	netcoreapp3.1	4.25μs	1.78ns	6.91ns	0.0299	0	0	2.2 KB
#5865	EnrichedLog	net472	4.78μs	0.876ns	3.39ns	0.32	0	0	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	SendReceive	net6.0	1.42μs	0.949ns	3.55ns	0.0157	0	0	1.14 KB
master	SendReceive	netcoreapp3.1	1.83μs	1.3ns	5.05ns	0.0155	0	0	1.14 KB
master	SendReceive	net472	2.14μs	1.01ns	3.9ns	0.183	0.00107	0	1.16 KB
#5865	SendReceive	net6.0	1.4μs	3.92ns	14.1ns	0.016	0	0	1.14 KB
#5865	SendReceive	netcoreapp3.1	1.79μs	1.56ns	6.03ns	0.0151	0	0	1.14 KB
#5865	SendReceive	net472	2.18μs	1.32ns	5.09ns	0.184	0.00109	0	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	EnrichedLog	net6.0	2.78μs	0.778ns	3.01ns	0.0223	0	0	1.6 KB
master	EnrichedLog	netcoreapp3.1	4μs	2.02ns	7.84ns	0.0219	0	0	1.65 KB
master	EnrichedLog	net472	4.46μs	1.73ns	6.72ns	0.323	0	0	2.04 KB
#5865	EnrichedLog	net6.0	2.78μs	1.22ns	4.72ns	0.0222	0	0	1.6 KB
#5865	EnrichedLog	netcoreapp3.1	3.98μs	3.17ns	12.3ns	0.0218	0	0	1.65 KB
#5865	EnrichedLog	net472	4.53μs	2.53ns	9.78ns	0.322	0	0	2.04 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5865

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.353	530.69	392.34
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472	1.163	665.91	572.55
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1	1.128	763.17	676.35

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	StartFinishSpan	net6.0	531ns	0.519ns	1.87ns	0.00811	0	0	576 B
master	StartFinishSpan	netcoreapp3.1	565ns	1.13ns	4.38ns	0.00775	0	0	576 B
master	StartFinishSpan	net472	666ns	0.577ns	2.16ns	0.0916	0	0	578 B
master	StartFinishScope	net6.0	503ns	0.737ns	2.85ns	0.00977	0	0	696 B
master	StartFinishScope	netcoreapp3.1	764ns	0.352ns	1.37ns	0.00942	0	0	696 B
master	StartFinishScope	net472	861ns	0.573ns	2.14ns	0.104	0	0	658 B
#5865	StartFinishSpan	net6.0	393ns	0.131ns	0.506ns	0.00809	0	0	576 B
#5865	StartFinishSpan	netcoreapp3.1	567ns	0.398ns	1.54ns	0.00774	0	0	576 B
#5865	StartFinishSpan	net472	572ns	0.236ns	0.914ns	0.0916	0	0	578 B
#5865	StartFinishScope	net6.0	534ns	0.146ns	0.564ns	0.00989	0	0	696 B
#5865	StartFinishScope	netcoreapp3.1	676ns	0.209ns	0.809ns	0.00935	0	0	696 B
#5865	StartFinishScope	net472	800ns	0.338ns	1.26ns	0.104	0	0	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5865

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.255	818.32	652.29
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1	1.144	1,067.92	933.69

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	RunOnMethodBegin	net6.0	820ns	1.96ns	7.35ns	0.00988	0	0	696 B
master	RunOnMethodBegin	netcoreapp3.1	1.07μs	0.629ns	2.44ns	0.00957	0	0	696 B
master	RunOnMethodBegin	net472	1.06μs	2.8ns	10.8ns	0.104	0	0	658 B
#5865	RunOnMethodBegin	net6.0	652ns	0.158ns	0.592ns	0.00979	0	0	696 B
#5865	RunOnMethodBegin	netcoreapp3.1	935ns	0.953ns	3.69ns	0.00936	0	0	696 B
#5865	RunOnMethodBegin	net472	1.06μs	0.562ns	2.18ns	0.104	0	0	658 B

[daniel-romano-DD]

Nothing to declare.