Remove RuntimeMetricsShutdownSmokeTest #6317

tonyredondo · 2024-11-19T15:29:36Z

Summary of changes

PR to remove the runtime metrics shutdown smoke test

Reason for change

The test regularly fails due to a runtime bug when doing dispose.

Implementation details

Remove the test

tonyredondo · 2024-11-19T15:35:28Z

closing in favor of #6316

andrewlock · 2024-11-19T16:13:54Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6317) - mean (72ms)  : 63, 80
     .   : milestone, 72,
    master - mean (72ms)  : 64, 79
     .   : milestone, 72,

    section CallTarget+Inlining+NGEN
    This PR (6317) - mean (1,109ms)  : 1090, 1127
     .   : milestone, 1109,
    master - mean (1,107ms)  : 1090, 1125
     .   : milestone, 1107,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6317) - mean (109ms)  : 106, 111
     .   : milestone, 109,
    master - mean (109ms)  : 106, 111
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (6317) - mean (771ms)  : 755, 787
     .   : milestone, 771,
    master - mean (768ms)  : 753, 782
     .   : milestone, 768,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6317) - mean (92ms)  : 90, 94
     .   : milestone, 92,
    master - mean (92ms)  : 89, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (6317) - mean (727ms)  : 711, 743
     .   : milestone, 727,
    master - mean (724ms)  : 710, 738
     .   : milestone, 724,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6317) - mean (191ms)  : 185, 196
     .   : milestone, 191,
    master - mean (191ms)  : 183, 200
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (6317) - mean (1,212ms)  : 1191, 1232
     .   : milestone, 1212,
    master - mean (1,210ms)  : 1186, 1234
     .   : milestone, 1210,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6317) - mean (276ms)  : 271, 281
     .   : milestone, 276,
    master - mean (276ms)  : 272, 281
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (6317) - mean (943ms)  : 921, 966
     .   : milestone, 943,
    master - mean (947ms)  : 927, 967
     .   : milestone, 947,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6317) - mean (266ms)  : 263, 270
     .   : milestone, 266,
    master - mean (266ms)  : 261, 270
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (6317) - mean (933ms)  : 914, 952
     .   : milestone, 933,
    master - mean (930ms)  : 912, 949
     .   : milestone, 930,

andrewlock · 2024-11-19T18:53:27Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6317) (11.137M)   : 0, 11136606
    master (10.953M)   : 0, 10953423
    benchmarks/2.9.0 (11.033M)   : 0, 11032866

    section Automatic
    This PR (6317) (7.273M)   : 0, 7272608
    master (7.258M)   : 0, 7258128
    benchmarks/2.9.0 (7.786M)   : 0, 7785853

    section Trace stats
    master (7.591M)   : 0, 7590970

    section Manual
    master (11.208M)   : 0, 11207771

    section Manual + Automatic
    This PR (6317) (6.615M)   : 0, 6615135
    master (6.722M)   : 0, 6721955

    section DD_TRACE_ENABLED=0
    master (10.178M)   : 0, 10178373

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6317) (9.658M)   : 0, 9657946
    master (9.269M)   : 0, 9268938
    benchmarks/2.9.0 (9.495M)   : 0, 9494821

    section Automatic
    This PR (6317) (6.421M)   : 0, 6421136
    master (6.299M)   : 0, 6299432

    section Trace stats
    master (6.605M)   : 0, 6605300

    section Manual
    master (9.212M)   : 0, 9211628

    section Manual + Automatic
    This PR (6317) (5.997M)   : 0, 5997039
    master (5.797M)   : 0, 5796944

    section DD_TRACE_ENABLED=0
    master (8.676M)   : 0, 8675777

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6317) (8.760M)   : 0, 8760449
    master (9.183M)   : 0, 9182720
    benchmarks/2.9.0 (10.020M)   : 0, 10019592

    section Automatic
    This PR (6317) (5.777M)   : crit ,0, 5776766
    master (6.143M)   : 0, 6143036
    benchmarks/2.9.0 (7.255M)   : 0, 7255257

    section Trace stats
    master (6.605M)   : 0, 6604886

    section Manual
    master (9.047M)   : 0, 9046916

    section Manual + Automatic
    This PR (6317) (5.472M)   : 0, 5471878
    master (5.730M)   : 0, 5730049

    section DD_TRACE_ENABLED=0
    master (8.828M)   : 0, 8828238

andrewlock · 2024-11-19T20:25:01Z

Benchmarks Report for tracer 🐌

Benchmarks for #6317 compared to master:

1 benchmarks are faster, with geometric mean 1.137
1 benchmarks are slower, with geometric mean 1.129
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.89μs	44.2ns	276ns	0.0161	0.00806	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.4μs	57.7ns	351ns	0.0206	0.0103	0	5.8 KB
master	`StartStopWithChild`	net472	16.4μs	55.3ns	214ns	1.04	0.301	0.0976	6.21 KB
#6317	`StartStopWithChild`	net6.0	8.02μs	46.1ns	348ns	0.0162	0.00811	0	5.61 KB
#6317	`StartStopWithChild`	netcoreapp3.1	10μs	55.4ns	318ns	0.0295	0.0148	0.00492	5.8 KB
#6317	`StartStopWithChild`	net472	16.2μs	47.4ns	183ns	1.05	0.31	0.103	6.22 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	493μs	247ns	924ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	661μs	389ns	1.51μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	868μs	488ns	1.83μs	0.434	3.3 KB
#6317	`WriteAndFlushEnrichedTraces`	net6.0	482μs	374ns	1.4μs	0	2.7 KB
#6317	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	654μs	361ns	1.4μs	0	2.7 KB
#6317	`WriteAndFlushEnrichedTraces`	net472	863μs	628ns	2.35μs	0.428	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	198μs	1.12μs	7.87μs	0.194	18.73 KB
master	`SendRequest`	netcoreapp3.1	218μs	1.18μs	7.34μs	0.226	20.89 KB
master	`SendRequest`	net472	0.00163ns	0.000628ns	0.00235ns	0	0 b
#6317	`SendRequest`	net6.0	198μs	1.15μs	9.99μs	0.198	18.73 KB
#6317	`SendRequest`	netcoreapp3.1	221μs	1.27μs	10.3μs	0.21	20.89 KB
#6317	`SendRequest`	net472	0ns	0ns	0ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	568μs	2.88μs	15μs	0.566	0	0	41.65 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	693μs	3.44μs	15.4μs	0.336	0	0	41.77 KB
master	`WriteAndFlushEnrichedTraces`	net472	868μs	3.99μs	16μs	8.68	2.6	0.434	53.26 KB
#6317	`WriteAndFlushEnrichedTraces`	net6.0	582μs	2.4μs	8.98μs	0.571	0	0	41.62 KB
#6317	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	685μs	2.92μs	10.9μs	0.342	0	0	41.7 KB
#6317	`WriteAndFlushEnrichedTraces`	net472	837μs	2.65μs	9.91μs	8.39	2.52	0.419	53.26 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.42μs	1.08ns	4.17ns	0.0141	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.73μs	1.12ns	4.35ns	0.0139	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.03μs	1.3ns	5.04ns	0.156	0.00101	987 B
#6317	`ExecuteNonQuery`	net6.0	1.37μs	1.43ns	5.33ns	0.0145	0	1.02 KB
#6317	`ExecuteNonQuery`	netcoreapp3.1	1.78μs	2.58ns	9.29ns	0.0134	0	1.02 KB
#6317	`ExecuteNonQuery`	net472	2.19μs	1.91ns	7.41ns	0.156	0.00109	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.22μs	0.535ns	1.93ns	0.0135	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.57μs	0.821ns	3.07ns	0.0133	976 B
master	`CallElasticsearch`	net472	2.59μs	1.2ns	4.47ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.18μs	0.626ns	2.43ns	0.0135	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.64μs	0.647ns	2.33ns	0.0139	1.02 KB
master	`CallElasticsearchAsync`	net472	2.69μs	6ns	23.2ns	0.167	1.05 KB
#6317	`CallElasticsearch`	net6.0	1.27μs	0.615ns	2.3ns	0.0134	976 B
#6317	`CallElasticsearch`	netcoreapp3.1	1.53μs	1.06ns	4.12ns	0.0131	976 B
#6317	`CallElasticsearch`	net472	2.49μs	2.03ns	7.87ns	0.158	995 B
#6317	`CallElasticsearchAsync`	net6.0	1.24μs	0.406ns	1.57ns	0.0131	952 B
#6317	`CallElasticsearchAsync`	netcoreapp3.1	1.71μs	1.25ns	4.7ns	0.0137	1.02 KB
#6317	`CallElasticsearchAsync`	net472	2.59μs	1.29ns	4.99ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.37μs	0.791ns	2.85ns	0.0137	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.69μs	2.1ns	7.87ns	0.0126	952 B
master	`ExecuteAsync`	net472	1.77μs	0.43ns	1.55ns	0.145	915 B
#6317	`ExecuteAsync`	net6.0	1.34μs	0.461ns	1.73ns	0.0135	952 B
#6317	`ExecuteAsync`	netcoreapp3.1	1.63μs	2.06ns	7.71ns	0.0129	952 B
#6317	`ExecuteAsync`	net472	1.85μs	0.452ns	1.75ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.31μs	2.13ns	7.98ns	0.0326	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.38μs	2.11ns	8.16ns	0.0377	2.85 KB
master	`SendAsync`	net472	7.4μs	2.23ns	8.65ns	0.494	3.12 KB
#6317	`SendAsync`	net6.0	4.39μs	2.03ns	7.84ns	0.0328	2.31 KB
#6317	`SendAsync`	netcoreapp3.1	5.37μs	6.49ns	24.3ns	0.0373	2.85 KB
#6317	`SendAsync`	net472	7.37μs	3.04ns	11.8ns	0.493	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.57μs	0.836ns	3.13ns	0.0229	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.22μs	0.871ns	3.14ns	0.0221	1.64 KB
master	`EnrichedLog`	net472	2.57μs	1.07ns	4.01ns	0.249	1.57 KB
#6317	`EnrichedLog`	net6.0	1.58μs	0.947ns	3.54ns	0.023	1.64 KB
#6317	`EnrichedLog`	netcoreapp3.1	2.1μs	1.38ns	5.18ns	0.022	1.64 KB
#6317	`EnrichedLog`	net472	2.62μs	2.03ns	7.88ns	0.25	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	119μs	126ns	489ns	0	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	124μs	89.6ns	323ns	0.0622	0	4.28 KB
master	`EnrichedLog`	net472	153μs	325ns	1.26μs	0.692	0.231	4.46 KB
#6317	`EnrichedLog`	net6.0	118μs	126ns	487ns	0.0595	0	4.28 KB
#6317	`EnrichedLog`	netcoreapp3.1	125μs	176ns	681ns	0	0	4.28 KB
#6317	`EnrichedLog`	net472	152μs	173ns	671ns	0.685	0.228	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.99μs	1.03ns	4ns	0.0315	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.13μs	2.2ns	7.95ns	0.0289	2.2 KB
master	`EnrichedLog`	net472	4.87μs	1.83ns	7.08ns	0.32	2.02 KB
#6317	`EnrichedLog`	net6.0	3.01μs	1.13ns	4.22ns	0.0312	2.2 KB
#6317	`EnrichedLog`	netcoreapp3.1	4.26μs	1.64ns	6.35ns	0.0299	2.2 KB
#6317	`EnrichedLog`	net472	4.82μs	1.51ns	5.86ns	0.319	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.29μs	0.556ns	2.15ns	0.0159	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.79μs	2.8ns	10.8ns	0.0152	1.14 KB
master	`SendReceive`	net472	2.09μs	0.988ns	3.83ns	0.184	1.16 KB
#6317	`SendReceive`	net6.0	1.31μs	1.62ns	6.26ns	0.0157	1.14 KB
#6317	`SendReceive`	netcoreapp3.1	1.85μs	0.743ns	2.88ns	0.0157	1.14 KB
#6317	`SendReceive`	net472	2.07μs	2.04ns	7.9ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.7μs	0.705ns	2.64ns	0.0216	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.99μs	1.08ns	3.9ns	0.0222	1.65 KB
master	`EnrichedLog`	net472	4.47μs	3.03ns	11.7ns	0.324	2.04 KB
#6317	`EnrichedLog`	net6.0	2.61μs	0.98ns	3.67ns	0.0222	1.6 KB
#6317	`EnrichedLog`	netcoreapp3.1	3.87μs	3.11ns	12ns	0.0211	1.65 KB
#6317	`EnrichedLog`	net472	4.4μs	3.15ns	10.9ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6317

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.137	453.29	398.78

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	452ns	0.784ns	3.03ns	0.00815	576 B
master	`StartFinishSpan`	netcoreapp3.1	595ns	1.21ns	4.7ns	0.00785	576 B
master	`StartFinishSpan`	net472	659ns	1.25ns	4.83ns	0.0916	578 B
master	`StartFinishScope`	net6.0	484ns	0.751ns	2.91ns	0.00985	696 B
master	`StartFinishScope`	netcoreapp3.1	768ns	1.32ns	5.11ns	0.00953	696 B
master	`StartFinishScope`	net472	845ns	2.72ns	10.5ns	0.105	658 B
#6317	`StartFinishSpan`	net6.0	398ns	0.544ns	2.11ns	0.00809	576 B
#6317	`StartFinishSpan`	netcoreapp3.1	631ns	0.687ns	2.66ns	0.00791	576 B
#6317	`StartFinishSpan`	net472	684ns	1.29ns	5.01ns	0.0917	578 B
#6317	`StartFinishScope`	net6.0	495ns	0.329ns	1.23ns	0.00967	696 B
#6317	`StartFinishScope`	netcoreapp3.1	742ns	2.02ns	7.82ns	0.0093	696 B
#6317	`StartFinishScope`	net472	922ns	2.01ns	7.79ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6317

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1	1.129	898.50	1,014.35

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	599ns	1.38ns	5.35ns	0.00976	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	898ns	1.15ns	4.46ns	0.00942	696 B
master	`RunOnMethodBegin`	net472	1.04μs	2.57ns	9.96ns	0.104	658 B
#6317	`RunOnMethodBegin`	net6.0	661ns	1.39ns	5.37ns	0.00961	696 B
#6317	`RunOnMethodBegin`	netcoreapp3.1	1.01μs	2.3ns	8.91ns	0.0091	696 B
#6317	`RunOnMethodBegin`	net472	1.08μs	2.6ns	10.1ns	0.104	658 B

Remove RuntimeMetricsShutdownSmokeTest

508d494

tonyredondo requested a review from a team as a code owner November 19, 2024 15:29

github-actions bot added the area:tests unit tests, integration tests label Nov 19, 2024

tonyredondo closed this Nov 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove RuntimeMetricsShutdownSmokeTest #6317

Remove RuntimeMetricsShutdownSmokeTest #6317

tonyredondo commented Nov 19, 2024

tonyredondo commented Nov 19, 2024

andrewlock commented Nov 19, 2024

andrewlock commented Nov 19, 2024

andrewlock commented Nov 19, 2024

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6317

Raw results

Slower ⚠️ in #6317

Raw results

Remove RuntimeMetricsShutdownSmokeTest #6317

Remove RuntimeMetricsShutdownSmokeTest #6317

Conversation

tonyredondo commented Nov 19, 2024

Summary of changes

Reason for change

Implementation details

tonyredondo commented Nov 19, 2024

andrewlock commented Nov 19, 2024

Execution-Time Benchmarks Report ⏱️

andrewlock commented Nov 19, 2024

Throughput/Crank Report ⚡

andrewlock commented Nov 19, 2024

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6317

Raw results

Slower ⚠️ in #6317

Raw results