Handle case where the applictaion is stopped without other telemetry sent

[bwoebi]

This lead down to a path where the Stop action was actually enqueued without being ever causing a removal of the Application instances.

[ekump]

LGTM, but is there any reasonable way to test this either in libdatadog or in php?

[pr-commenter]

Benchmarks

Comparison

Benchmark execution time: 2024-11-15 21:59:55

Comparing candidate commit a03f96d in PR branch bob/handle-stop-without-telemetry with baseline commit 4660457 in branch main.

Found 2 performance improvements and 2 performance regressions! Performance is the same for 47 metrics, 2 unstable metrics.

scenario:credit_card/is_card_number_no_luhn/378282246310005

• 🟩 execution_time [-8.569µs; -7.762µs] or [-12.124%; -10.981%]
• 🟩 throughput [+1731552.331op/s; +1911466.413op/s] or [+12.216%; +13.485%]

scenario:normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて

• 🟥 execution_time [+32.682µs; +32.810µs] or [+6.978%; +7.005%]
• 🟥 throughput [-139788.300op/s; -139245.874op/s] or [-6.547%; -6.522%]

Candidate

Candidate benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
write only interface	execution_time	1.402µs	3.285µs ± 1.425µs	3.137µs ± 0.024µs	3.155µs	3.195µs	14.038µs	15.082µs	380.78%	7.555	57.317	43.28%	0.101µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
write only interface	execution_time	[3.087µs; 3.482µs] or [-6.013%; +6.013%]	None	None	None

Group 2

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_trace/test_trace	execution_time	293.902ns	306.763ns ± 14.972ns	301.045ns ± 5.773ns	308.656ns	342.567ns	357.839ns	359.040ns	19.26%	1.785	2.418	4.87%	1.059ns	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_trace/test_trace	execution_time	[304.688ns; 308.838ns] or [-0.676%; +0.676%]	None	None	None

Group 3

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
two way interface	execution_time	18.414µs	24.525µs ± 10.182µs	19.121µs ± 0.426µs	34.498µs	39.695µs	42.213µs	96.764µs	406.06%	2.740	12.787	41.41%	0.720µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
two way interface	execution_time	[23.114µs; 25.936µs] or [-5.754%; +5.754%]	None	None	None

Group 4

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
concentrator/add_spans_to_concentrator	execution_time	9.027ms	9.063ms ± 0.023ms	9.061ms ± 0.009ms	9.070ms	9.086ms	9.112ms	9.315ms	2.81%	6.852	72.136	0.25%	0.002ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
concentrator/add_spans_to_concentrator	execution_time	[9.059ms; 9.066ms] or [-0.035%; +0.035%]	None	None	None

Group 5

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	506.114µs	507.048µs ± 0.481µs	507.030µs ± 0.241µs	507.258µs	507.804µs	507.953µs	511.126µs	0.81%	3.146	24.217	0.09%	0.034µs	1	200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	1956464.972op/s	1972203.356op/s ± 1866.358op/s	1972270.915op/s ± 938.495op/s	1973345.615op/s	1974505.900op/s	1975192.489op/s	1975840.584op/s	0.18%	-3.100	23.719	0.09%	131.971op/s	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	499.668µs	501.117µs ± 0.362µs	501.149µs ± 0.209µs	501.350µs	501.618µs	501.812µs	502.294µs	0.23%	-0.632	1.585	0.07%	0.026µs	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	1990864.586op/s	1995543.268op/s ± 1440.838op/s	1995412.879op/s ± 832.277op/s	1996364.233op/s	1997893.122op/s	1999978.267op/s	2001330.885op/s	0.30%	0.639	1.598	0.07%	101.883op/s	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	179.836µs	180.231µs ± 0.191µs	180.231µs ± 0.131µs	180.352µs	180.531µs	180.677µs	180.756µs	0.29%	0.158	-0.360	0.11%	0.014µs	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	5532328.770op/s	5548449.419op/s ± 5879.574op/s	5548445.576op/s ± 4032.041op/s	5552588.720op/s	5558209.827op/s	5560337.928op/s	5560627.933op/s	0.22%	-0.153	-0.363	0.11%	415.749op/s	1	200
normalization/normalize_service/normalize_service/[empty string]	execution_time	44.905µs	45.117µs ± 0.067µs	45.123µs ± 0.048µs	45.165µs	45.223µs	45.270µs	45.335µs	0.47%	0.026	0.594	0.15%	0.005µs	1	200
normalization/normalize_service/normalize_service/[empty string]	throughput	22057833.135op/s	22164733.824op/s ± 33017.822op/s	22161885.667op/s ± 23635.685op/s	22186643.169op/s	22219600.870op/s	22240923.641op/s	22269002.922op/s	0.48%	-0.015	0.589	0.15%	2334.713op/s	1	200
normalization/normalize_service/normalize_service/test_ASCII	execution_time	48.946µs	49.151µs ± 0.081µs	49.146µs ± 0.049µs	49.212µs	49.288µs	49.331µs	49.355µs	0.43%	0.012	-0.143	0.16%	0.006µs	1	200
normalization/normalize_service/normalize_service/test_ASCII	throughput	20261391.014op/s	20345333.259op/s ± 33394.484op/s	20347552.961op/s ± 20505.038op/s	20362729.689op/s	20398806.511op/s	20421938.452op/s	20430585.404op/s	0.41%	-0.003	-0.141	0.16%	2361.347op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	[506.981µs; 507.114µs] or [-0.013%; +0.013%]	None	None	None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	[1971944.697op/s; 1972462.015op/s] or [-0.013%; +0.013%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	[501.067µs; 501.167µs] or [-0.010%; +0.010%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	[1995343.582op/s; 1995742.954op/s] or [-0.010%; +0.010%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	[180.204µs; 180.257µs] or [-0.015%; +0.015%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	[5547634.566op/s; 5549264.271op/s] or [-0.015%; +0.015%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	execution_time	[45.108µs; 45.126µs] or [-0.021%; +0.021%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	throughput	[22160157.872op/s; 22169309.777op/s] or [-0.021%; +0.021%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	execution_time	[49.140µs; 49.163µs] or [-0.023%; +0.023%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	throughput	[20340705.105op/s; 20349961.414op/s] or [-0.023%; +0.023%]	None	None	None

Group 6

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching string interning on wordpress profile	execution_time	141.505µs	142.599µs ± 0.456µs	142.555µs ± 0.189µs	142.739µs	143.093µs	143.849µs	146.933µs	3.07%	4.835	41.165	0.32%	0.032µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching string interning on wordpress profile	execution_time	[142.535µs; 142.662µs] or [-0.044%; +0.044%]	None	None	None

Group 7

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
redis/obfuscate_redis_string	execution_time	37.019µs	37.412µs ± 0.657µs	37.125µs ± 0.066µs	37.221µs	38.761µs	38.794µs	40.450µs	8.96%	1.882	2.375	1.75%	0.046µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
redis/obfuscate_redis_string	execution_time	[37.321µs; 37.503µs] or [-0.243%; +0.243%]	None	None	None

Group 8

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
tags/replace_trace_tags	execution_time	2.626µs	2.686µs ± 0.019µs	2.687µs ± 0.009µs	2.695µs	2.713µs	2.715µs	2.718µs	1.16%	-1.291	2.340	0.70%	0.001µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
tags/replace_trace_tags	execution_time	[2.683µs; 2.688µs] or [-0.097%; +0.097%]	None	None	None

Group 9

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
sql/obfuscate_sql_string	execution_time	69.246µs	69.410µs ± 0.166µs	69.397µs ± 0.028µs	69.422µs	69.473µs	69.819µs	71.461µs	2.98%	10.048	117.130	0.24%	0.012µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
sql/obfuscate_sql_string	execution_time	[69.387µs; 69.433µs] or [-0.033%; +0.033%]	None	None	None

Group 10

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching deserializing traces from msgpack to their internal representation	execution_time	59.090ms	59.613ms ± 0.300ms	59.619ms ± 0.184ms	59.720ms	60.132ms	60.666ms	61.079ms	2.45%	1.547	4.716	0.50%	0.021ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching deserializing traces from msgpack to their internal representation	execution_time	[59.572ms; 59.655ms] or [-0.070%; +0.070%]	None	None	None

Group 11

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
credit_card/is_card_number/	execution_time	4.596µs	4.615µs ± 0.008µs	4.615µs ± 0.005µs	4.620µs	4.628µs	4.630µs	4.631µs	0.36%	-0.024	-0.696	0.17%	0.001µs	1	200
credit_card/is_card_number/	throughput	215919294.564op/s	216701432.873op/s ± 378918.522op/s	216689345.474op/s ± 254085.310op/s	216981612.267op/s	217354762.681op/s	217421883.836op/s	217568675.567op/s	0.41%	0.031	-0.696	0.17%	26793.586op/s	1	200
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	93.073µs	93.515µs ± 0.325µs	93.495µs ± 0.100µs	93.577µs	93.744µs	94.296µs	97.259µs	4.03%	7.912	86.321	0.35%	0.023µs	1	200
credit_card/is_card_number/ 3782-8224-6310-005	throughput	10281846.554op/s	10693620.837op/s ± 36240.630op/s	10695746.845op/s ± 11471.427op/s	10711515.147op/s	10719104.284op/s	10738566.575op/s	10744230.775op/s	0.45%	-7.648	82.148	0.34%	2562.600op/s	1	200
credit_card/is_card_number/ 378282246310005	execution_time	85.140µs	85.380µs ± 0.337µs	85.315µs ± 0.031µs	85.352µs	85.764µs	86.216µs	89.418µs	4.81%	9.209	102.978	0.39%	0.024µs	1	200
credit_card/is_card_number/ 378282246310005	throughput	11183380.814op/s	11712559.160op/s ± 44579.282op/s	11721332.861op/s ± 4211.575op/s	11724698.709op/s	11729829.171op/s	11732699.373op/s	11745309.143op/s	0.20%	-8.975	98.705	0.38%	3152.231op/s	1	200
credit_card/is_card_number/37828224631	execution_time	4.600µs	4.616µs ± 0.009µs	4.616µs ± 0.007µs	4.621µs	4.629µs	4.634µs	4.644µs	0.60%	0.144	-0.545	0.19%	0.001µs	1	200
credit_card/is_card_number/37828224631	throughput	215345134.551op/s	216647350.132op/s ± 410003.953op/s	216639070.285op/s ± 306327.504op/s	216962921.095op/s	217324214.913op/s	217405613.834op/s	217412133.443op/s	0.36%	-0.136	-0.554	0.19%	28991.658op/s	1	200
credit_card/is_card_number/378282246310005	execution_time	81.596µs	82.016µs ± 0.218µs	81.987µs ± 0.118µs	82.103µs	82.448µs	82.739µs	82.794µs	0.99%	1.279	2.278	0.26%	0.015µs	1	200
credit_card/is_card_number/378282246310005	throughput	12078104.149op/s	12192854.980op/s ± 32234.934op/s	12197104.407op/s ± 17571.502op/s	12216484.927op/s	12235917.988op/s	12240757.782op/s	12255439.934op/s	0.48%	-1.258	2.215	0.26%	2279.354op/s	1	200
credit_card/is_card_number/37828224631000521389798	execution_time	58.856µs	58.887µs ± 0.019µs	58.887µs ± 0.015µs	58.900µs	58.922µs	58.934µs	58.940µs	0.09%	0.491	-0.485	0.03%	0.001µs	1	200
credit_card/is_card_number/37828224631000521389798	throughput	16966414.982op/s	16981573.516op/s ± 5566.501op/s	16981571.599op/s ± 4385.830op/s	16986335.767op/s	16989125.006op/s	16989906.704op/s	16990709.480op/s	0.05%	-0.490	-0.488	0.03%	393.611op/s	1	200
credit_card/is_card_number/x371413321323331	execution_time	6.818µs	6.831µs ± 0.025µs	6.822µs ± 0.002µs	6.825µs	6.898µs	6.933µs	6.982µs	2.33%	3.004	9.498	0.37%	0.002µs	1	200
credit_card/is_card_number/x371413321323331	throughput	143235441.739op/s	146385830.490op/s ± 537840.745op/s	146576430.263op/s ± 45210.024op/s	146618482.137op/s	146650831.787op/s	146664536.159op/s	146671195.884op/s	0.06%	-2.977	9.254	0.37%	38031.084op/s	1	200
credit_card/is_card_number_no_luhn/	execution_time	4.599µs	4.614µs ± 0.009µs	4.614µs ± 0.006µs	4.619µs	4.628µs	4.632µs	4.633µs	0.42%	0.192	-0.827	0.19%	0.001µs	1	200
credit_card/is_card_number_no_luhn/	throughput	215846884.910op/s	216747464.442op/s ± 403456.468op/s	216749898.502op/s ± 300393.079op/s	217085728.250op/s	217366864.527op/s	217401197.360op/s	217421215.898op/s	0.31%	-0.185	-0.831	0.19%	28528.680op/s	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	73.334µs	73.692µs ± 0.144µs	73.702µs ± 0.082µs	73.769µs	73.893µs	73.964µs	74.758µs	1.43%	1.837	13.590	0.20%	0.010µs	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	13376569.817op/s	13569981.192op/s ± 26471.256op/s	13568170.515op/s ± 15068.442op/s	13586714.571op/s	13608728.590op/s	13629004.049op/s	13636148.065op/s	0.50%	-1.766	12.996	0.19%	1871.800op/s	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	65.253µs	65.437µs ± 0.121µs	65.418µs ± 0.097µs	65.523µs	65.683µs	65.715µs	65.746µs	0.50%	0.595	-0.500	0.18%	0.009µs	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	15209949.494op/s	15281883.568op/s ± 28230.452op/s	15286404.273op/s ± 22704.539op/s	15305219.845op/s	15318762.344op/s	15320243.635op/s	15325034.977op/s	0.25%	-0.589	-0.510	0.18%	1996.194op/s	1	200
credit_card/is_card_number_no_luhn/37828224631	execution_time	4.597µs	4.613µs ± 0.009µs	4.613µs ± 0.006µs	4.618µs	4.628µs	4.630µs	4.633µs	0.44%	0.198	-0.741	0.19%	0.001µs	1	200
credit_card/is_card_number_no_luhn/37828224631	throughput	215823558.420op/s	216794275.139op/s ± 403593.970op/s	216775430.028op/s ± 279768.126op/s	217081653.938op/s	217399596.230op/s	217439675.272op/s	217536970.002op/s	0.35%	-0.191	-0.746	0.19%	28538.403op/s	1	200
credit_card/is_card_number_no_luhn/378282246310005	execution_time	62.158µs	62.514µs ± 0.129µs	62.505µs ± 0.064µs	62.610µs	62.706µs	62.813µs	62.955µs	0.72%	-0.044	0.923	0.21%	0.009µs	1	200
credit_card/is_card_number_no_luhn/378282246310005	throughput	15884236.474op/s	15996439.684op/s ± 32938.758op/s	15998722.487op/s ± 16506.690op/s	16008170.730op/s	16050523.625op/s	16084256.926op/s	16088093.202op/s	0.56%	0.062	0.918	0.21%	2329.122op/s	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	58.856µs	58.889µs ± 0.019µs	58.891µs ± 0.017µs	58.902µs	58.925µs	58.933µs	58.953µs	0.11%	0.446	-0.449	0.03%	0.001µs	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	16962556.852op/s	16981132.209op/s ± 5571.155op/s	16980564.047op/s ± 4889.890op/s	16986264.906op/s	16988446.412op/s	16989072.781op/s	16990544.762op/s	0.06%	-0.444	-0.452	0.03%	393.940op/s	1	200
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	6.818µs	6.827µs ± 0.017µs	6.822µs ± 0.002µs	6.825µs	6.855µs	6.905µs	6.933µs	1.62%	3.828	15.680	0.25%	0.001µs	1	200
credit_card/is_card_number_no_luhn/x371413321323331	throughput	144237091.841op/s	146474358.639op/s ± 360893.390op/s	146578134.948op/s ± 47694.913op/s	146625224.147op/s	146646586.516op/s	146653317.285op/s	146674712.708op/s	0.07%	-3.806	15.479	0.25%	25519.016op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
credit_card/is_card_number/	execution_time	[4.614µs; 4.616µs] or [-0.024%; +0.024%]	None	None	None
credit_card/is_card_number/	throughput	[216648918.410op/s; 216753947.336op/s] or [-0.024%; +0.024%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	[93.470µs; 93.560µs] or [-0.048%; +0.048%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	throughput	[10688598.234op/s; 10698643.440op/s] or [-0.047%; +0.047%]	None	None	None
credit_card/is_card_number/ 378282246310005	execution_time	[85.333µs; 85.426µs] or [-0.055%; +0.055%]	None	None	None
credit_card/is_card_number/ 378282246310005	throughput	[11706380.901op/s; 11718737.420op/s] or [-0.053%; +0.053%]	None	None	None
credit_card/is_card_number/37828224631	execution_time	[4.615µs; 4.617µs] or [-0.026%; +0.026%]	None	None	None
credit_card/is_card_number/37828224631	throughput	[216590527.527op/s; 216704172.736op/s] or [-0.026%; +0.026%]	None	None	None
credit_card/is_card_number/378282246310005	execution_time	[81.986µs; 82.046µs] or [-0.037%; +0.037%]	None	None	None
credit_card/is_card_number/378282246310005	throughput	[12188387.528op/s; 12197322.432op/s] or [-0.037%; +0.037%]	None	None	None
credit_card/is_card_number/37828224631000521389798	execution_time	[58.885µs; 58.890µs] or [-0.005%; +0.005%]	None	None	None
credit_card/is_card_number/37828224631000521389798	throughput	[16980802.053op/s; 16982344.980op/s] or [-0.005%; +0.005%]	None	None	None
credit_card/is_card_number/x371413321323331	execution_time	[6.828µs; 6.835µs] or [-0.051%; +0.051%]	None	None	None
credit_card/is_card_number/x371413321323331	throughput	[146311290.936op/s; 146460370.045op/s] or [-0.051%; +0.051%]	None	None	None
credit_card/is_card_number_no_luhn/	execution_time	[4.612µs; 4.615µs] or [-0.026%; +0.026%]	None	None	None
credit_card/is_card_number_no_luhn/	throughput	[216691549.256op/s; 216803379.628op/s] or [-0.026%; +0.026%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	[73.672µs; 73.712µs] or [-0.027%; +0.027%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	[13566312.531op/s; 13573649.854op/s] or [-0.027%; +0.027%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	[65.420µs; 65.454µs] or [-0.026%; +0.026%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	[15277971.099op/s; 15285796.037op/s] or [-0.026%; +0.026%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	execution_time	[4.611µs; 4.614µs] or [-0.026%; +0.026%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	throughput	[216738340.897op/s; 216850209.382op/s] or [-0.026%; +0.026%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	execution_time	[62.496µs; 62.532µs] or [-0.029%; +0.029%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	throughput	[15991874.689op/s; 16001004.679op/s] or [-0.029%; +0.029%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	[58.886µs; 58.892µs] or [-0.005%; +0.005%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	[16980360.100op/s; 16981904.317op/s] or [-0.005%; +0.005%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	[6.825µs; 6.830µs] or [-0.034%; +0.034%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	throughput	[146424342.286op/s; 146524374.992op/s] or [-0.034%; +0.034%]	None	None	None

Group 12

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	a03f96d	1731707333	bob/handle-stop-without-telemetry

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	271.843µs	273.461µs ± 0.822µs	273.350µs ± 0.632µs	274.167µs	274.855µs	275.451µs	275.645µs	0.84%	0.339	-0.599	0.30%	0.058µs	1	200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	3627849.404op/s	3656862.421op/s ± 10986.133op/s	3658311.299op/s ± 8461.595op/s	3665298.167op/s	3673738.203op/s	3676777.510op/s	3678593.894op/s	0.55%	-0.327	-0.608	0.30%	776.837op/s	1	200
normalization/normalize_name/normalize_name/bad-name	execution_time	25.769µs	26.130µs ± 0.371µs	25.895µs ± 0.107µs	26.477µs	26.779µs	26.982µs	27.028µs	4.38%	0.709	-0.996	1.42%	0.026µs	1	200
normalization/normalize_name/normalize_name/bad-name	throughput	36998194.696op/s	38277494.351op/s ± 538690.922op/s	38618086.089op/s ± 160258.361op/s	38732299.091op/s	38782461.238op/s	38804289.695op/s	38806711.748op/s	0.49%	-0.688	-1.043	1.40%	38091.200op/s	1	200
normalization/normalize_name/normalize_name/good	execution_time	15.468µs	15.718µs ± 0.067µs	15.714µs ± 0.018µs	15.740µs	15.802µs	15.836µs	16.207µs	3.14%	1.148	17.020	0.42%	0.005µs	1	200
normalization/normalize_name/normalize_name/good	throughput	61701384.836op/s	63620977.238op/s ± 268841.248op/s	63637031.344op/s ± 71797.092op/s	63693430.785op/s	63853878.497op/s	64540888.425op/s	64650191.284op/s	1.59%	-0.928	15.882	0.42%	19009.947op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	[273.347µs; 273.575µs] or [-0.042%; +0.042%]	None	None	None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	[3655339.849op/s; 3658384.993op/s] or [-0.042%; +0.042%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	execution_time	[26.079µs; 26.182µs] or [-0.197%; +0.197%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	throughput	[38202836.970op/s; 38352151.732op/s] or [-0.195%; +0.195%]	None	None	None
normalization/normalize_name/normalize_name/good	execution_time	[15.709µs; 15.728µs] or [-0.059%; +0.059%]	None	None	None
normalization/normalize_name/normalize_name/good	throughput	[63583718.426op/s; 63658236.049op/s] or [-0.059%; +0.059%]	None	None	None

Baseline

Omitted due to size.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 12 lines in your changes missing coverage. Please review.

Project coverage is 70.55%. Comparing base (4660457) to head (a03f96d).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #743      +/-   ##
==========================================
+ Coverage   70.54%   70.55%   +0.01%     
==========================================
  Files         296      296              
  Lines       43277    43281       +4     
==========================================
+ Hits        30530    30539       +9     
+ Misses      12747    12742       -5     

Components	Coverage Δ
crashtracker	37.14% <ø> (ø)
crashtracker-ffi	8.67% <ø> (ø)
datadog-alloc	98.73% <ø> (ø)
data-pipeline	91.95% <ø> (+0.06%)	⬆️
data-pipeline-ffi	0.00% <ø> (ø)
ddcommon	83.46% <ø> (ø)
ddcommon-ffi	69.12% <ø> (ø)
ddtelemetry	59.10% <ø> (ø)
ddtelemetry-ffi	22.13% <ø> (ø)
dogstatsd	89.45% <ø> (ø)
dogstatsd-client	79.77% <ø> (ø)
ipc	82.76% <ø> (ø)
profiling	84.30% <ø> (ø)
profiling-ffi	77.46% <ø> (ø)
serverless	0.00% <ø> (ø)
sidecar	38.00% <0.00%> (-0.03%)	⬇️
sidecar-ffi	0.00% <ø> (ø)
spawn-worker	50.36% <ø> (ø)
tinybytes	94.77% <ø> (ø)
trace-mini-agent	72.18% <ø> (ø)
trace-normalization	98.25% <ø> (ø)
trace-obfuscation	95.77% <ø> (ø)
trace-protobuf	77.67% <ø> (ø)
trace-utils	93.15% <ø> (ø)

[bwoebi]

@ekump Some scernarios are indeed hard to test with end-to-end coverage, as testing that would need to setup a webserver onto which to fire multiple requests from the php side. And then it doesn't really have a visible side effect (except memory increase).

Testing the sidecar with actual #[tokio::test] is also not trivial either, as a lot of the handlers do tokio::spawn(), and there's no synchronization point on it. The sidecar_server as a whole is quite a bit unwieldy and possibly could benefit from refactors to make it more testable. Which probably isn't easy either.