APMSP-1711 check CARGO_TARGET_DIR when calculating artifact location for bin tests

[ekump]

What does this PR do?

When bin_tests generate binaries check if $CARGO_TARGET_DIR is set, and the target directory is not the default target/. If this is the case then use this as the root of the artifact location.

Motivation

bin_tests fail when attempting to execute them inside a linux docker container that has $CARGO_TARGET_DIR set in order to avoid build conflicts with the host OS.

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 70.93%. Comparing base (ccca1d5) to head (4c56fb6).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #820      +/-   ##
==========================================
- Coverage   70.94%   70.93%   -0.02%     
==========================================
  Files         315      315              
  Lines       46037    46041       +4     
==========================================
- Hits        32663    32659       -4     
- Misses      13374    13382       +8     

Components	Coverage Δ
crashtracker	38.00% <ø> (ø)
crashtracker-ffi	5.63% <ø> (ø)
datadog-alloc	98.73% <ø> (ø)
data-pipeline	91.71% <ø> (ø)
data-pipeline-ffi	90.08% <ø> (ø)
ddcommon	82.08% <ø> (ø)
ddcommon-ffi	65.84% <ø> (ø)
ddtelemetry	59.51% <ø> (ø)
ddtelemetry-ffi	22.46% <ø> (ø)
dogstatsd	89.59% <ø> (ø)
dogstatsd-client	79.77% <ø> (ø)
ipc	82.69% <ø> (ø)
profiling	84.29% <ø> (ø)
profiling-ffi	77.55% <ø> (ø)
serverless	0.00% <ø> (ø)
sidecar	40.91% <ø> (ø)
sidecar-ffi	3.98% <ø> (ø)
spawn-worker	54.37% <ø> (ø)
tinybytes	93.60% <ø> (ø)
trace-mini-agent	72.48% <ø> (ø)
trace-normalization	98.23% <ø> (ø)
trace-obfuscation	95.77% <ø> (ø)
trace-protobuf	77.67% <ø> (ø)
trace-utils	93.79% <ø> (ø)

[pr-commenter]

Benchmarks

Comparison

Benchmark execution time: 2025-01-14 20:31:09

Comparing candidate commit 4c56fb6 in PR branch ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests with baseline commit ccca1d5 in branch main.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 49 metrics, 2 unstable metrics.

scenario:redis/obfuscate_redis_string

• 🟥 execution_time [+6.156µs; +6.598µs] or [+16.586%; +17.776%]

scenario:tags/replace_trace_tags

• 🟥 execution_time [+202.160ns; +218.523ns] or [+7.147%; +7.726%]

Candidate

Candidate benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
tags/replace_trace_tags	execution_time	2.934µs	3.039µs ± 0.045µs	3.041µs ± 0.018µs	3.069µs	3.107µs	3.124µs	3.134µs	3.08%	-0.448	-0.120	1.46%	0.003µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
tags/replace_trace_tags	execution_time	[3.033µs; 3.045µs] or [-0.203%; +0.203%]	None	None	None

Group 2

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
redis/obfuscate_redis_string	execution_time	42.787µs	43.492µs ± 1.093µs	43.011µs ± 0.047µs	43.058µs	45.898µs	45.960µs	46.113µs	7.21%	1.706	0.953	2.51%	0.077µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
redis/obfuscate_redis_string	execution_time	[43.340µs; 43.643µs] or [-0.348%; +0.348%]	None	None	None

Group 3

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
two way interface	execution_time	18.535µs	27.603µs ± 13.621µs	18.717µs ± 0.147µs	36.103µs	45.892µs	47.912µs	143.494µs	666.65%	3.940	27.725	49.22%	0.963µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
two way interface	execution_time	[25.715µs; 29.491µs] or [-6.839%; +6.839%]	None	None	None

Group 4

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_trace/test_trace	execution_time	298.445ns	308.517ns ± 13.144ns	301.939ns ± 2.689ns	310.838ns	337.866ns	351.833ns	353.837ns	17.19%	1.841	2.744	4.25%	0.929ns	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_trace/test_trace	execution_time	[306.696ns; 310.339ns] or [-0.590%; +0.590%]	None	None	None

Group 5

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	544.121µs	545.960µs ± 1.148µs	545.625µs ± 0.515µs	546.330µs	548.267µs	549.628µs	552.152µs	1.20%	1.846	4.976	0.21%	0.081µs	1	200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	1811095.085op/s	1831644.618op/s ± 3835.256op/s	1832759.336op/s ± 1730.302op/s	1834154.332op/s	1835470.574op/s	1837343.568op/s	1837827.991op/s	0.28%	-1.824	4.846	0.21%	271.194op/s	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	475.049µs	475.893µs ± 0.260µs	475.891µs ± 0.166µs	476.057µs	476.288µs	476.547µs	476.662µs	0.16%	-0.137	0.789	0.05%	0.018µs	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	2097921.247op/s	2101312.855op/s ± 1148.327op/s	2101319.546op/s ± 730.867op/s	2102047.491op/s	2102996.943op/s	2104582.156op/s	2105046.705op/s	0.18%	0.142	0.793	0.05%	81.199op/s	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	194.416µs	194.992µs ± 0.405µs	194.959µs ± 0.105µs	195.066µs	195.209µs	195.342µs	199.150µs	2.15%	7.836	72.443	0.21%	0.029µs	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	5021343.352op/s	5128431.718op/s ± 10492.807op/s	5129275.285op/s ± 2758.240op/s	5131915.552op/s	5136739.187op/s	5138910.992op/s	5143608.883op/s	0.28%	-7.755	71.377	0.20%	741.953op/s	1	200
normalization/normalize_service/normalize_service/[empty string]	execution_time	46.937µs	47.398µs ± 0.169µs	47.378µs ± 0.114µs	47.511µs	47.685µs	47.755µs	47.871µs	1.04%	0.121	-0.299	0.36%	0.012µs	1	200
normalization/normalize_service/normalize_service/[empty string]	throughput	20889439.261op/s	21098390.410op/s ± 75097.202op/s	21106719.446op/s ± 50992.508op/s	21150321.090op/s	21215388.241op/s	21248503.334op/s	21305013.300op/s	0.94%	-0.103	-0.301	0.36%	5310.174op/s	1	200
normalization/normalize_service/normalize_service/test_ASCII	execution_time	49.885µs	50.205µs ± 0.151µs	50.200µs ± 0.082µs	50.281µs	50.395µs	50.510µs	51.454µs	2.50%	2.911	22.493	0.30%	0.011µs	1	200
normalization/normalize_service/normalize_service/test_ASCII	throughput	19434666.140op/s	19918449.730op/s ± 59245.605op/s	19920323.398op/s ± 32590.814op/s	19955492.211op/s	19997453.281op/s	20024940.284op/s	20045978.399op/s	0.63%	-2.771	21.071	0.30%	4189.297op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	[545.801µs; 546.119µs] or [-0.029%; +0.029%]	None	None	None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	[1831113.089op/s; 1832176.148op/s] or [-0.029%; +0.029%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	[475.857µs; 475.929µs] or [-0.008%; +0.008%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	[2101153.708op/s; 2101472.003op/s] or [-0.008%; +0.008%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	[194.936µs; 195.048µs] or [-0.029%; +0.029%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	[5126977.516op/s; 5129885.920op/s] or [-0.028%; +0.028%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	execution_time	[47.374µs; 47.421µs] or [-0.049%; +0.049%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	throughput	[21087982.660op/s; 21108798.160op/s] or [-0.049%; +0.049%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	execution_time	[50.184µs; 50.226µs] or [-0.042%; +0.042%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	throughput	[19910238.859op/s; 19926660.601op/s] or [-0.041%; +0.041%]	None	None	None

Group 6

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
sql/obfuscate_sql_string	execution_time	76.514µs	76.727µs ± 0.150µs	76.713µs ± 0.049µs	76.766µs	76.843µs	76.958µs	78.255µs	2.01%	6.674	60.701	0.19%	0.011µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
sql/obfuscate_sql_string	execution_time	[76.706µs; 76.748µs] or [-0.027%; +0.027%]	None	None	None

Group 7

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching deserializing traces from msgpack to their internal representation	execution_time	58.359ms	58.937ms ± 0.169ms	58.924ms ± 0.054ms	58.983ms	59.273ms	59.486ms	59.568ms	1.09%	0.277	3.577	0.29%	0.012ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching deserializing traces from msgpack to their internal representation	execution_time	[58.914ms; 58.961ms] or [-0.040%; +0.040%]	None	None	None

Group 8

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
credit_card/is_card_number/	execution_time	4.279µs	4.293µs ± 0.005µs	4.293µs ± 0.002µs	4.295µs	4.298µs	4.301µs	4.337µs	1.04%	4.203	35.105	0.11%	0.000µs	1	200
credit_card/is_card_number/	throughput	230557298.830op/s	232932105.566op/s ± 263631.787op/s	232949852.214op/s ± 123434.871op/s	233064381.939op/s	233255625.881op/s	233331678.818op/s	233712068.536op/s	0.33%	-4.137	34.389	0.11%	18641.582op/s	1	200
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	92.608µs	93.614µs ± 0.527µs	93.627µs ± 0.268µs	93.863µs	94.253µs	94.619µs	98.480µs	5.18%	3.863	34.482	0.56%	0.037µs	1	200
credit_card/is_card_number/ 3782-8224-6310-005	throughput	10154392.316op/s	10682497.519op/s ± 58960.414op/s	10680628.014op/s ± 30592.836op/s	10713247.500op/s	10759736.119op/s	10789699.596op/s	10798228.243op/s	1.10%	-3.515	30.481	0.55%	4169.131op/s	1	200
credit_card/is_card_number/ 378282246310005	execution_time	87.372µs	88.011µs ± 0.455µs	87.972µs ± 0.154µs	88.143µs	88.401µs	88.630µs	93.525µs	6.31%	8.978	106.523	0.52%	0.032µs	1	200
credit_card/is_card_number/ 378282246310005	throughput	10692295.160op/s	11362517.266op/s ± 56158.200op/s	11367274.191op/s ± 19827.620op/s	11385582.437op/s	11414296.864op/s	11423539.107op/s	11445264.878op/s	0.69%	-8.550	99.702	0.49%	3970.984op/s	1	200
credit_card/is_card_number/37828224631	execution_time	4.273µs	4.292µs ± 0.004µs	4.291µs ± 0.002µs	4.293µs	4.297µs	4.304µs	4.335µs	1.03%	4.672	47.638	0.10%	0.000µs	1	200
credit_card/is_card_number/37828224631	throughput	230666674.187op/s	233011794.031op/s ± 238898.648op/s	233031244.131op/s ± 82457.090op/s	233108939.732op/s	233257536.318op/s	233433786.974op/s	234027497.779op/s	0.43%	-4.587	46.712	0.10%	16892.685op/s	1	200
credit_card/is_card_number/378282246310005	execution_time	85.121µs	85.566µs ± 0.206µs	85.564µs ± 0.150µs	85.711µs	85.926µs	86.103µs	86.116µs	0.65%	0.335	-0.296	0.24%	0.015µs	1	200
credit_card/is_card_number/378282246310005	throughput	11612178.257op/s	11686952.686op/s ± 28102.012op/s	11687217.107op/s ± 20440.582op/s	11708490.127op/s	11726629.607op/s	11740065.975op/s	11748042.445op/s	0.52%	-0.324	-0.308	0.24%	1987.112op/s	1	200
credit_card/is_card_number/37828224631000521389798	execution_time	58.974µs	59.071µs ± 0.037µs	59.071µs ± 0.021µs	59.091µs	59.131µs	59.167µs	59.183µs	0.19%	-0.097	0.638	0.06%	0.003µs	1	200
credit_card/is_card_number/37828224631000521389798	throughput	16896779.265op/s	16928721.210op/s ± 10607.804op/s	16928666.830op/s ± 5900.931op/s	16934515.921op/s	16947911.276op/s	16954777.811op/s	16956623.262op/s	0.17%	0.102	0.639	0.06%	750.085op/s	1	200
credit_card/is_card_number/x371413321323331	execution_time	6.836µs	6.845µs ± 0.006µs	6.844µs ± 0.003µs	6.847µs	6.851µs	6.855µs	6.910µs	0.96%	6.158	62.287	0.09%	0.000µs	1	200
credit_card/is_card_number/x371413321323331	throughput	144715264.798op/s	146093909.172op/s ± 130192.659op/s	146104812.721op/s ± 58249.940op/s	146162129.392op/s	146224589.859op/s	146268468.746op/s	146294872.225op/s	0.13%	-6.088	61.292	0.09%	9206.011op/s	1	200
credit_card/is_card_number_no_luhn/	execution_time	4.267µs	4.290µs ± 0.003µs	4.290µs ± 0.002µs	4.292µs	4.294µs	4.297µs	4.299µs	0.20%	-2.181	14.574	0.07%	0.000µs	1	200
credit_card/is_card_number_no_luhn/	throughput	232620336.865op/s	233095733.441op/s ± 168578.291op/s	233077857.604op/s ± 89283.944op/s	233169531.157op/s	233369660.983op/s	233472490.990op/s	234358316.674op/s	0.55%	2.207	14.797	0.07%	11920.285op/s	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	70.204µs	71.216µs ± 0.440µs	71.211µs ± 0.295µs	71.488µs	71.969µs	72.227µs	72.395µs	1.66%	0.098	-0.282	0.62%	0.031µs	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	13813081.576op/s	14042321.367op/s ± 86709.153op/s	14042719.884op/s ± 57997.351op/s	14105223.169op/s	14183512.414op/s	14224778.653op/s	14244162.318op/s	1.43%	-0.066	-0.293	0.62%	6131.263op/s	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	65.469µs	66.293µs ± 0.322µs	66.311µs ± 0.219µs	66.509µs	66.856µs	67.123µs	67.140µs	1.25%	0.141	-0.153	0.48%	0.023µs	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	14894347.782op/s	15084978.642op/s ± 73182.854op/s	15080400.166op/s ± 49846.143op/s	15137540.723op/s	15199684.108op/s	15242338.771op/s	15274460.303op/s	1.29%	-0.114	-0.167	0.48%	5174.809op/s	1	200
credit_card/is_card_number_no_luhn/37828224631	execution_time	4.274µs	4.291µs ± 0.004µs	4.291µs ± 0.002µs	4.293µs	4.295µs	4.298µs	4.324µs	0.76%	3.641	30.528	0.09%	0.000µs	1	200
credit_card/is_card_number_no_luhn/37828224631	throughput	231293619.957op/s	233036616.821op/s ± 214866.749op/s	233043219.020op/s ± 83128.704op/s	233126115.965op/s	233286952.707op/s	233385574.791op/s	233973187.937op/s	0.40%	-3.588	30.135	0.09%	15193.373op/s	1	200
credit_card/is_card_number_no_luhn/378282246310005	execution_time	62.747µs	63.440µs ± 0.306µs	63.391µs ± 0.206µs	63.622µs	64.023µs	64.287µs	64.320µs	1.47%	0.564	0.262	0.48%	0.022µs	1	200
credit_card/is_card_number_no_luhn/378282246310005	throughput	15547176.480op/s	15763410.356op/s ± 75836.674op/s	15775073.819op/s ± 51342.487op/s	15813622.260op/s	15871342.478op/s	15913833.995op/s	15937010.479op/s	1.03%	-0.536	0.227	0.48%	5362.463op/s	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	58.968µs	59.058µs ± 0.037µs	59.059µs ± 0.022µs	59.080µs	59.119µs	59.146µs	59.179µs	0.20%	0.045	0.360	0.06%	0.003µs	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	16897879.356op/s	16932376.096op/s ± 10567.098op/s	16932135.156op/s ± 6379.981op/s	16939032.610op/s	16950232.254op/s	16956503.383op/s	16958348.600op/s	0.15%	-0.041	0.358	0.06%	747.207op/s	1	200
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	6.830µs	6.846µs ± 0.006µs	6.845µs ± 0.003µs	6.848µs	6.857µs	6.860µs	6.864µs	0.28%	0.339	1.116	0.08%	0.000µs	1	200
credit_card/is_card_number_no_luhn/x371413321323331	throughput	145684304.406op/s	146077113.969op/s ± 118459.407op/s	146089167.493op/s ± 61510.256op/s	146140952.358op/s	146235370.134op/s	146364012.808op/s	146413001.715op/s	0.22%	-0.332	1.112	0.08%	8376.345op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
credit_card/is_card_number/	execution_time	[4.292µs; 4.294µs] or [-0.016%; +0.016%]	None	None	None
credit_card/is_card_number/	throughput	[232895568.736op/s; 232968642.396op/s] or [-0.016%; +0.016%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	[93.541µs; 93.687µs] or [-0.078%; +0.078%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	throughput	[10674326.172op/s; 10690668.865op/s] or [-0.076%; +0.076%]	None	None	None
credit_card/is_card_number/ 378282246310005	execution_time	[87.948µs; 88.074µs] or [-0.072%; +0.072%]	None	None	None
credit_card/is_card_number/ 378282246310005	throughput	[11354734.280op/s; 11370300.253op/s] or [-0.068%; +0.068%]	None	None	None
credit_card/is_card_number/37828224631	execution_time	[4.291µs; 4.292µs] or [-0.014%; +0.014%]	None	None	None
credit_card/is_card_number/37828224631	throughput	[232978684.976op/s; 233044903.086op/s] or [-0.014%; +0.014%]	None	None	None
credit_card/is_card_number/378282246310005	execution_time	[85.537µs; 85.595µs] or [-0.033%; +0.033%]	None	None	None
credit_card/is_card_number/378282246310005	throughput	[11683058.017op/s; 11690847.355op/s] or [-0.033%; +0.033%]	None	None	None
credit_card/is_card_number/37828224631000521389798	execution_time	[59.066µs; 59.076µs] or [-0.009%; +0.009%]	None	None	None
credit_card/is_card_number/37828224631000521389798	throughput	[16927251.070op/s; 16930191.349op/s] or [-0.009%; +0.009%]	None	None	None
credit_card/is_card_number/x371413321323331	execution_time	[6.844µs; 6.846µs] or [-0.012%; +0.012%]	None	None	None
credit_card/is_card_number/x371413321323331	throughput	[146075865.722op/s; 146111952.622op/s] or [-0.012%; +0.012%]	None	None	None
credit_card/is_card_number_no_luhn/	execution_time	[4.290µs; 4.291µs] or [-0.010%; +0.010%]	None	None	None
credit_card/is_card_number_no_luhn/	throughput	[233072370.112op/s; 233119096.771op/s] or [-0.010%; +0.010%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	[71.155µs; 71.277µs] or [-0.086%; +0.086%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	[14030304.313op/s; 14054338.422op/s] or [-0.086%; +0.086%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	[66.248µs; 66.337µs] or [-0.067%; +0.067%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	[15074836.202op/s; 15095121.081op/s] or [-0.067%; +0.067%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	execution_time	[4.291µs; 4.292µs] or [-0.013%; +0.013%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	throughput	[233006838.357op/s; 233066395.286op/s] or [-0.013%; +0.013%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	execution_time	[63.397µs; 63.482µs] or [-0.067%; +0.067%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	throughput	[15752900.123op/s; 15773920.590op/s] or [-0.067%; +0.067%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	[59.053µs; 59.064µs] or [-0.009%; +0.009%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	[16930911.598op/s; 16933840.594op/s] or [-0.009%; +0.009%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	[6.845µs; 6.846µs] or [-0.011%; +0.011%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	throughput	[146060696.634op/s; 146093531.303op/s] or [-0.011%; +0.011%]	None	None	None

Group 9

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
write only interface	execution_time	1.377µs	3.354µs ± 1.486µs	3.124µs ± 0.028µs	3.160µs	3.880µs	14.438µs	15.536µs	397.38%	7.334	54.983	44.20%	0.105µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
write only interface	execution_time	[3.148µs; 3.560µs] or [-6.141%; +6.141%]	None	None	None

Group 10

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	299.055µs	304.158µs ± 1.548µs	304.283µs ± 1.122µs	305.219µs	306.690µs	307.071µs	307.653µs	1.11%	-0.363	-0.068	0.51%	0.109µs	1	200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	3250411.328op/s	3287845.374op/s ± 16765.582op/s	3286412.092op/s ± 12071.264op/s	3299958.633op/s	3314748.991op/s	3326856.061op/s	3343870.654op/s	1.75%	0.390	-0.028	0.51%	1185.506op/s	1	200
normalization/normalize_name/normalize_name/bad-name	execution_time	27.069µs	27.301µs ± 0.107µs	27.305µs ± 0.072µs	27.367µs	27.482µs	27.551µs	27.599µs	1.08%	0.155	-0.281	0.39%	0.008µs	1	200
normalization/normalize_name/normalize_name/bad-name	throughput	36233809.526op/s	36629001.709op/s ± 143586.936op/s	36623808.092op/s ± 96579.820op/s	36723308.582op/s	36874294.582op/s	36908542.000op/s	36942225.174op/s	0.87%	-0.135	-0.296	0.39%	10153.130op/s	1	200
normalization/normalize_name/normalize_name/good	execution_time	15.880µs	15.973µs ± 0.057µs	15.962µs ± 0.033µs	16.001µs	16.067µs	16.151µs	16.365µs	2.52%	2.141	10.461	0.36%	0.004µs	1	200
normalization/normalize_name/normalize_name/good	throughput	61106208.805op/s	62606060.957op/s ± 223091.805op/s	62648158.917op/s ± 129201.971op/s	62747875.757op/s	62871211.547op/s	62949490.714op/s	62971351.237op/s	0.52%	-2.059	9.778	0.36%	15774.973op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	[303.944µs; 304.373µs] or [-0.071%; +0.071%]	None	None	None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	[3285521.825op/s; 3290168.922op/s] or [-0.071%; +0.071%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	execution_time	[27.286µs; 27.316µs] or [-0.054%; +0.054%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	throughput	[36609101.940op/s; 36648901.477op/s] or [-0.054%; +0.054%]	None	None	None
normalization/normalize_name/normalize_name/good	execution_time	[15.965µs; 15.981µs] or [-0.050%; +0.050%]	None	None	None
normalization/normalize_name/normalize_name/good	throughput	[62575142.578op/s; 62636979.336op/s] or [-0.049%; +0.049%]	None	None	None

Group 11

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching string interning on wordpress profile	execution_time	138.176µs	139.263µs ± 0.379µs	139.220µs ± 0.155µs	139.427µs	139.740µs	140.325µs	142.367µs	2.26%	2.824	22.075	0.27%	0.027µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching string interning on wordpress profile	execution_time	[139.211µs; 139.316µs] or [-0.038%; +0.038%]	None	None	None

Group 12

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	4c56fb6	1736886011	ekump/APMSP-1711-check-cargo-target-dir-env-in-bin-tests

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
concentrator/add_spans_to_concentrator	execution_time	6.312ms	6.328ms ± 0.012ms	6.326ms ± 0.003ms	6.330ms	6.337ms	6.377ms	6.461ms	2.14%	7.299	70.528	0.19%	0.001ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
concentrator/add_spans_to_concentrator	execution_time	[6.326ms; 6.330ms] or [-0.027%; +0.027%]	None	None	None

Baseline

Omitted due to size.