[BugFix] Fix hold_out_net

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1719

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (5 Unrelated Failures)

As of commit 5277b05 with merge base 07fcfb1 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Unit-tests on Linux CPU / tests (3.10) / linux-job (gh)
• Unit-tests on MacOS CPU / tests (3.8) / macos-job (gh)

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
• Unit-tests on Linux GPU, latest stable release / tests (3.8, 11.8) / linux-job (gh)
• Unit-tests on Windows GPU / unittests / windows-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#35bf28}\textsf{\Large✔\kern{0.2cm}\normalsize OK}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}38$. Worsened: $\large\color{#d91a1a}0$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	63.6715ms	62.8990ms	15.8985 Ops/s	14.7796 Ops/s	$\textbf{\color{#35bf28}+7.57\%}$
test_sync	39.1521ms	34.7917ms	28.7425 Ops/s	27.2660 Ops/s	$\textbf{\color{#35bf28}+5.42\%}$
test_async	57.1990ms	33.0391ms	30.2672 Ops/s	29.1278 Ops/s	$\color{#35bf28}+3.91\%$
test_simple	0.4970s	0.4420s	2.2624 Ops/s	2.2092 Ops/s	$\color{#35bf28}+2.41\%$
test_transformed	0.6452s	0.5953s	1.6798 Ops/s	1.6101 Ops/s	$\color{#35bf28}+4.33\%$
test_serial	1.3597s	1.3223s	0.7562 Ops/s	0.7336 Ops/s	$\color{#35bf28}+3.09\%$
test_parallel	1.3940s	1.3540s	0.7385 Ops/s	0.7443 Ops/s	$\color{#d91a1a}-0.77\%$
test_step_mdp_speed[True-True-True-True-True]	0.1564ms	22.7700μs	43.9174 KOps/s	43.4893 KOps/s	$\color{#35bf28}+0.98\%$
test_step_mdp_speed[True-True-True-True-False]	39.2730μs	13.8129μs	72.3963 KOps/s	72.1683 KOps/s	$\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-True-True-False-True]	0.1615ms	14.0446μs	71.2019 KOps/s	71.1246 KOps/s	$\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-True-True-False-False]	30.4060μs	8.3815μs	119.3100 KOps/s	117.1152 KOps/s	$\color{#35bf28}+1.87\%$
test_step_mdp_speed[True-True-False-True-True]	0.2823ms	24.1220μs	41.4559 KOps/s	40.9557 KOps/s	$\color{#35bf28}+1.22\%$
test_step_mdp_speed[True-True-False-True-False]	95.8180μs	15.0619μs	66.3927 KOps/s	65.8020 KOps/s	$\color{#35bf28}+0.90\%$
test_step_mdp_speed[True-True-False-False-True]	47.8090μs	15.0007μs	66.6637 KOps/s	65.3145 KOps/s	$\color{#35bf28}+2.07\%$
test_step_mdp_speed[True-True-False-False-False]	31.3680μs	9.6868μs	103.2335 KOps/s	102.3240 KOps/s	$\color{#35bf28}+0.89\%$
test_step_mdp_speed[True-False-True-True-True]	64.6400μs	25.4270μs	39.3283 KOps/s	38.5106 KOps/s	$\color{#35bf28}+2.12\%$
test_step_mdp_speed[True-False-True-True-False]	46.0160μs	16.6324μs	60.1236 KOps/s	59.9768 KOps/s	$\color{#35bf28}+0.24\%$
test_step_mdp_speed[True-False-True-False-True]	46.5260μs	15.2852μs	65.4226 KOps/s	65.2322 KOps/s	$\color{#35bf28}+0.29\%$
test_step_mdp_speed[True-False-True-False-False]	45.1240μs	9.8507μs	101.5157 KOps/s	101.9242 KOps/s	$\color{#d91a1a}-0.40\%$
test_step_mdp_speed[True-False-False-True-True]	62.4160μs	26.4765μs	37.7693 KOps/s	37.2301 KOps/s	$\color{#35bf28}+1.45\%$
test_step_mdp_speed[True-False-False-True-False]	46.0660μs	17.7227μs	56.4248 KOps/s	55.9602 KOps/s	$\color{#35bf28}+0.83\%$
test_step_mdp_speed[True-False-False-False-True]	40.7160μs	16.3454μs	61.1792 KOps/s	60.6060 KOps/s	$\color{#35bf28}+0.95\%$
test_step_mdp_speed[True-False-False-False-False]	65.9100μs	11.0182μs	90.7586 KOps/s	90.1874 KOps/s	$\color{#35bf28}+0.63\%$
test_step_mdp_speed[False-True-True-True-True]	58.3390μs	25.6448μs	38.9943 KOps/s	38.4369 KOps/s	$\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-True-True-True-False]	63.3070μs	16.6069μs	60.2158 KOps/s	59.4243 KOps/s	$\color{#35bf28}+1.33\%$
test_step_mdp_speed[False-True-True-False-True]	47.1680μs	17.2697μs	57.9047 KOps/s	56.3232 KOps/s	$\color{#35bf28}+2.81\%$
test_step_mdp_speed[False-True-True-False-False]	43.1400μs	10.8695μs	92.0008 KOps/s	88.7027 KOps/s	$\color{#35bf28}+3.72\%$
test_step_mdp_speed[False-True-False-True-True]	53.5090μs	26.5022μs	37.7327 KOps/s	36.7131 KOps/s	$\color{#35bf28}+2.78\%$
test_step_mdp_speed[False-True-False-True-False]	42.5700μs	17.7760μs	56.2558 KOps/s	55.8633 KOps/s	$\color{#35bf28}+0.70\%$
test_step_mdp_speed[False-True-False-False-True]	55.1930μs	18.4062μs	54.3296 KOps/s	53.0420 KOps/s	$\color{#35bf28}+2.43\%$
test_step_mdp_speed[False-True-False-False-False]	33.9740μs	12.2211μs	81.8258 KOps/s	82.2713 KOps/s	$\color{#d91a1a}-0.54\%$
test_step_mdp_speed[False-False-True-True-True]	58.3280μs	27.9288μs	35.8054 KOps/s	34.8214 KOps/s	$\color{#35bf28}+2.83\%$
test_step_mdp_speed[False-False-True-True-False]	47.4170μs	19.0853μs	52.3962 KOps/s	51.9380 KOps/s	$\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-False-True-False-True]	52.9490μs	18.6720μs	53.5560 KOps/s	52.4416 KOps/s	$\color{#35bf28}+2.13\%$
test_step_mdp_speed[False-False-True-False-False]	1.6198ms	12.2387μs	81.7082 KOps/s	81.2926 KOps/s	$\color{#35bf28}+0.51\%$
test_step_mdp_speed[False-False-False-True-True]	58.7090μs	28.6700μs	34.8796 KOps/s	33.9905 KOps/s	$\color{#35bf28}+2.62\%$
test_step_mdp_speed[False-False-False-True-False]	51.4050μs	20.2625μs	49.3523 KOps/s	49.3282 KOps/s	$\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-False-False-False-True]	47.4280μs	19.3570μs	51.6609 KOps/s	50.0874 KOps/s	$\color{#35bf28}+3.14\%$
test_step_mdp_speed[False-False-False-False-False]	35.8570μs	13.0379μs	76.6994 KOps/s	75.1500 KOps/s	$\color{#35bf28}+2.06\%$
test_values[generalized_advantage_estimate-True-True]	13.6661ms	12.0996ms	82.6476 Ops/s	75.3050 Ops/s	$\textbf{\color{#35bf28}+9.75\%}$
test_values[vec_generalized_advantage_estimate-True-True]	48.5979ms	26.8644ms	37.2240 Ops/s	36.6323 Ops/s	$\color{#35bf28}+1.62\%$
test_values[td0_return_estimate-False-False]	0.2268ms	0.1763ms	5.6713 KOps/s	5.0749 KOps/s	$\textbf{\color{#35bf28}+11.75\%}$
test_values[td1_return_estimate-False-False]	28.9859ms	26.1942ms	38.1764 Ops/s	37.6806 Ops/s	$\color{#35bf28}+1.32\%$
test_values[vec_td1_return_estimate-False-False]	34.0405ms	26.5214ms	37.7054 Ops/s	37.0078 Ops/s	$\color{#35bf28}+1.88\%$
test_values[td_lambda_return_estimate-True-False]	37.1783ms	36.5741ms	27.3418 Ops/s	26.7332 Ops/s	$\color{#35bf28}+2.28\%$
test_values[vec_td_lambda_return_estimate-True-False]	34.1162ms	26.8668ms	37.2207 Ops/s	36.5264 Ops/s	$\color{#35bf28}+1.90\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.2637ms	8.0526ms	124.1836 Ops/s	119.2311 Ops/s	$\color{#35bf28}+4.15\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.2511ms	1.8363ms	544.5856 Ops/s	509.3637 Ops/s	$\textbf{\color{#35bf28}+6.91\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	8.3377ms	0.4327ms	2.3109 KOps/s	2.2615 KOps/s	$\color{#35bf28}+2.19\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	43.8273ms	39.5493ms	25.2849 Ops/s	24.9388 Ops/s	$\color{#35bf28}+1.39\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	10.6880ms	2.5001ms	399.9864 Ops/s	394.7171 Ops/s	$\color{#35bf28}+1.33\%$
test_dqn_speed	9.3661ms	1.6215ms	616.7213 Ops/s	567.8443 Ops/s	$\textbf{\color{#35bf28}+8.61\%}$
test_ddpg_speed	11.4198ms	3.6104ms	276.9811 Ops/s	265.1816 Ops/s	$\color{#35bf28}+4.45\%$
test_sac_speed	18.8321ms	10.2215ms	97.8333 Ops/s	89.3777 Ops/s	$\textbf{\color{#35bf28}+9.46\%}$
test_redq_speed	25.3480ms	19.1950ms	52.0970 Ops/s	49.1970 Ops/s	$\textbf{\color{#35bf28}+5.89\%}$
test_redq_deprec_speed	88.6259ms	16.2886ms	61.3925 Ops/s	59.8661 Ops/s	$\color{#35bf28}+2.55\%$
test_td3_speed	18.3069ms	10.4214ms	95.9567 Ops/s	87.2150 Ops/s	$\textbf{\color{#35bf28}+10.02\%}$
test_cql_speed	46.1024ms	38.7498ms	25.8066 Ops/s	24.6492 Ops/s	$\color{#35bf28}+4.70\%$
test_a2c_speed	16.9432ms	8.0839ms	123.7027 Ops/s	110.2529 Ops/s	$\textbf{\color{#35bf28}+12.20\%}$
test_ppo_speed	16.9244ms	8.4372ms	118.5222 Ops/s	107.4793 Ops/s	$\textbf{\color{#35bf28}+10.27\%}$
test_reinforce_speed	15.4009ms	7.1860ms	139.1603 Ops/s	125.5579 Ops/s	$\textbf{\color{#35bf28}+10.83\%}$
test_iql_speed	42.2329ms	34.0095ms	29.4036 Ops/s	27.9632 Ops/s	$\textbf{\color{#35bf28}+5.15\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.1209ms	1.8639ms	536.5138 Ops/s	455.2262 Ops/s	$\textbf{\color{#35bf28}+17.86\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.2236ms	1.9750ms	506.3372 Ops/s	432.1777 Ops/s	$\textbf{\color{#35bf28}+17.16\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.1281ms	2.0842ms	479.7914 Ops/s	432.4745 Ops/s	$\textbf{\color{#35bf28}+10.94\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.8436ms	1.9307ms	517.9550 Ops/s	466.2305 Ops/s	$\textbf{\color{#35bf28}+11.09\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.9773ms	1.9901ms	502.4930 Ops/s	430.5077 Ops/s	$\textbf{\color{#35bf28}+16.72\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.5014ms	1.9923ms	501.9440 Ops/s	431.4831 Ops/s	$\textbf{\color{#35bf28}+16.33\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.4268ms	1.8919ms	528.5637 Ops/s	464.2506 Ops/s	$\textbf{\color{#35bf28}+13.85\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.0352ms	1.9905ms	502.3973 Ops/s	400.4039 Ops/s	$\textbf{\color{#35bf28}+25.47\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.0135ms	2.0338ms	491.6841 Ops/s	424.8948 Ops/s	$\textbf{\color{#35bf28}+15.72\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.7278ms	1.8707ms	534.5656 Ops/s	464.8622 Ops/s	$\textbf{\color{#35bf28}+14.99\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.3248ms	2.1238ms	470.8553 Ops/s	438.0837 Ops/s	$\textbf{\color{#35bf28}+7.48\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	2.9010ms	1.9927ms	501.8435 Ops/s	469.5147 Ops/s	$\textbf{\color{#35bf28}+6.89\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.4622ms	1.8728ms	533.9642 Ops/s	516.5813 Ops/s	$\color{#35bf28}+3.36\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.7976ms	1.9988ms	500.3077 Ops/s	477.2958 Ops/s	$\color{#35bf28}+4.82\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	2.5299ms	2.0128ms	496.8264 Ops/s	458.2240 Ops/s	$\textbf{\color{#35bf28}+8.42\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.5098ms	1.8528ms	539.7278 Ops/s	466.6043 Ops/s	$\textbf{\color{#35bf28}+15.67\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.1628ms	2.0138ms	496.5783 Ops/s	434.9662 Ops/s	$\textbf{\color{#35bf28}+14.16\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	3.0897ms	1.9712ms	507.3117 Ops/s	453.0746 Ops/s	$\textbf{\color{#35bf28}+11.97\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1708s	17.5362ms	57.0249 Ops/s	50.9844 Ops/s	$\textbf{\color{#35bf28}+11.85\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1225s	16.4629ms	60.7425 Ops/s	56.0441 Ops/s	$\textbf{\color{#35bf28}+8.38\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	0.1030s	15.9625ms	62.6467 Ops/s	54.8179 Ops/s	$\textbf{\color{#35bf28}+14.28\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1038s	15.9887ms	62.5443 Ops/s	54.3662 Ops/s	$\textbf{\color{#35bf28}+15.04\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1021s	16.0016ms	62.4939 Ops/s	54.3630 Ops/s	$\textbf{\color{#35bf28}+14.96\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	0.1186s	16.4095ms	60.9402 Ops/s	48.8991 Ops/s	$\textbf{\color{#35bf28}+24.62\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1054s	16.2329ms	61.6034 Ops/s	54.4110 Ops/s	$\textbf{\color{#35bf28}+13.22\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1131s	16.2656ms	61.4793 Ops/s	55.6564 Ops/s	$\textbf{\color{#35bf28}+10.46\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	0.1064s	16.1545ms	61.9022 Ops/s	55.8370 Ops/s	$\textbf{\color{#35bf28}+10.86\%}$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1206s	0.1202s	8.3179 Ops/s	7.9543 Ops/s	$\color{#35bf28}+4.57\%$
test_sync	0.1049s	0.1030s	9.7108 Ops/s	9.7260 Ops/s	$\color{#d91a1a}-0.16\%$
test_async	0.2690s	0.1005s	9.9517 Ops/s	9.9021 Ops/s	$\color{#35bf28}+0.50\%$
test_single_pixels	0.1437s	0.1433s	6.9775 Ops/s	6.9105 Ops/s	$\color{#35bf28}+0.97\%$
test_sync_pixels	98.2667ms	95.5233ms	10.4687 Ops/s	10.5316 Ops/s	$\color{#d91a1a}-0.60\%$
test_async_pixels	0.2463s	91.9310ms	10.8777 Ops/s	10.8870 Ops/s	$\color{#d91a1a}-0.08\%$
test_simple	0.9352s	0.8696s	1.1499 Ops/s	1.1354 Ops/s	$\color{#35bf28}+1.29\%$
test_transformed	1.1771s	1.1156s	0.8964 Ops/s	0.8870 Ops/s	$\color{#35bf28}+1.06\%$
test_serial	2.4939s	2.4430s	0.4093 Ops/s	0.4083 Ops/s	$\color{#35bf28}+0.25\%$
test_parallel	2.5713s	2.4867s	0.4021 Ops/s	0.3983 Ops/s	$\color{#35bf28}+0.97\%$
test_step_mdp_speed[True-True-True-True-True]	0.1021ms	34.0662μs	29.3546 KOps/s	28.6555 KOps/s	$\color{#35bf28}+2.44\%$
test_step_mdp_speed[True-True-True-True-False]	42.3800μs	19.8181μs	50.4589 KOps/s	47.7248 KOps/s	$\textbf{\color{#35bf28}+5.73\%}$
test_step_mdp_speed[True-True-True-False-True]	42.4910μs	19.7229μs	50.7024 KOps/s	49.1082 KOps/s	$\color{#35bf28}+3.25\%$
test_step_mdp_speed[True-True-True-False-False]	26.6800μs	11.9219μs	83.8794 KOps/s	82.4117 KOps/s	$\color{#35bf28}+1.78\%$
test_step_mdp_speed[True-True-False-True-True]	67.6010μs	35.9944μs	27.7821 KOps/s	27.1625 KOps/s	$\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-True-False-True-False]	46.9400μs	21.7074μs	46.0671 KOps/s	44.4498 KOps/s	$\color{#35bf28}+3.64\%$
test_step_mdp_speed[True-True-False-False-True]	55.1610μs	21.9925μs	45.4700 KOps/s	45.1458 KOps/s	$\color{#35bf28}+0.72\%$
test_step_mdp_speed[True-True-False-False-False]	32.8200μs	13.6621μs	73.1952 KOps/s	70.8024 KOps/s	$\color{#35bf28}+3.38\%$
test_step_mdp_speed[True-False-True-True-True]	62.8300μs	38.0592μs	26.2749 KOps/s	25.5856 KOps/s	$\color{#35bf28}+2.69\%$
test_step_mdp_speed[True-False-True-True-False]	45.9400μs	23.4730μs	42.6022 KOps/s	42.1107 KOps/s	$\color{#35bf28}+1.17\%$
test_step_mdp_speed[True-False-True-False-True]	43.2610μs	22.1135μs	45.2212 KOps/s	45.1953 KOps/s	$\color{#35bf28}+0.06\%$
test_step_mdp_speed[True-False-True-False-False]	38.7500μs	13.5584μs	73.7551 KOps/s	71.3787 KOps/s	$\color{#35bf28}+3.33\%$
test_step_mdp_speed[True-False-False-True-True]	70.4700μs	39.6940μs	25.1928 KOps/s	24.5900 KOps/s	$\color{#35bf28}+2.45\%$
test_step_mdp_speed[True-False-False-True-False]	48.7910μs	25.4217μs	39.3365 KOps/s	38.3247 KOps/s	$\color{#35bf28}+2.64\%$
test_step_mdp_speed[True-False-False-False-True]	43.1510μs	24.3401μs	41.0844 KOps/s	40.9329 KOps/s	$\color{#35bf28}+0.37\%$
test_step_mdp_speed[True-False-False-False-False]	38.1400μs	15.3056μs	65.3355 KOps/s	61.7167 KOps/s	$\textbf{\color{#35bf28}+5.86\%}$
test_step_mdp_speed[False-True-True-True-True]	62.4710μs	37.0925μs	26.9596 KOps/s	25.5339 KOps/s	$\textbf{\color{#35bf28}+5.58\%}$
test_step_mdp_speed[False-True-True-True-False]	42.8500μs	23.6607μs	42.2641 KOps/s	41.3171 KOps/s	$\color{#35bf28}+2.29\%$
test_step_mdp_speed[False-True-True-False-True]	52.0300μs	26.1444μs	38.2491 KOps/s	37.7717 KOps/s	$\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-True-True-False-False]	36.6700μs	15.5149μs	64.4540 KOps/s	63.2594 KOps/s	$\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-True-False-True-True]	67.3910μs	40.1403μs	24.9126 KOps/s	24.6116 KOps/s	$\color{#35bf28}+1.22\%$
test_step_mdp_speed[False-True-False-True-False]	53.6510μs	25.4137μs	39.3489 KOps/s	38.4927 KOps/s	$\color{#35bf28}+2.22\%$
test_step_mdp_speed[False-True-False-False-True]	53.7710μs	27.8214μs	35.9435 KOps/s	36.2267 KOps/s	$\color{#d91a1a}-0.78\%$
test_step_mdp_speed[False-True-False-False-False]	63.2510μs	17.4534μs	57.2953 KOps/s	56.5978 KOps/s	$\color{#35bf28}+1.23\%$
test_step_mdp_speed[False-False-True-True-True]	65.0400μs	41.7258μs	23.9660 KOps/s	22.9196 KOps/s	$\color{#35bf28}+4.57\%$
test_step_mdp_speed[False-False-True-True-False]	52.8110μs	27.2513μs	36.6955 KOps/s	35.8671 KOps/s	$\color{#35bf28}+2.31\%$
test_step_mdp_speed[False-False-True-False-True]	64.0900μs	28.1572μs	35.5149 KOps/s	36.1501 KOps/s	$\color{#d91a1a}-1.76\%$
test_step_mdp_speed[False-False-True-False-False]	38.6210μs	17.3638μs	57.5911 KOps/s	56.8824 KOps/s	$\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-False-False-True-True]	77.9210μs	43.5402μs	22.9673 KOps/s	22.8305 KOps/s	$\color{#35bf28}+0.60\%$
test_step_mdp_speed[False-False-False-True-False]	55.0100μs	29.2170μs	34.2267 KOps/s	33.5362 KOps/s	$\color{#35bf28}+2.06\%$
test_step_mdp_speed[False-False-False-False-True]	80.1410μs	29.7017μs	33.6682 KOps/s	34.0349 KOps/s	$\color{#d91a1a}-1.08\%$
test_step_mdp_speed[False-False-False-False-False]	38.1900μs	19.1343μs	52.2623 KOps/s	51.8511 KOps/s	$\color{#35bf28}+0.79\%$
test_values[generalized_advantage_estimate-True-True]	26.7438ms	25.9058ms	38.6014 Ops/s	38.4227 Ops/s	$\color{#35bf28}+0.46\%$
test_values[vec_generalized_advantage_estimate-True-True]	87.3655ms	3.3111ms	302.0115 Ops/s	292.4080 Ops/s	$\color{#35bf28}+3.28\%$
test_values[td0_return_estimate-False-False]	99.8210μs	65.4611μs	15.2762 KOps/s	14.9053 KOps/s	$\color{#35bf28}+2.49\%$
test_values[td1_return_estimate-False-False]	58.0451ms	57.4366ms	17.4105 Ops/s	17.2458 Ops/s	$\color{#35bf28}+0.96\%$
test_values[vec_td1_return_estimate-False-False]	2.0371ms	1.7333ms	576.9477 Ops/s	565.3974 Ops/s	$\color{#35bf28}+2.04\%$
test_values[td_lambda_return_estimate-True-False]	91.6951ms	90.7572ms	11.0184 Ops/s	10.8326 Ops/s	$\color{#35bf28}+1.72\%$
test_values[vec_td_lambda_return_estimate-True-False]	2.0297ms	1.7393ms	574.9429 Ops/s	571.1725 Ops/s	$\color{#35bf28}+0.66\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	26.5629ms	26.3739ms	37.9163 Ops/s	37.8278 Ops/s	$\color{#35bf28}+0.23\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.8807ms	0.7267ms	1.3761 KOps/s	1.3658 KOps/s	$\color{#35bf28}+0.75\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7338ms	0.6818ms	1.4667 KOps/s	1.4492 KOps/s	$\color{#35bf28}+1.21\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5297ms	1.4788ms	676.2338 Ops/s	672.6086 Ops/s	$\color{#35bf28}+0.54\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.9778ms	0.7122ms	1.4041 KOps/s	1.3932 KOps/s	$\color{#35bf28}+0.78\%$
test_dqn_speed	7.9590ms	1.4684ms	681.0111 Ops/s	669.5175 Ops/s	$\color{#35bf28}+1.72\%$
test_ddpg_speed	6.1186ms	3.3408ms	299.3278 Ops/s	297.8699 Ops/s	$\color{#35bf28}+0.49\%$
test_sac_speed	10.0431ms	9.3080ms	107.4346 Ops/s	98.3091 Ops/s	$\textbf{\color{#35bf28}+9.28\%}$
test_redq_speed	16.9625ms	16.5959ms	60.2557 Ops/s	60.0064 Ops/s	$\color{#35bf28}+0.42\%$
test_redq_deprec_speed	14.6246ms	13.1313ms	76.1538 Ops/s	75.9185 Ops/s	$\color{#35bf28}+0.31\%$
test_td3_speed	18.3240ms	9.5257ms	104.9790 Ops/s	104.5350 Ops/s	$\color{#35bf28}+0.42\%$
test_cql_speed	33.6705ms	32.3970ms	30.8671 Ops/s	31.0205 Ops/s	$\color{#d91a1a}-0.49\%$
test_a2c_speed	0.1062s	7.9760ms	125.3766 Ops/s	135.9512 Ops/s	$\textbf{\color{#d91a1a}-7.78\%}$
test_ppo_speed	8.9902ms	7.5305ms	132.7939 Ops/s	130.9210 Ops/s	$\color{#35bf28}+1.43\%$
test_reinforce_speed	7.8837ms	6.2807ms	159.2180 Ops/s	159.0967 Ops/s	$\color{#35bf28}+0.08\%$
test_iql_speed	28.8807ms	27.4132ms	36.4788 Ops/s	36.7462 Ops/s	$\color{#d91a1a}-0.73\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.0860ms	2.4784ms	403.4829 Ops/s	396.6510 Ops/s	$\color{#35bf28}+1.72\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.7670ms	2.6555ms	376.5751 Ops/s	367.9371 Ops/s	$\color{#35bf28}+2.35\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	4.0305ms	2.6359ms	379.3806 Ops/s	367.8756 Ops/s	$\color{#35bf28}+3.13\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.0514ms	2.4427ms	409.3775 Ops/s	399.9672 Ops/s	$\color{#35bf28}+2.35\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	3.5412ms	2.6517ms	377.1126 Ops/s	367.9844 Ops/s	$\color{#35bf28}+2.48\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	4.0394ms	2.6504ms	377.3002 Ops/s	369.7711 Ops/s	$\color{#35bf28}+2.04\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.1540ms	2.4429ms	409.3462 Ops/s	397.2771 Ops/s	$\color{#35bf28}+3.04\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.7709ms	2.6501ms	377.3403 Ops/s	369.2400 Ops/s	$\color{#35bf28}+2.19\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.0298ms	2.6526ms	376.9896 Ops/s	366.9394 Ops/s	$\color{#35bf28}+2.74\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.1694ms	2.4689ms	405.0423 Ops/s	395.7832 Ops/s	$\color{#35bf28}+2.34\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.5401ms	2.6363ms	379.3158 Ops/s	367.8216 Ops/s	$\color{#35bf28}+3.12\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.5702ms	2.6494ms	377.4495 Ops/s	367.8294 Ops/s	$\color{#35bf28}+2.62\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.6498ms	2.4340ms	410.8393 Ops/s	396.9177 Ops/s	$\color{#35bf28}+3.51\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	3.8462ms	2.6638ms	375.4095 Ops/s	368.3216 Ops/s	$\color{#35bf28}+1.92\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.8580ms	2.6472ms	377.7538 Ops/s	368.3727 Ops/s	$\color{#35bf28}+2.55\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.8791ms	2.4492ms	408.2895 Ops/s	396.9183 Ops/s	$\color{#35bf28}+2.86\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	4.1265ms	2.6537ms	376.8281 Ops/s	367.1591 Ops/s	$\color{#35bf28}+2.63\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	3.6591ms	2.6534ms	376.8683 Ops/s	368.8242 Ops/s	$\color{#35bf28}+2.18\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.2189s	19.3291ms	51.7355 Ops/s	51.5905 Ops/s	$\color{#35bf28}+0.28\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1248s	15.1179ms	66.1469 Ops/s	65.1245 Ops/s	$\color{#35bf28}+1.57\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	0.1247s	15.1078ms	66.1912 Ops/s	56.9547 Ops/s	$\textbf{\color{#35bf28}+16.22\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1267s	17.3477ms	57.6444 Ops/s	56.8746 Ops/s	$\color{#35bf28}+1.35\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1248s	17.4170ms	57.4150 Ops/s	56.8272 Ops/s	$\color{#35bf28}+1.03\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	0.1249s	17.4212ms	57.4012 Ops/s	56.9247 Ops/s	$\color{#35bf28}+0.84\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1271s	17.3682ms	57.5767 Ops/s	56.9798 Ops/s	$\color{#35bf28}+1.05\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1265s	17.3995ms	57.4728 Ops/s	65.3705 Ops/s	$\textbf{\color{#d91a1a}-12.08\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	0.1253s	15.1711ms	65.9149 Ops/s	56.7638 Ops/s	$\textbf{\color{#35bf28}+16.12\%}$