Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Make casting to 'meta' device uniform across cost modules #1715

Merged
merged 2 commits into from
Nov 27, 2023
Merged

[BugFix] Make casting to 'meta' device uniform across cost modules #1715

merged 2 commits into from
Nov 27, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Nov 27, 2023 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Nov 27, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1715

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (23 Unrelated Failures)

As of commit 7a08805 with merge base 0f93943 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 27, 2023
@vmoens vmoens added the bug Something isn't working label Nov 27, 2023
@vmoens vmoens marked this pull request as ready for review November 27, 2023 13:30
@vmoens vmoens merged commit 38d9cb7 into main Nov 27, 2023
@vmoens vmoens deleted the fix_meta_params branch November 27, 2023 13:30
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}9$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 64.4815ms 64.0260ms 15.6186 Ops/s 15.2153 Ops/s $\color{#35bf28}+2.65\%$
test_sync 36.7696ms 35.2955ms 28.3322 Ops/s 28.9217 Ops/s $\color{#d91a1a}-2.04\%$
test_async 66.3117ms 33.5281ms 29.8257 Ops/s 29.9371 Ops/s $\color{#d91a1a}-0.37\%$
test_simple 0.4950s 0.4400s 2.2725 Ops/s 2.2897 Ops/s $\color{#d91a1a}-0.75\%$
test_transformed 0.6658s 0.6187s 1.6163 Ops/s 1.6466 Ops/s $\color{#d91a1a}-1.84\%$
test_serial 1.3725s 1.3247s 0.7549 Ops/s 0.7425 Ops/s $\color{#35bf28}+1.67\%$
test_parallel 1.4089s 1.3619s 0.7342 Ops/s 0.7386 Ops/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[True-True-True-True-True] 0.2645ms 23.1958μs 43.1112 KOps/s 43.6164 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[True-True-True-True-False] 42.7200μs 14.0718μs 71.0641 KOps/s 72.4257 KOps/s $\color{#d91a1a}-1.88\%$
test_step_mdp_speed[True-True-True-False-True] 53.5400μs 14.1821μs 70.5114 KOps/s 69.8538 KOps/s $\color{#35bf28}+0.94\%$
test_step_mdp_speed[True-True-True-False-False] 30.8080μs 8.5461μs 117.0119 KOps/s 117.5366 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-True-False-True-True] 61.8660μs 24.7388μs 40.4223 KOps/s 40.6968 KOps/s $\color{#d91a1a}-0.67\%$
test_step_mdp_speed[True-True-False-True-False] 96.7220μs 15.4293μs 64.8116 KOps/s 65.8268 KOps/s $\color{#d91a1a}-1.54\%$
test_step_mdp_speed[True-True-False-False-True] 43.5620μs 15.5474μs 64.3193 KOps/s 65.1306 KOps/s $\color{#d91a1a}-1.25\%$
test_step_mdp_speed[True-True-False-False-False] 55.6350μs 9.8643μs 101.3756 KOps/s 103.3089 KOps/s $\color{#d91a1a}-1.87\%$
test_step_mdp_speed[True-False-True-True-True] 83.7770μs 25.7412μs 38.8482 KOps/s 38.7760 KOps/s $\color{#35bf28}+0.19\%$
test_step_mdp_speed[True-False-True-True-False] 55.3640μs 16.5457μs 60.4388 KOps/s 60.1850 KOps/s $\color{#35bf28}+0.42\%$
test_step_mdp_speed[True-False-True-False-True] 42.2990μs 15.4762μs 64.6155 KOps/s 65.0960 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[True-False-True-False-False] 34.5150μs 9.8447μs 101.5772 KOps/s 101.6949 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-False-False-True-True] 62.1880μs 27.2611μs 36.6823 KOps/s 37.1179 KOps/s $\color{#d91a1a}-1.17\%$
test_step_mdp_speed[True-False-False-True-False] 49.7740μs 17.9023μs 55.8586 KOps/s 56.4988 KOps/s $\color{#d91a1a}-1.13\%$
test_step_mdp_speed[True-False-False-False-True] 80.5510μs 16.7423μs 59.7290 KOps/s 60.2441 KOps/s $\color{#d91a1a}-0.86\%$
test_step_mdp_speed[True-False-False-False-False] 39.5140μs 10.8902μs 91.8258 KOps/s 91.3829 KOps/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[False-True-True-True-True] 68.7490μs 26.1795μs 38.1978 KOps/s 38.8349 KOps/s $\color{#d91a1a}-1.64\%$
test_step_mdp_speed[False-True-True-True-False] 44.8650μs 16.5696μs 60.3514 KOps/s 59.7905 KOps/s $\color{#35bf28}+0.94\%$
test_step_mdp_speed[False-True-True-False-True] 61.2350μs 18.0395μs 55.4340 KOps/s 55.9470 KOps/s $\color{#d91a1a}-0.92\%$
test_step_mdp_speed[False-True-True-False-False] 54.1420μs 11.0206μs 90.7388 KOps/s 89.8888 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[False-True-False-True-True] 64.7710μs 26.8827μs 37.1987 KOps/s 36.9233 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[False-True-False-True-False] 48.1910μs 17.8075μs 56.1561 KOps/s 56.3771 KOps/s $\color{#d91a1a}-0.39\%$
test_step_mdp_speed[False-True-False-False-True] 44.9340μs 18.7892μs 53.2219 KOps/s 52.5853 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[False-True-False-False-False] 42.3190μs 12.1623μs 82.2210 KOps/s 82.0115 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[False-False-True-True-True] 69.0600μs 28.2683μs 35.3753 KOps/s 35.1140 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[False-False-True-True-False] 47.6290μs 19.0494μs 52.4951 KOps/s 52.7253 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[False-False-True-False-True] 49.3420μs 18.7566μs 53.3145 KOps/s 52.4085 KOps/s $\color{#35bf28}+1.73\%$
test_step_mdp_speed[False-False-True-False-False] 53.5200μs 12.0747μs 82.8181 KOps/s 81.5073 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[False-False-False-True-True] 69.9110μs 28.8511μs 34.6607 KOps/s 33.8337 KOps/s $\color{#35bf28}+2.44\%$
test_step_mdp_speed[False-False-False-True-False] 80.1600μs 20.0026μs 49.9936 KOps/s 49.7036 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-False-False-False-True] 85.4680μs 19.5918μs 51.0417 KOps/s 50.6084 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-False-False-False-False] 77.5860μs 13.3452μs 74.9336 KOps/s 75.0234 KOps/s $\color{#d91a1a}-0.12\%$
test_values[generalized_advantage_estimate-True-True] 13.7506ms 12.3804ms 80.7727 Ops/s 80.7798 Ops/s $-0.01\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.0675ms 26.7224ms 37.4217 Ops/s 37.2033 Ops/s $\color{#35bf28}+0.59\%$
test_values[td0_return_estimate-False-False] 1.2673ms 0.2123ms 4.7095 KOps/s 5.1334 KOps/s $\textbf{\color{#d91a1a}-8.26\%}$
test_values[td1_return_estimate-False-False] 28.5196ms 26.5593ms 37.6516 Ops/s 37.4888 Ops/s $\color{#35bf28}+0.43\%$
test_values[vec_td1_return_estimate-False-False] 34.4285ms 26.7552ms 37.3759 Ops/s 37.6277 Ops/s $\color{#d91a1a}-0.67\%$
test_values[td_lambda_return_estimate-True-False] 40.4909ms 37.1083ms 26.9482 Ops/s 26.8487 Ops/s $\color{#35bf28}+0.37\%$
test_values[vec_td_lambda_return_estimate-True-False] 34.6516ms 26.8107ms 37.2985 Ops/s 36.8349 Ops/s $\color{#35bf28}+1.26\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.4381ms 8.2545ms 121.1461 Ops/s 120.1140 Ops/s $\color{#35bf28}+0.86\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3800ms 1.8563ms 538.7184 Ops/s 506.5228 Ops/s $\textbf{\color{#35bf28}+6.36\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 9.9152ms 0.4456ms 2.2440 KOps/s 2.2522 KOps/s $\color{#d91a1a}-0.37\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 42.1824ms 39.5727ms 25.2699 Ops/s 25.4226 Ops/s $\color{#d91a1a}-0.60\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.6467ms 2.4941ms 400.9401 Ops/s 400.5394 Ops/s $\color{#35bf28}+0.10\%$
test_dqn_speed 10.1390ms 1.6648ms 600.6757 Ops/s 562.5377 Ops/s $\textbf{\color{#35bf28}+6.78\%}$
test_ddpg_speed 12.5504ms 3.7330ms 267.8831 Ops/s 272.3631 Ops/s $\color{#d91a1a}-1.64\%$
test_sac_speed 18.8621ms 10.6777ms 93.6533 Ops/s 93.8955 Ops/s $\color{#d91a1a}-0.26\%$
test_redq_speed 29.8982ms 20.2652ms 49.3456 Ops/s 49.3811 Ops/s $\color{#d91a1a}-0.07\%$
test_redq_deprec_speed 17.8300ms 16.3544ms 61.1455 Ops/s 60.9205 Ops/s $\color{#35bf28}+0.37\%$
test_td3_speed 19.0935ms 10.9220ms 91.5585 Ops/s 90.7754 Ops/s $\color{#35bf28}+0.86\%$
test_cql_speed 48.4273ms 40.7891ms 24.5164 Ops/s 25.1916 Ops/s $\color{#d91a1a}-2.68\%$
test_a2c_speed 93.3174ms 9.2405ms 108.2190 Ops/s 116.0949 Ops/s $\textbf{\color{#d91a1a}-6.78\%}$
test_ppo_speed 17.9482ms 8.9802ms 111.3557 Ops/s 113.2718 Ops/s $\color{#d91a1a}-1.69\%$
test_reinforce_speed 15.7586ms 7.5767ms 131.9844 Ops/s 135.7332 Ops/s $\color{#d91a1a}-2.76\%$
test_iql_speed 42.9473ms 35.0206ms 28.5546 Ops/s 26.3566 Ops/s $\textbf{\color{#35bf28}+8.34\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.4325ms 1.9875ms 503.1358 Ops/s 518.5547 Ops/s $\color{#d91a1a}-2.97\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.2779ms 2.1420ms 466.8598 Ops/s 482.0667 Ops/s $\color{#d91a1a}-3.15\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.6431ms 2.1460ms 465.9865 Ops/s 443.5373 Ops/s $\textbf{\color{#35bf28}+5.06\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.9849ms 1.9845ms 503.9079 Ops/s 536.8483 Ops/s $\textbf{\color{#d91a1a}-6.14\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.1480ms 2.1506ms 464.9948 Ops/s 480.1258 Ops/s $\color{#d91a1a}-3.15\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.3348ms 2.1569ms 463.6389 Ops/s 411.1646 Ops/s $\textbf{\color{#35bf28}+12.76\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0137ms 2.0056ms 498.5946 Ops/s 520.3262 Ops/s $\color{#d91a1a}-4.18\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.9772ms 2.1194ms 471.8346 Ops/s 479.9422 Ops/s $\color{#d91a1a}-1.69\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.4163ms 2.0983ms 476.5766 Ops/s 434.7288 Ops/s $\textbf{\color{#35bf28}+9.63\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.2061ms 1.9432ms 514.6255 Ops/s 531.4470 Ops/s $\color{#d91a1a}-3.17\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.1711ms 2.0828ms 480.1329 Ops/s 479.0053 Ops/s $\color{#35bf28}+0.24\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.5582ms 2.2319ms 448.0393 Ops/s 425.4289 Ops/s $\textbf{\color{#35bf28}+5.31\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.8195ms 2.0048ms 498.8003 Ops/s 516.5135 Ops/s $\color{#d91a1a}-3.43\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.2644ms 2.1301ms 469.4561 Ops/s 480.3933 Ops/s $\color{#d91a1a}-2.28\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.8659ms 2.2038ms 453.7673 Ops/s 431.0323 Ops/s $\textbf{\color{#35bf28}+5.27\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.7446ms 2.0200ms 495.0607 Ops/s 520.4005 Ops/s $\color{#d91a1a}-4.87\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.7292ms 2.1323ms 468.9666 Ops/s 460.5228 Ops/s $\color{#35bf28}+1.83\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.4708ms 2.1574ms 463.5158 Ops/s 434.2240 Ops/s $\textbf{\color{#35bf28}+6.75\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2356s 19.8326ms 50.4220 Ops/s 58.4674 Ops/s $\textbf{\color{#d91a1a}-13.76\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1243s 17.4191ms 57.4083 Ops/s 59.0327 Ops/s $\color{#d91a1a}-2.75\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1210s 17.2680ms 57.9107 Ops/s 58.6364 Ops/s $\color{#d91a1a}-1.24\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1179s 17.1222ms 58.4036 Ops/s 57.9510 Ops/s $\color{#35bf28}+0.78\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1233s 17.2736ms 57.8919 Ops/s 58.4021 Ops/s $\color{#d91a1a}-0.87\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1187s 17.1879ms 58.1804 Ops/s 57.6693 Ops/s $\color{#35bf28}+0.89\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1268s 17.4175ms 57.4134 Ops/s 58.4096 Ops/s $\color{#d91a1a}-1.71\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1202s 16.8602ms 59.3112 Ops/s 58.8985 Ops/s $\color{#35bf28}+0.70\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1210s 17.3441ms 57.6566 Ops/s 57.7159 Ops/s $\color{#d91a1a}-0.10\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}10$. Worsened: $\large\color{#d91a1a}1$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1203s 0.1191s 8.3985 Ops/s 8.3555 Ops/s $\color{#35bf28}+0.51\%$
test_sync 0.1030s 0.1026s 9.7435 Ops/s 9.6096 Ops/s $\color{#35bf28}+1.39\%$
test_async 0.2762s 99.7407ms 10.0260 Ops/s 10.0241 Ops/s $\color{#35bf28}+0.02\%$
test_single_pixels 0.1308s 0.1298s 7.7038 Ops/s 7.0212 Ops/s $\textbf{\color{#35bf28}+9.72\%}$
test_sync_pixels 95.2122ms 93.9370ms 10.6454 Ops/s 10.5504 Ops/s $\color{#35bf28}+0.90\%$
test_async_pixels 0.2438s 90.9065ms 11.0003 Ops/s 11.1052 Ops/s $\color{#d91a1a}-0.94\%$
test_simple 0.9281s 0.8622s 1.1598 Ops/s 1.1559 Ops/s $\color{#35bf28}+0.34\%$
test_transformed 1.1652s 1.1035s 0.9062 Ops/s 0.9118 Ops/s $\color{#d91a1a}-0.62\%$
test_serial 2.4596s 2.4107s 0.4148 Ops/s 0.4164 Ops/s $\color{#d91a1a}-0.38\%$
test_parallel 2.5748s 2.5276s 0.3956 Ops/s 0.4012 Ops/s $\color{#d91a1a}-1.39\%$
test_step_mdp_speed[True-True-True-True-True] 0.1025ms 35.2770μs 28.3471 KOps/s 28.0273 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[True-True-True-True-False] 56.8140μs 20.9777μs 47.6697 KOps/s 48.5661 KOps/s $\color{#d91a1a}-1.85\%$
test_step_mdp_speed[True-True-True-False-True] 50.3430μs 20.3866μs 49.0518 KOps/s 47.8073 KOps/s $\color{#35bf28}+2.60\%$
test_step_mdp_speed[True-True-True-False-False] 39.7720μs 12.1594μs 82.2407 KOps/s 82.5118 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[True-True-False-True-True] 96.2160μs 37.5852μs 26.6062 KOps/s 27.3884 KOps/s $\color{#d91a1a}-2.86\%$
test_step_mdp_speed[True-True-False-True-False] 51.2540μs 22.9274μs 43.6160 KOps/s 44.8168 KOps/s $\color{#d91a1a}-2.68\%$
test_step_mdp_speed[True-True-False-False-True] 49.8840μs 22.3013μs 44.8404 KOps/s 43.9237 KOps/s $\color{#35bf28}+2.09\%$
test_step_mdp_speed[True-True-False-False-False] 82.9660μs 13.9708μs 71.5777 KOps/s 71.8160 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[True-False-True-True-True] 80.8650μs 39.2044μs 25.5073 KOps/s 25.8385 KOps/s $\color{#d91a1a}-1.28\%$
test_step_mdp_speed[True-False-True-True-False] 48.7130μs 23.8776μs 41.8802 KOps/s 40.6536 KOps/s $\color{#35bf28}+3.02\%$
test_step_mdp_speed[True-False-True-False-True] 52.0630μs 22.2785μs 44.8863 KOps/s 43.6038 KOps/s $\color{#35bf28}+2.94\%$
test_step_mdp_speed[True-False-True-False-False] 32.9120μs 14.0008μs 71.4244 KOps/s 71.4209 KOps/s $+0.00\%$
test_step_mdp_speed[True-False-False-True-True] 61.3340μs 40.9487μs 24.4208 KOps/s 24.7190 KOps/s $\color{#d91a1a}-1.21\%$
test_step_mdp_speed[True-False-False-True-False] 98.4060μs 26.0147μs 38.4398 KOps/s 37.7552 KOps/s $\color{#35bf28}+1.81\%$
test_step_mdp_speed[True-False-False-False-True] 40.7120μs 23.5819μs 42.4054 KOps/s 40.1239 KOps/s $\textbf{\color{#35bf28}+5.69\%}$
test_step_mdp_speed[True-False-False-False-False] 40.4330μs 16.0306μs 62.3806 KOps/s 62.6807 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-True-True-True-True] 71.0950μs 38.9157μs 25.6966 KOps/s 25.9282 KOps/s $\color{#d91a1a}-0.89\%$
test_step_mdp_speed[False-True-True-True-False] 47.7130μs 24.4818μs 40.8467 KOps/s 41.5412 KOps/s $\color{#d91a1a}-1.67\%$
test_step_mdp_speed[False-True-True-False-True] 95.0760μs 26.3334μs 37.9745 KOps/s 37.1267 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[False-True-True-False-False] 36.1120μs 16.1805μs 61.8027 KOps/s 60.6876 KOps/s $\color{#35bf28}+1.84\%$
test_step_mdp_speed[False-True-False-True-True] 63.1440μs 40.5704μs 24.6485 KOps/s 24.0250 KOps/s $\color{#35bf28}+2.60\%$
test_step_mdp_speed[False-True-False-True-False] 54.0630μs 26.5721μs 37.6334 KOps/s 37.7548 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-True-False-False-True] 51.3440μs 28.2911μs 35.3468 KOps/s 35.0413 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[False-True-False-False-False] 36.1220μs 17.7092μs 56.4678 KOps/s 55.8140 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-False-True-True-True] 0.1122ms 42.0864μs 23.7606 KOps/s 24.1047 KOps/s $\color{#d91a1a}-1.43\%$
test_step_mdp_speed[False-False-True-True-False] 65.7540μs 27.9555μs 35.7711 KOps/s 36.2938 KOps/s $\color{#d91a1a}-1.44\%$
test_step_mdp_speed[False-False-True-False-True] 52.5730μs 27.9019μs 35.8399 KOps/s 35.2624 KOps/s $\color{#35bf28}+1.64\%$
test_step_mdp_speed[False-False-True-False-False] 56.6130μs 17.9748μs 55.6333 KOps/s 55.9035 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-False-False-True-True] 69.4750μs 43.5550μs 22.9595 KOps/s 23.1325 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[False-False-False-True-False] 0.1047ms 29.9943μs 33.3396 KOps/s 33.1380 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[False-False-False-False-True] 51.7230μs 29.3137μs 34.1137 KOps/s 33.6461 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[False-False-False-False-False] 40.2930μs 19.2561μs 51.9316 KOps/s 51.7208 KOps/s $\color{#35bf28}+0.41\%$
test_values[generalized_advantage_estimate-True-True] 25.1269ms 24.4118ms 40.9639 Ops/s 38.6245 Ops/s $\textbf{\color{#35bf28}+6.06\%}$
test_values[vec_generalized_advantage_estimate-True-True] 84.3618ms 3.2342ms 309.1928 Ops/s 304.4901 Ops/s $\color{#35bf28}+1.54\%$
test_values[td0_return_estimate-False-False] 99.5760μs 63.2206μs 15.8176 KOps/s 15.0126 KOps/s $\textbf{\color{#35bf28}+5.36\%}$
test_values[td1_return_estimate-False-False] 53.0220ms 52.3765ms 19.0925 Ops/s 18.1375 Ops/s $\textbf{\color{#35bf28}+5.27\%}$
test_values[vec_td1_return_estimate-False-False] 2.0397ms 1.7128ms 583.8433 Ops/s 571.3073 Ops/s $\color{#35bf28}+2.19\%$
test_values[td_lambda_return_estimate-True-False] 86.3399ms 84.1708ms 11.8806 Ops/s 11.2583 Ops/s $\textbf{\color{#35bf28}+5.53\%}$
test_values[vec_td_lambda_return_estimate-True-False] 2.0385ms 1.7104ms 584.6569 Ops/s 576.3958 Ops/s $\color{#35bf28}+1.43\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.6991ms 23.4527ms 42.6390 Ops/s 40.0781 Ops/s $\textbf{\color{#35bf28}+6.39\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8532ms 0.7000ms 1.4285 KOps/s 1.4067 KOps/s $\color{#35bf28}+1.55\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7336ms 0.6687ms 1.4954 KOps/s 1.4342 KOps/s $\color{#35bf28}+4.27\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5720ms 1.4578ms 685.9865 Ops/s 676.4333 Ops/s $\color{#35bf28}+1.41\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9612ms 0.6940ms 1.4409 KOps/s 1.3968 KOps/s $\color{#35bf28}+3.16\%$
test_dqn_speed 7.7843ms 1.4555ms 687.0357 Ops/s 676.4411 Ops/s $\color{#35bf28}+1.57\%$
test_ddpg_speed 4.6642ms 3.2903ms 303.9282 Ops/s 302.1893 Ops/s $\color{#35bf28}+0.58\%$
test_sac_speed 94.4989ms 9.9349ms 100.6552 Ops/s 105.9409 Ops/s $\color{#d91a1a}-4.99\%$
test_redq_speed 17.3301ms 16.5486ms 60.4281 Ops/s 59.9894 Ops/s $\color{#35bf28}+0.73\%$
test_redq_deprec_speed 14.0479ms 12.8658ms 77.7253 Ops/s 77.6273 Ops/s $\color{#35bf28}+0.13\%$
test_td3_speed 18.9899ms 9.4081ms 106.2911 Ops/s 106.8548 Ops/s $\color{#d91a1a}-0.53\%$
test_cql_speed 33.1826ms 31.3428ms 31.9052 Ops/s 32.1572 Ops/s $\color{#d91a1a}-0.78\%$
test_a2c_speed 8.1945ms 6.8678ms 145.6080 Ops/s 139.9261 Ops/s $\color{#35bf28}+4.06\%$
test_ppo_speed 8.5495ms 7.1479ms 139.9015 Ops/s 133.9031 Ops/s $\color{#35bf28}+4.48\%$
test_reinforce_speed 7.3769ms 5.9974ms 166.7390 Ops/s 162.8891 Ops/s $\color{#35bf28}+2.36\%$
test_iql_speed 28.2481ms 26.6524ms 37.5200 Ops/s 36.9347 Ops/s $\color{#35bf28}+1.58\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0085ms 2.4762ms 403.8446 Ops/s 406.7439 Ops/s $\color{#d91a1a}-0.71\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 4.0675ms 2.6716ms 374.3090 Ops/s 340.1050 Ops/s $\textbf{\color{#35bf28}+10.06\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.0047ms 2.6650ms 375.2298 Ops/s 376.5828 Ops/s $\color{#d91a1a}-0.36\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0293ms 2.4741ms 404.1848 Ops/s 404.0037 Ops/s $\color{#35bf28}+0.04\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.7521ms 2.6669ms 374.9620 Ops/s 336.5678 Ops/s $\textbf{\color{#35bf28}+11.41\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.1632ms 2.6492ms 377.4699 Ops/s 377.6291 Ops/s $\color{#d91a1a}-0.04\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0633ms 2.4854ms 402.3418 Ops/s 408.8949 Ops/s $\color{#d91a1a}-1.60\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.6109ms 2.6698ms 374.5619 Ops/s 377.7535 Ops/s $\color{#d91a1a}-0.84\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.5706ms 2.6641ms 375.3652 Ops/s 375.4916 Ops/s $\color{#d91a1a}-0.03\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1349ms 2.4840ms 402.5730 Ops/s 402.5860 Ops/s $-0.00\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.5169ms 2.6641ms 375.3573 Ops/s 375.9821 Ops/s $\color{#d91a1a}-0.17\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.3115ms 2.6791ms 373.2540 Ops/s 373.7724 Ops/s $\color{#d91a1a}-0.14\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.8262ms 2.4785ms 403.4777 Ops/s 404.3620 Ops/s $\color{#d91a1a}-0.22\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.9524ms 2.6678ms 374.8353 Ops/s 375.2318 Ops/s $\color{#d91a1a}-0.11\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.7792ms 2.6638ms 375.4016 Ops/s 375.0912 Ops/s $\color{#35bf28}+0.08\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.6959ms 2.4788ms 403.4214 Ops/s 401.5348 Ops/s $\color{#35bf28}+0.47\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.9597ms 2.6630ms 375.5232 Ops/s 374.5941 Ops/s $\color{#35bf28}+0.25\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.8883ms 2.6679ms 374.8308 Ops/s 373.3625 Ops/s $\color{#35bf28}+0.39\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2068s 18.9506ms 52.7687 Ops/s 52.1871 Ops/s $\color{#35bf28}+1.11\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1251s 17.3218ms 57.7308 Ops/s 57.1377 Ops/s $\color{#35bf28}+1.04\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1252s 17.3680ms 57.5773 Ops/s 65.9906 Ops/s $\textbf{\color{#d91a1a}-12.75\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1273s 15.0907ms 66.2658 Ops/s 57.5076 Ops/s $\textbf{\color{#35bf28}+15.23\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1252s 17.2802ms 57.8696 Ops/s 57.5899 Ops/s $\color{#35bf28}+0.49\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1253s 17.3520ms 57.6302 Ops/s 57.3017 Ops/s $\color{#35bf28}+0.57\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1269s 17.4233ms 57.3943 Ops/s 57.2151 Ops/s $\color{#35bf28}+0.31\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1269s 17.3520ms 57.6302 Ops/s 57.1808 Ops/s $\color{#35bf28}+0.79\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1248s 15.0516ms 66.4380 Ops/s 65.7266 Ops/s $\color{#35bf28}+1.08\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot