Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix load_state_dict and is_empty td bugfix impact #1869

Merged
merged 2 commits into from
Feb 4, 2024
Merged

[BugFix] Fix load_state_dict and is_empty td bugfix impact #1869

merged 2 commits into from
Feb 4, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 4, 2024 •
edited
Loading

Bug fixes in tensordict (namely is_empty and state_dict issues) caused tests to fail in torchrl.
Since these tests were relying on brittle APIs we simply upgrade to the consolidated tensordict features.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 4, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1869

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (3 Unrelated Failures)

As of commit af9a036 with merge base c2f43e8 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 4, 2024
@vmoens vmoens added the bug Something isn't working label Feb 4, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 4, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1332s 67.8209ms 14.7447 Ops/s 16.0526 Ops/s $\textbf{\color{#d91a1a}-8.15\%}$
test_sync 54.5075ms 35.0597ms 28.5228 Ops/s 29.2339 Ops/s $\color{#d91a1a}-2.43\%$
test_async 0.1304s 33.0978ms 30.2135 Ops/s 30.0242 Ops/s $\color{#35bf28}+0.63\%$
test_simple 0.4921s 0.4317s 2.3162 Ops/s 2.2708 Ops/s $\color{#35bf28}+2.00\%$
test_transformed 0.6495s 0.5952s 1.6802 Ops/s 1.6436 Ops/s $\color{#35bf28}+2.23\%$
test_serial 1.4186s 1.3563s 0.7373 Ops/s 0.7190 Ops/s $\color{#35bf28}+2.55\%$
test_parallel 1.2998s 1.2271s 0.8150 Ops/s 0.8133 Ops/s $\color{#35bf28}+0.20\%$
test_step_mdp_speed[True-True-True-True-True] 46.7970μs 21.0861μs 47.4247 KOps/s 47.8559 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[True-True-True-True-False] 55.8640μs 12.9881μs 76.9938 KOps/s 79.3503 KOps/s $\color{#d91a1a}-2.97\%$
test_step_mdp_speed[True-True-True-False-True] 33.5020μs 12.4046μs 80.6155 KOps/s 81.2824 KOps/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[True-True-True-False-False] 49.0110μs 7.4895μs 133.5209 KOps/s 136.5145 KOps/s $\color{#d91a1a}-2.19\%$
test_step_mdp_speed[True-True-False-True-True] 85.5410μs 22.1924μs 45.0605 KOps/s 44.0606 KOps/s $\color{#35bf28}+2.27\%$
test_step_mdp_speed[True-True-False-True-False] 45.5850μs 14.2171μs 70.3376 KOps/s 71.6861 KOps/s $\color{#d91a1a}-1.88\%$
test_step_mdp_speed[True-True-False-False-True] 41.0760μs 13.5544μs 73.7769 KOps/s 72.7503 KOps/s $\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-True-False-False-False] 30.9280μs 8.7535μs 114.2406 KOps/s 115.0943 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[True-False-True-True-True] 44.5330μs 23.9516μs 41.7508 KOps/s 41.5944 KOps/s $\color{#35bf28}+0.38\%$
test_step_mdp_speed[True-False-True-True-False] 67.8970μs 15.6627μs 63.8461 KOps/s 65.1047 KOps/s $\color{#d91a1a}-1.93\%$
test_step_mdp_speed[True-False-True-False-True] 41.4770μs 13.6075μs 73.4889 KOps/s 73.6610 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[True-False-True-False-False] 26.8000μs 8.7597μs 114.1589 KOps/s 115.0691 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[True-False-False-True-True] 69.9310μs 25.0887μs 39.8585 KOps/s 39.0678 KOps/s $\color{#35bf28}+2.02\%$
test_step_mdp_speed[True-False-False-True-False] 44.9030μs 16.6466μs 60.0722 KOps/s 60.8701 KOps/s $\color{#d91a1a}-1.31\%$
test_step_mdp_speed[True-False-False-False-True] 37.2400μs 14.8858μs 67.1782 KOps/s 67.1691 KOps/s $\color{#35bf28}+0.01\%$
test_step_mdp_speed[True-False-False-False-False] 32.1900μs 9.9282μs 100.7236 KOps/s 101.2392 KOps/s $\color{#d91a1a}-0.51\%$
test_step_mdp_speed[False-True-True-True-True] 54.2210μs 24.0583μs 41.5657 KOps/s 41.7895 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[False-True-True-True-False] 41.1460μs 15.6403μs 63.9376 KOps/s 65.6781 KOps/s $\color{#d91a1a}-2.65\%$
test_step_mdp_speed[False-True-True-False-True] 44.4030μs 15.8631μs 63.0396 KOps/s 63.6492 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[False-True-True-False-False] 29.8050μs 9.9216μs 100.7905 KOps/s 102.2047 KOps/s $\color{#d91a1a}-1.38\%$
test_step_mdp_speed[False-True-False-True-True] 50.2930μs 24.8947μs 40.1691 KOps/s 39.9488 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[False-True-False-True-False] 37.3800μs 16.7013μs 59.8757 KOps/s 60.6868 KOps/s $\color{#d91a1a}-1.34\%$
test_step_mdp_speed[False-True-False-False-True] 43.0000μs 16.9280μs 59.0738 KOps/s 58.7248 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[False-True-False-False-False] 80.3490μs 11.0024μs 90.8895 KOps/s 91.8047 KOps/s $\color{#d91a1a}-1.00\%$
test_step_mdp_speed[False-False-True-True-True] 55.4130μs 25.8666μs 38.6599 KOps/s 37.5996 KOps/s $\color{#35bf28}+2.82\%$
test_step_mdp_speed[False-False-True-True-False] 38.6220μs 17.9522μs 55.7035 KOps/s 56.6559 KOps/s $\color{#d91a1a}-1.68\%$
test_step_mdp_speed[False-False-True-False-True] 46.7970μs 16.8651μs 59.2940 KOps/s 58.8233 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-False-True-False-False] 55.9440μs 11.3003μs 88.4930 KOps/s 90.7012 KOps/s $\color{#d91a1a}-2.43\%$
test_step_mdp_speed[False-False-False-True-True] 53.9300μs 27.0424μs 36.9790 KOps/s 36.1047 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[False-False-False-True-False] 42.6290μs 19.2830μs 51.8591 KOps/s 52.7730 KOps/s $\color{#d91a1a}-1.73\%$
test_step_mdp_speed[False-False-False-False-True] 66.9330μs 17.8367μs 56.0641 KOps/s 55.1904 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[False-False-False-False-False] 50.0730μs 12.2964μs 81.3248 KOps/s 82.1278 KOps/s $\color{#d91a1a}-0.98\%$
test_values[generalized_advantage_estimate-True-True] 9.7003ms 9.2411ms 108.2120 Ops/s 107.6183 Ops/s $\color{#35bf28}+0.55\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.1092ms 34.9390ms 28.6213 Ops/s 28.4169 Ops/s $\color{#35bf28}+0.72\%$
test_values[td0_return_estimate-False-False] 0.2240ms 0.1707ms 5.8573 KOps/s 5.6602 KOps/s $\color{#35bf28}+3.48\%$
test_values[td1_return_estimate-False-False] 24.8848ms 23.3817ms 42.7684 Ops/s 43.3645 Ops/s $\color{#d91a1a}-1.37\%$
test_values[vec_td1_return_estimate-False-False] 37.3198ms 35.0768ms 28.5089 Ops/s 28.2622 Ops/s $\color{#35bf28}+0.87\%$
test_values[td_lambda_return_estimate-True-False] 36.7488ms 34.1831ms 29.2543 Ops/s 29.7800 Ops/s $\color{#d91a1a}-1.77\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.3950ms 35.1312ms 28.4647 Ops/s 28.2940 Ops/s $\color{#35bf28}+0.60\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.4604ms 7.9908ms 125.1438 Ops/s 122.9902 Ops/s $\color{#35bf28}+1.75\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.5994ms 1.9236ms 519.8508 Ops/s 465.2752 Ops/s $\textbf{\color{#35bf28}+11.73\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5389ms 0.3445ms 2.9030 KOps/s 2.8282 KOps/s $\color{#35bf28}+2.65\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 45.3620ms 42.2800ms 23.6518 Ops/s 22.5558 Ops/s $\color{#35bf28}+4.86\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.6936ms 3.0549ms 327.3416 Ops/s 326.2168 Ops/s $\color{#35bf28}+0.34\%$
test_dqn_speed 86.8663ms 6.2821ms 159.1829 Ops/s 167.1807 Ops/s $\color{#d91a1a}-4.78\%$
test_ddpg_speed 12.3459ms 11.3448ms 88.1460 Ops/s 79.5771 Ops/s $\textbf{\color{#35bf28}+10.77\%}$
test_sac_speed 25.6809ms 23.2961ms 42.9257 Ops/s 42.2163 Ops/s $\color{#35bf28}+1.68\%$
test_redq_speed 15.1819ms 14.1834ms 70.5049 Ops/s 69.1598 Ops/s $\color{#35bf28}+1.94\%$
test_redq_deprec_speed 23.7664ms 22.0204ms 45.4123 Ops/s 44.7306 Ops/s $\color{#35bf28}+1.52\%$
test_td3_speed 25.9691ms 16.3796ms 61.0516 Ops/s 59.5626 Ops/s $\color{#35bf28}+2.50\%$
test_cql_speed 75.0100ms 72.8999ms 13.7174 Ops/s 13.4308 Ops/s $\color{#35bf28}+2.13\%$
test_a2c_speed 24.1202ms 21.9149ms 45.6311 Ops/s 44.8159 Ops/s $\color{#35bf28}+1.82\%$
test_ppo_speed 24.3273ms 22.0390ms 45.3741 Ops/s 44.3821 Ops/s $\color{#35bf28}+2.24\%$
test_reinforce_speed 23.9479ms 21.0668ms 47.4681 Ops/s 46.9469 Ops/s $\color{#35bf28}+1.11\%$
test_iql_speed 56.9494ms 55.7295ms 17.9438 Ops/s 17.7341 Ops/s $\color{#35bf28}+1.18\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.1518ms 2.8378ms 352.3835 Ops/s 354.5691 Ops/s $\color{#d91a1a}-0.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9431ms 0.5072ms 1.9716 KOps/s 1.9675 KOps/s $\color{#35bf28}+0.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9195ms 0.4793ms 2.0863 KOps/s 2.0801 KOps/s $\color{#35bf28}+0.30\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.7608ms 2.8469ms 351.2622 Ops/s 345.2868 Ops/s $\color{#35bf28}+1.73\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7680ms 0.5023ms 1.9909 KOps/s 1.9828 KOps/s $\color{#35bf28}+0.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6334ms 0.4806ms 2.0808 KOps/s 2.1008 KOps/s $\color{#d91a1a}-0.95\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2406ms 2.6357ms 379.4079 Ops/s 377.5422 Ops/s $\color{#35bf28}+0.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9740ms 0.6242ms 1.6022 KOps/s 1.5985 KOps/s $\color{#35bf28}+0.23\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0395ms 0.5959ms 1.6781 KOps/s 1.6629 KOps/s $\color{#35bf28}+0.91\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.1031ms 2.8993ms 344.9110 Ops/s 347.7627 Ops/s $\color{#d91a1a}-0.82\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8461ms 0.5136ms 1.9472 KOps/s 1.9431 KOps/s $\color{#35bf28}+0.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6096ms 0.4873ms 2.0523 KOps/s 2.0641 KOps/s $\color{#d91a1a}-0.57\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.2380ms 2.8976ms 345.1096 Ops/s 350.5697 Ops/s $\color{#d91a1a}-1.56\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7077ms 0.5061ms 1.9760 KOps/s 1.9694 KOps/s $\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6003ms 0.4778ms 2.0929 KOps/s 2.0919 KOps/s $\color{#35bf28}+0.05\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.9277ms 2.6631ms 375.5079 Ops/s 379.2038 Ops/s $\color{#d91a1a}-0.97\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8892ms 0.6226ms 1.6062 KOps/s 1.5942 KOps/s $\color{#35bf28}+0.75\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0452ms 0.5987ms 1.6703 KOps/s 1.6695 KOps/s $\color{#35bf28}+0.05\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1046s 8.0470ms 124.2704 Ops/s 121.6771 Ops/s $\color{#35bf28}+2.13\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.2453ms 12.9560ms 77.1840 Ops/s 75.9044 Ops/s $\color{#35bf28}+1.69\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.9288ms 2.5141ms 397.7494 Ops/s 399.8755 Ops/s $\color{#d91a1a}-0.53\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1057s 9.9232ms 100.7741 Ops/s 100.3912 Ops/s $\color{#35bf28}+0.38\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.9015ms 13.3437ms 74.9419 Ops/s 75.6621 Ops/s $\color{#d91a1a}-0.95\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.2692ms 2.9142ms 343.1481 Ops/s 395.3388 Ops/s $\textbf{\color{#d91a1a}-13.20\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1132s 10.8969ms 91.7694 Ops/s 121.4623 Ops/s $\textbf{\color{#d91a1a}-24.45\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.9258ms 13.4809ms 74.1788 Ops/s 74.0725 Ops/s $\color{#35bf28}+0.14\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 3.7016ms 2.7609ms 362.1988 Ops/s 367.3018 Ops/s $\color{#d91a1a}-1.39\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 4, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1247s 0.1237s 8.0859 Ops/s 8.2017 Ops/s $\color{#d91a1a}-1.41\%$
test_sync 0.1404s 0.1385s 7.2178 Ops/s 7.1255 Ops/s $\color{#35bf28}+1.29\%$
test_async 0.3372s 0.1344s 7.4394 Ops/s 7.3345 Ops/s $\color{#35bf28}+1.43\%$
test_single_pixels 0.1395s 0.1391s 7.1913 Ops/s 7.4742 Ops/s $\color{#d91a1a}-3.79\%$
test_sync_pixels 0.1027s 0.1018s 9.8232 Ops/s 9.7050 Ops/s $\color{#35bf28}+1.22\%$
test_async_pixels 0.2684s 99.0344ms 10.0975 Ops/s 10.0344 Ops/s $\color{#35bf28}+0.63\%$
test_simple 0.9522s 0.8811s 1.1349 Ops/s 1.1665 Ops/s $\color{#d91a1a}-2.71\%$
test_transformed 1.2011s 1.1321s 0.8834 Ops/s 0.8854 Ops/s $\color{#d91a1a}-0.23\%$
test_serial 2.4480s 2.4455s 0.4089 Ops/s 0.4052 Ops/s $\color{#35bf28}+0.91\%$
test_parallel 3.2916s 3.1915s 0.3133 Ops/s 0.3113 Ops/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[True-True-True-True-True] 0.1211ms 32.2666μs 30.9918 KOps/s 31.4132 KOps/s $\color{#d91a1a}-1.34\%$
test_step_mdp_speed[True-True-True-True-False] 40.7710μs 19.0326μs 52.5413 KOps/s 51.7192 KOps/s $\color{#35bf28}+1.59\%$
test_step_mdp_speed[True-True-True-False-True] 41.1310μs 18.5812μs 53.8179 KOps/s 54.9297 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[True-True-True-False-False] 26.9600μs 11.0131μs 90.8013 KOps/s 90.2351 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-True-False-True-True] 57.7310μs 33.8186μs 29.5695 KOps/s 29.1856 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-True-False-True-False] 45.9410μs 21.0523μs 47.5008 KOps/s 47.1674 KOps/s $\color{#35bf28}+0.71\%$
test_step_mdp_speed[True-True-False-False-True] 40.3110μs 20.2555μs 49.3693 KOps/s 49.5749 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[True-True-False-False-False] 30.5400μs 12.9289μs 77.3458 KOps/s 77.3737 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-False-True-True-True] 59.5000μs 35.8517μs 27.8927 KOps/s 27.6573 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[True-False-True-True-False] 46.0110μs 23.3182μs 42.8849 KOps/s 42.6715 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[True-False-True-False-True] 37.1110μs 20.2910μs 49.2830 KOps/s 50.1450 KOps/s $\color{#d91a1a}-1.72\%$
test_step_mdp_speed[True-False-True-False-False] 37.2710μs 12.7304μs 78.5522 KOps/s 77.9213 KOps/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[True-False-False-True-True] 67.9000μs 37.5380μs 26.6397 KOps/s 26.4609 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[True-False-False-True-False] 48.4510μs 25.1341μs 39.7865 KOps/s 40.1069 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[True-False-False-False-True] 38.1810μs 21.8990μs 45.6641 KOps/s 45.7168 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-False-False-False-False] 35.5600μs 14.5374μs 68.7880 KOps/s 67.6727 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[False-True-True-True-True] 60.2000μs 35.8095μs 27.9255 KOps/s 27.5311 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[False-True-True-True-False] 46.5210μs 23.2083μs 43.0881 KOps/s 43.0622 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[False-True-True-False-True] 42.3400μs 24.0450μs 41.5887 KOps/s 41.9054 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[False-True-True-False-False] 35.4500μs 14.5395μs 68.7783 KOps/s 68.1291 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[False-True-False-True-True] 62.3410μs 38.2661μs 26.1328 KOps/s 26.2391 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[False-True-False-True-False] 42.3710μs 25.1538μs 39.7554 KOps/s 39.8880 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-True-False-False-True] 50.8210μs 25.5678μs 39.1118 KOps/s 38.5196 KOps/s $\color{#35bf28}+1.54\%$
test_step_mdp_speed[False-True-False-False-False] 34.1710μs 16.4206μs 60.8992 KOps/s 60.5820 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[False-False-True-True-True] 62.7820μs 39.7635μs 25.1487 KOps/s 25.0004 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[False-False-True-True-False] 49.0310μs 27.4484μs 36.4320 KOps/s 36.8383 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[False-False-True-False-True] 53.0010μs 25.5536μs 39.1334 KOps/s 38.5938 KOps/s $\color{#35bf28}+1.40\%$
test_step_mdp_speed[False-False-True-False-False] 42.9110μs 16.6049μs 60.2233 KOps/s 60.2959 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[False-False-False-True-True] 68.4010μs 41.4963μs 24.0986 KOps/s 23.9486 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[False-False-False-True-False] 47.2910μs 29.0934μs 34.3720 KOps/s 34.8266 KOps/s $\color{#d91a1a}-1.31\%$
test_step_mdp_speed[False-False-False-False-True] 54.7500μs 27.1398μs 36.8462 KOps/s 36.1599 KOps/s $\color{#35bf28}+1.90\%$
test_step_mdp_speed[False-False-False-False-False] 40.8910μs 18.1872μs 54.9838 KOps/s 54.8277 KOps/s $\color{#35bf28}+0.28\%$
test_values[generalized_advantage_estimate-True-True] 27.4710ms 26.5742ms 37.6305 Ops/s 40.2518 Ops/s $\textbf{\color{#d91a1a}-6.51\%}$
test_values[vec_generalized_advantage_estimate-True-True] 88.5256ms 3.3488ms 298.6160 Ops/s 305.9069 Ops/s $\color{#d91a1a}-2.38\%$
test_values[td0_return_estimate-False-False] 0.1082ms 68.7967μs 14.5356 KOps/s 16.1679 KOps/s $\textbf{\color{#d91a1a}-10.10\%}$
test_values[td1_return_estimate-False-False] 59.5654ms 58.5096ms 17.0912 Ops/s 18.6471 Ops/s $\textbf{\color{#d91a1a}-8.34\%}$
test_values[vec_td1_return_estimate-False-False] 2.1980ms 1.8047ms 554.1129 Ops/s 562.6140 Ops/s $\color{#d91a1a}-1.51\%$
test_values[td_lambda_return_estimate-True-False] 95.2213ms 94.6755ms 10.5624 Ops/s 11.1489 Ops/s $\textbf{\color{#d91a1a}-5.26\%}$
test_values[vec_td_lambda_return_estimate-True-False] 4.2231ms 1.8568ms 538.5597 Ops/s 549.2694 Ops/s $\color{#d91a1a}-1.95\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 26.9556ms 26.6633ms 37.5047 Ops/s 42.0347 Ops/s $\textbf{\color{#d91a1a}-10.78\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9326ms 0.7274ms 1.3748 KOps/s 1.4035 KOps/s $\color{#d91a1a}-2.05\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7381ms 0.6725ms 1.4870 KOps/s 1.5187 KOps/s $\color{#d91a1a}-2.09\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5474ms 1.4746ms 678.1664 Ops/s 685.3640 Ops/s $\color{#d91a1a}-1.05\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9624ms 0.6868ms 1.4560 KOps/s 1.4801 KOps/s $\color{#d91a1a}-1.63\%$
test_dqn_speed 7.6277ms 7.1182ms 140.4859 Ops/s 137.2995 Ops/s $\color{#35bf28}+2.32\%$
test_ddpg_speed 14.8322ms 13.8527ms 72.1880 Ops/s 71.5620 Ops/s $\color{#35bf28}+0.87\%$
test_sac_speed 28.5614ms 27.7984ms 35.9732 Ops/s 35.7734 Ops/s $\color{#35bf28}+0.56\%$
test_redq_speed 12.8781ms 11.7395ms 85.1828 Ops/s 78.4462 Ops/s $\textbf{\color{#35bf28}+8.59\%}$
test_redq_deprec_speed 23.2792ms 22.4080ms 44.6270 Ops/s 44.1699 Ops/s $\color{#35bf28}+1.03\%$
test_td3_speed 28.1580ms 18.8768ms 52.9751 Ops/s 53.1961 Ops/s $\color{#d91a1a}-0.42\%$
test_cql_speed 76.0532ms 75.1329ms 13.3097 Ops/s 13.4329 Ops/s $\color{#d91a1a}-0.92\%$
test_a2c_speed 25.7250ms 24.7545ms 40.3966 Ops/s 37.1271 Ops/s $\textbf{\color{#35bf28}+8.81\%}$
test_ppo_speed 25.8662ms 25.0800ms 39.8724 Ops/s 40.2789 Ops/s $\color{#d91a1a}-1.01\%$
test_reinforce_speed 24.6733ms 23.6624ms 42.2611 Ops/s 42.1877 Ops/s $\color{#35bf28}+0.17\%$
test_iql_speed 51.0739ms 49.8147ms 20.0744 Ops/s 20.0201 Ops/s $\color{#35bf28}+0.27\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7547ms 3.6509ms 273.9076 Ops/s 277.5960 Ops/s $\color{#d91a1a}-1.33\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9968ms 0.5644ms 1.7717 KOps/s 1.7953 KOps/s $\color{#d91a1a}-1.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6802ms 0.5319ms 1.8801 KOps/s 1.8784 KOps/s $\color{#35bf28}+0.09\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8814ms 3.6745ms 272.1439 Ops/s 272.7190 Ops/s $\color{#d91a1a}-0.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7195ms 0.5530ms 1.8082 KOps/s 1.8080 KOps/s $\color{#35bf28}+0.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6565ms 0.5248ms 1.9054 KOps/s 1.8994 KOps/s $\color{#35bf28}+0.31\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.4785ms 3.3467ms 298.8050 Ops/s 299.4091 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8387ms 0.6890ms 1.4514 KOps/s 1.4610 KOps/s $\color{#d91a1a}-0.66\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8262ms 0.6645ms 1.5050 KOps/s 1.5163 KOps/s $\color{#d91a1a}-0.75\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7400ms 3.6458ms 274.2881 Ops/s 273.0303 Ops/s $\color{#35bf28}+0.46\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7259ms 0.5645ms 1.7714 KOps/s 1.7815 KOps/s $\color{#d91a1a}-0.56\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6614ms 0.5338ms 1.8733 KOps/s 1.8705 KOps/s $\color{#35bf28}+0.15\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.0303ms 3.6897ms 271.0259 Ops/s 270.7582 Ops/s $\color{#35bf28}+0.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6808ms 0.5592ms 1.7881 KOps/s 1.8060 KOps/s $\color{#d91a1a}-0.99\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6798ms 0.5316ms 1.8811 KOps/s 1.8933 KOps/s $\color{#d91a1a}-0.65\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.4491ms 3.3317ms 300.1477 Ops/s 300.8303 Ops/s $\color{#d91a1a}-0.23\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8282ms 0.6867ms 1.4563 KOps/s 1.4596 KOps/s $\color{#d91a1a}-0.23\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8225ms 0.6665ms 1.5003 KOps/s 1.5137 KOps/s $\color{#d91a1a}-0.88\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1389s 10.5228ms 95.0318 Ops/s 95.9284 Ops/s $\color{#d91a1a}-0.93\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 20.4468ms 17.3403ms 57.6692 Ops/s 61.5676 Ops/s $\textbf{\color{#d91a1a}-6.33\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.0396ms 3.1736ms 315.0983 Ops/s 315.6785 Ops/s $\color{#d91a1a}-0.18\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1213s 10.1195ms 98.8190 Ops/s 97.7346 Ops/s $\color{#35bf28}+1.11\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 19.5914ms 17.0729ms 58.5724 Ops/s 60.9909 Ops/s $\color{#d91a1a}-3.97\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.6056ms 3.2051ms 312.0063 Ops/s 320.6391 Ops/s $\color{#d91a1a}-2.69\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1231s 12.7565ms 78.3913 Ops/s 96.5943 Ops/s $\textbf{\color{#d91a1a}-18.84\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 20.3915ms 17.2997ms 57.8044 Ops/s 52.8318 Ops/s $\textbf{\color{#35bf28}+9.41\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.9101ms 3.3886ms 295.1073 Ops/s 300.4942 Ops/s $\color{#d91a1a}-1.79\%$

@vmoens vmoens merged commit 0672359 into main Feb 4, 2024
65 of 68 checks passed
@vmoens vmoens deleted the fix-empty-and-sd branch February 4, 2024 08:43
vmoens added a commit that referenced this pull request Feb 6, 2024
@vmoens
[BugFix] Fix load_state_dict and is_empty td bugfix impact (#1869)
ba804ab
@vmoens vmoens mentioned this pull request Feb 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot