Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Minor] Add env.shape attribute #1938

Merged
merged 1 commit into from
Feb 20, 2024
Merged

[Minor] Add env.shape attribute #1938

merged 1 commit into from
Feb 20, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 20, 2024 •
edited
Loading

Closes #1925

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 20, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1938

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (14 Unrelated Failures)

As of commit 464a7e0 with merge base c45ee1f (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 20, 2024
@vmoens vmoens added bug Something isn't working documentation Improvements or additions to documentation Suitable for minor Suitable to be integrated in minor release (no new feature) labels Feb 20, 2024
@vmoens vmoens merged commit 799f939 into main Feb 20, 2024
39 of 53 checks passed
@vmoens vmoens deleted the env-shape branch February 20, 2024 17:54
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 72.5774ms 60.8913ms 16.4227 Ops/s 15.4451 Ops/s $\textbf{\color{#35bf28}+6.33\%}$
test_sync 39.8617ms 33.8062ms 29.5804 Ops/s 29.9145 Ops/s $\color{#d91a1a}-1.12\%$
test_async 58.9144ms 30.6278ms 32.6501 Ops/s 32.7393 Ops/s $\color{#d91a1a}-0.27\%$
test_simple 0.4848s 0.4347s 2.3005 Ops/s 2.3494 Ops/s $\color{#d91a1a}-2.08\%$
test_transformed 0.6363s 0.5859s 1.7068 Ops/s 1.7153 Ops/s $\color{#d91a1a}-0.49\%$
test_serial 1.4629s 1.4152s 0.7066 Ops/s 0.7138 Ops/s $\color{#d91a1a}-1.01\%$
test_parallel 1.4446s 1.3911s 0.7188 Ops/s 0.6995 Ops/s $\color{#35bf28}+2.76\%$
test_step_mdp_speed[True-True-True-True-True] 0.1783ms 21.0133μs 47.5890 KOps/s 47.6301 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[True-True-True-True-False] 34.3340μs 12.7324μs 78.5397 KOps/s 77.6960 KOps/s $\color{#35bf28}+1.09\%$
test_step_mdp_speed[True-True-True-False-True] 50.9450μs 12.5618μs 79.6062 KOps/s 80.1074 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-True-True-False-False] 33.6830μs 7.5545μs 132.3706 KOps/s 132.1065 KOps/s $\color{#35bf28}+0.20\%$
test_step_mdp_speed[True-True-False-True-True] 66.6730μs 22.2448μs 44.9542 KOps/s 44.0144 KOps/s $\color{#35bf28}+2.14\%$
test_step_mdp_speed[True-True-False-True-False] 33.8530μs 13.9657μs 71.6039 KOps/s 69.5110 KOps/s $\color{#35bf28}+3.01\%$
test_step_mdp_speed[True-True-False-False-True] 32.3700μs 13.5806μs 73.6345 KOps/s 72.8630 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-True-False-False-False] 33.0620μs 8.7663μs 114.0734 KOps/s 112.4471 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[True-False-True-True-True] 52.6080μs 23.7031μs 42.1885 KOps/s 41.9209 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[True-False-True-True-False] 37.7110μs 15.2761μs 65.4618 KOps/s 64.1433 KOps/s $\color{#35bf28}+2.06\%$
test_step_mdp_speed[True-False-True-False-True] 34.0340μs 13.6387μs 73.3207 KOps/s 72.8029 KOps/s $\color{#35bf28}+0.71\%$
test_step_mdp_speed[True-False-True-False-False] 31.4190μs 8.6695μs 115.3473 KOps/s 112.9864 KOps/s $\color{#35bf28}+2.09\%$
test_step_mdp_speed[True-False-False-True-True] 52.0670μs 24.7265μs 40.4425 KOps/s 39.7692 KOps/s $\color{#35bf28}+1.69\%$
test_step_mdp_speed[True-False-False-True-False] 36.5280μs 16.5597μs 60.3874 KOps/s 58.8185 KOps/s $\color{#35bf28}+2.67\%$
test_step_mdp_speed[True-False-False-False-True] 42.5290μs 14.6988μs 68.0327 KOps/s 67.5345 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-False-False-False-False] 25.6980μs 9.9571μs 100.4313 KOps/s 99.5444 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-True-True-True-True] 50.4640μs 24.0527μs 41.5753 KOps/s 41.7339 KOps/s $\color{#d91a1a}-0.38\%$
test_step_mdp_speed[False-True-True-True-False] 35.1050μs 15.4631μs 64.6701 KOps/s 63.7104 KOps/s $\color{#35bf28}+1.51\%$
test_step_mdp_speed[False-True-True-False-True] 45.0540μs 15.8113μs 63.2459 KOps/s 63.1106 KOps/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[False-True-True-False-False] 27.2210μs 9.8348μs 101.6803 KOps/s 99.5800 KOps/s $\color{#35bf28}+2.11\%$
test_step_mdp_speed[False-True-False-True-True] 45.8260μs 25.3182μs 39.4972 KOps/s 39.1269 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[False-True-False-True-False] 39.7040μs 16.4342μs 60.8488 KOps/s 58.7051 KOps/s $\color{#35bf28}+3.65\%$
test_step_mdp_speed[False-True-False-False-True] 37.4900μs 17.0165μs 58.7665 KOps/s 58.6042 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[False-True-False-False-False] 50.9850μs 11.0811μs 90.2437 KOps/s 88.0984 KOps/s $\color{#35bf28}+2.44\%$
test_step_mdp_speed[False-False-True-True-True] 52.4580μs 26.2450μs 38.1025 KOps/s 37.9943 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[False-False-True-True-False] 54.1510μs 17.8128μs 56.1394 KOps/s 55.0161 KOps/s $\color{#35bf28}+2.04\%$
test_step_mdp_speed[False-False-True-False-True] 61.4550μs 17.2260μs 58.0517 KOps/s 58.6255 KOps/s $\color{#d91a1a}-0.98\%$
test_step_mdp_speed[False-False-True-False-False] 43.0510μs 11.0761μs 90.2842 KOps/s 89.5402 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[False-False-False-True-True] 54.3320μs 27.1686μs 36.8072 KOps/s 36.0800 KOps/s $\color{#35bf28}+2.02\%$
test_step_mdp_speed[False-False-False-True-False] 47.6690μs 18.8195μs 53.1364 KOps/s 52.0832 KOps/s $\color{#35bf28}+2.02\%$
test_step_mdp_speed[False-False-False-False-True] 41.9080μs 17.8727μs 55.9512 KOps/s 55.0181 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[False-False-False-False-False] 31.1790μs 12.1259μs 82.4680 KOps/s 80.9608 KOps/s $\color{#35bf28}+1.86\%$
test_values[generalized_advantage_estimate-True-True] 9.8013ms 9.2095ms 108.5839 Ops/s 107.7428 Ops/s $\color{#35bf28}+0.78\%$
test_values[vec_generalized_advantage_estimate-True-True] 49.4651ms 33.8619ms 29.5318 Ops/s 28.4787 Ops/s $\color{#35bf28}+3.70\%$
test_values[td0_return_estimate-False-False] 0.2219ms 0.1653ms 6.0500 KOps/s 5.8607 KOps/s $\color{#35bf28}+3.23\%$
test_values[td1_return_estimate-False-False] 24.3528ms 22.7963ms 43.8668 Ops/s 43.6733 Ops/s $\color{#35bf28}+0.44\%$
test_values[vec_td1_return_estimate-False-False] 34.2909ms 33.3655ms 29.9711 Ops/s 28.1778 Ops/s $\textbf{\color{#35bf28}+6.36\%}$
test_values[td_lambda_return_estimate-True-False] 35.4317ms 32.8433ms 30.4476 Ops/s 29.9759 Ops/s $\color{#35bf28}+1.57\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.1729ms 33.3612ms 29.9750 Ops/s 28.0290 Ops/s $\textbf{\color{#35bf28}+6.94\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 10.1664ms 8.0970ms 123.5026 Ops/s 120.6734 Ops/s $\color{#35bf28}+2.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.1619ms 1.7961ms 556.7581 Ops/s 498.6975 Ops/s $\textbf{\color{#35bf28}+11.64\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6400ms 0.3485ms 2.8697 KOps/s 2.7804 KOps/s $\color{#35bf28}+3.21\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.2172ms 45.7262ms 21.8693 Ops/s 21.6176 Ops/s $\color{#35bf28}+1.16\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.5886ms 3.0477ms 328.1199 Ops/s 313.1848 Ops/s $\color{#35bf28}+4.77\%$
test_dqn_speed 64.4271ms 1.4876ms 672.2017 Ops/s 721.6316 Ops/s $\textbf{\color{#d91a1a}-6.85\%}$
test_ddpg_speed 3.6069ms 2.8140ms 355.3710 Ops/s 357.5759 Ops/s $\color{#d91a1a}-0.62\%$
test_sac_speed 8.9599ms 8.3854ms 119.2552 Ops/s 119.9374 Ops/s $\color{#d91a1a}-0.57\%$
test_redq_speed 14.8633ms 13.3663ms 74.8151 Ops/s 75.9006 Ops/s $\color{#d91a1a}-1.43\%$
test_redq_deprec_speed 14.3691ms 13.3049ms 75.1603 Ops/s 75.9471 Ops/s $\color{#d91a1a}-1.04\%$
test_td3_speed 8.5843ms 8.3401ms 119.9024 Ops/s 120.1575 Ops/s $\color{#d91a1a}-0.21\%$
test_cql_speed 38.2104ms 36.5500ms 27.3598 Ops/s 27.6063 Ops/s $\color{#d91a1a}-0.89\%$
test_a2c_speed 7.9502ms 7.4098ms 134.9565 Ops/s 136.1050 Ops/s $\color{#d91a1a}-0.84\%$
test_ppo_speed 8.1044ms 7.6548ms 130.6368 Ops/s 130.6301 Ops/s $+0.01\%$
test_reinforce_speed 7.6279ms 6.6431ms 150.5331 Ops/s 152.5182 Ops/s $\color{#d91a1a}-1.30\%$
test_iql_speed 33.9642ms 33.0175ms 30.2870 Ops/s 30.7318 Ops/s $\color{#d91a1a}-1.45\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7532ms 2.7881ms 358.6677 Ops/s 345.8958 Ops/s $\color{#35bf28}+3.69\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7325ms 0.5103ms 1.9596 KOps/s 1.9703 KOps/s $\color{#d91a1a}-0.55\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7075ms 0.4859ms 2.0580 KOps/s 2.0701 KOps/s $\color{#d91a1a}-0.58\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0760ms 2.8024ms 356.8342 Ops/s 345.1850 Ops/s $\color{#35bf28}+3.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8290ms 0.5050ms 1.9803 KOps/s 1.9843 KOps/s $\color{#d91a1a}-0.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7689ms 0.4848ms 2.0628 KOps/s 2.0952 KOps/s $\color{#d91a1a}-1.54\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.4120ms 2.9076ms 343.9318 Ops/s 338.8919 Ops/s $\color{#35bf28}+1.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0651ms 0.6283ms 1.5916 KOps/s 1.6080 KOps/s $\color{#d91a1a}-1.03\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7251ms 0.6026ms 1.6594 KOps/s 1.6795 KOps/s $\color{#d91a1a}-1.20\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9223ms 2.7647ms 361.6988 Ops/s 350.8606 Ops/s $\color{#35bf28}+3.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8011ms 0.5110ms 1.9569 KOps/s 1.9492 KOps/s $\color{#35bf28}+0.40\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8471ms 0.4866ms 2.0552 KOps/s 2.0684 KOps/s $\color{#d91a1a}-0.64\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.3024ms 2.8281ms 353.5969 Ops/s 343.7176 Ops/s $\color{#35bf28}+2.87\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8629ms 0.5044ms 1.9826 KOps/s 1.9341 KOps/s $\color{#35bf28}+2.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9174ms 0.4926ms 2.0301 KOps/s 2.0722 KOps/s $\color{#d91a1a}-2.03\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.7830ms 2.9203ms 342.4297 Ops/s 325.4892 Ops/s $\textbf{\color{#35bf28}+5.20\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8281ms 0.6259ms 1.5977 KOps/s 1.5842 KOps/s $\color{#35bf28}+0.85\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9655ms 0.6043ms 1.6548 KOps/s 1.6776 KOps/s $\color{#d91a1a}-1.36\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 98.3962ms 7.6761ms 130.2745 Ops/s 109.1718 Ops/s $\textbf{\color{#35bf28}+19.33\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 94.0724ms 14.6829ms 68.1064 Ops/s 76.5421 Ops/s $\textbf{\color{#d91a1a}-11.02\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.1474ms 2.5141ms 397.7553 Ops/s 392.6092 Ops/s $\color{#35bf28}+1.31\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 91.9030ms 7.5305ms 132.7935 Ops/s 136.0884 Ops/s $\color{#d91a1a}-2.42\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.1075ms 13.2523ms 75.4587 Ops/s 67.7627 Ops/s $\textbf{\color{#35bf28}+11.36\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.0493ms 2.5333ms 394.7496 Ops/s 387.7137 Ops/s $\color{#35bf28}+1.81\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 96.9099ms 9.7525ms 102.5376 Ops/s 130.1804 Ops/s $\textbf{\color{#d91a1a}-21.23\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.6843ms 13.3580ms 74.8618 Ops/s 74.4198 Ops/s $\color{#35bf28}+0.59\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 3.8566ms 2.7233ms 367.2063 Ops/s 356.3873 Ops/s $\color{#35bf28}+3.04\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1156s 0.1144s 8.7427 Ops/s 8.0114 Ops/s $\textbf{\color{#35bf28}+9.13\%}$
test_sync 95.8605ms 95.5396ms 10.4669 Ops/s 10.4207 Ops/s $\color{#35bf28}+0.44\%$
test_async 0.1813s 91.7031ms 10.9048 Ops/s 10.9100 Ops/s $\color{#d91a1a}-0.05\%$
test_single_pixels 0.2078s 0.1400s 7.1452 Ops/s 7.6585 Ops/s $\textbf{\color{#d91a1a}-6.70\%}$
test_sync_pixels 83.5772ms 82.2205ms 12.1624 Ops/s 12.1759 Ops/s $\color{#d91a1a}-0.11\%$
test_async_pixels 0.1561s 67.9433ms 14.7182 Ops/s 13.3163 Ops/s $\textbf{\color{#35bf28}+10.53\%}$
test_simple 0.8244s 0.8226s 1.2156 Ops/s 1.1715 Ops/s $\color{#35bf28}+3.77\%$
test_transformed 1.0608s 1.0571s 0.9460 Ops/s 0.9204 Ops/s $\color{#35bf28}+2.78\%$
test_serial 2.5460s 2.4850s 0.4024 Ops/s 0.3976 Ops/s $\color{#35bf28}+1.20\%$
test_parallel 2.1797s 2.1163s 0.4725 Ops/s 0.4688 Ops/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-True-True-True-True] 0.2141ms 33.2549μs 30.0708 KOps/s 30.1854 KOps/s $\color{#d91a1a}-0.38\%$
test_step_mdp_speed[True-True-True-True-False] 99.1010μs 19.6010μs 51.0178 KOps/s 50.6927 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[True-True-True-False-True] 71.8910μs 18.7585μs 53.3092 KOps/s 53.8189 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[True-True-True-False-False] 0.1756ms 11.2445μs 88.9325 KOps/s 88.7789 KOps/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[True-True-False-True-True] 65.0110μs 34.9185μs 28.6381 KOps/s 28.8576 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[True-True-False-True-False] 0.1262ms 21.7506μs 45.9757 KOps/s 46.4443 KOps/s $\color{#d91a1a}-1.01\%$
test_step_mdp_speed[True-True-False-False-True] 92.7210μs 20.6022μs 48.5384 KOps/s 49.0282 KOps/s $\color{#d91a1a}-1.00\%$
test_step_mdp_speed[True-True-False-False-False] 35.0000μs 13.1942μs 75.7909 KOps/s 76.1567 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-False-True-True-True] 64.9110μs 36.8932μs 27.1053 KOps/s 27.0906 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[True-False-True-True-False] 46.4710μs 23.6814μs 42.2272 KOps/s 42.6380 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[True-False-True-False-True] 0.1013ms 20.4643μs 48.8656 KOps/s 48.7701 KOps/s $\color{#35bf28}+0.20\%$
test_step_mdp_speed[True-False-True-False-False] 57.0310μs 13.3255μs 75.0440 KOps/s 76.6138 KOps/s $\color{#d91a1a}-2.05\%$
test_step_mdp_speed[True-False-False-True-True] 0.1294ms 38.4162μs 26.0307 KOps/s 26.0023 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-False-False-True-False] 53.3010μs 25.6769μs 38.9455 KOps/s 39.3966 KOps/s $\color{#d91a1a}-1.15\%$
test_step_mdp_speed[True-False-False-False-True] 48.6510μs 22.3047μs 44.8336 KOps/s 45.1283 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-False-False-False-False] 74.1900μs 15.1561μs 65.9802 KOps/s 67.4825 KOps/s $\color{#d91a1a}-2.23\%$
test_step_mdp_speed[False-True-True-True-True] 67.9110μs 36.8366μs 27.1469 KOps/s 26.8140 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[False-True-True-True-False] 43.9400μs 23.5783μs 42.4119 KOps/s 42.5546 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[False-True-True-False-True] 38.8810μs 24.3837μs 41.0109 KOps/s 40.9709 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-True-True-False-False] 32.4010μs 15.0643μs 66.3819 KOps/s 67.5153 KOps/s $\color{#d91a1a}-1.68\%$
test_step_mdp_speed[False-True-False-True-True] 65.4810μs 38.9536μs 25.6716 KOps/s 25.7294 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-True-False-True-False] 0.1082ms 25.6666μs 38.9611 KOps/s 39.4351 KOps/s $\color{#d91a1a}-1.20\%$
test_step_mdp_speed[False-True-False-False-True] 49.8710μs 26.1441μs 38.2495 KOps/s 38.2101 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-True-False-False-False] 33.3710μs 16.8394μs 59.3844 KOps/s 60.3679 KOps/s $\color{#d91a1a}-1.63\%$
test_step_mdp_speed[False-False-True-True-True] 59.3310μs 40.5447μs 24.6642 KOps/s 24.7451 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-False-True-True-False] 41.5110μs 27.4735μs 36.3987 KOps/s 36.7356 KOps/s $\color{#d91a1a}-0.92\%$
test_step_mdp_speed[False-False-True-False-True] 46.2510μs 26.1605μs 38.2255 KOps/s 38.4399 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[False-False-True-False-False] 32.7800μs 16.9073μs 59.1460 KOps/s 60.5869 KOps/s $\color{#d91a1a}-2.38\%$
test_step_mdp_speed[False-False-False-True-True] 70.2710μs 42.5655μs 23.4932 KOps/s 23.8702 KOps/s $\color{#d91a1a}-1.58\%$
test_step_mdp_speed[False-False-False-True-False] 63.3510μs 29.2184μs 34.2250 KOps/s 34.4744 KOps/s $\color{#d91a1a}-0.72\%$
test_step_mdp_speed[False-False-False-False-True] 44.8310μs 27.6974μs 36.1044 KOps/s 36.0838 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[False-False-False-False-False] 79.4810μs 18.6299μs 53.6772 KOps/s 54.6524 KOps/s $\color{#d91a1a}-1.78\%$
test_values[generalized_advantage_estimate-True-True] 26.2211ms 25.3907ms 39.3845 Ops/s 37.0702 Ops/s $\textbf{\color{#35bf28}+6.24\%}$
test_values[vec_generalized_advantage_estimate-True-True] 91.8249ms 3.4033ms 293.8299 Ops/s 102.4160 Ops/s $\textbf{\color{#35bf28}+186.90\%}$
test_values[td0_return_estimate-False-False] 98.5220μs 62.5631μs 15.9839 KOps/s 16.1101 KOps/s $\color{#d91a1a}-0.78\%$
test_values[td1_return_estimate-False-False] 55.7417ms 54.5058ms 18.3467 Ops/s 18.0743 Ops/s $\color{#35bf28}+1.51\%$
test_values[vec_td1_return_estimate-False-False] 2.1092ms 1.7683ms 565.5069 Ops/s 564.6930 Ops/s $\color{#35bf28}+0.14\%$
test_values[td_lambda_return_estimate-True-False] 89.6868ms 87.1154ms 11.4790 Ops/s 11.3002 Ops/s $\color{#35bf28}+1.58\%$
test_values[vec_td_lambda_return_estimate-True-False] 4.0783ms 1.8022ms 554.8712 Ops/s 530.5721 Ops/s $\color{#35bf28}+4.58\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.6357ms 23.5357ms 42.4886 Ops/s 40.5956 Ops/s $\color{#35bf28}+4.66\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8938ms 0.7016ms 1.4253 KOps/s 1.3658 KOps/s $\color{#35bf28}+4.36\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8011ms 0.6503ms 1.5378 KOps/s 1.5009 KOps/s $\color{#35bf28}+2.46\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6120ms 1.4557ms 686.9463 Ops/s 681.0400 Ops/s $\color{#35bf28}+0.87\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9700ms 0.6721ms 1.4879 KOps/s 1.4689 KOps/s $\color{#35bf28}+1.29\%$
test_dqn_speed 4.1455ms 1.4775ms 676.7975 Ops/s 660.0346 Ops/s $\color{#35bf28}+2.54\%$
test_ddpg_speed 3.0556ms 2.8802ms 347.1973 Ops/s 343.0657 Ops/s $\color{#35bf28}+1.20\%$
test_sac_speed 8.9474ms 8.3740ms 119.4168 Ops/s 119.2705 Ops/s $\color{#35bf28}+0.12\%$
test_redq_speed 12.4213ms 10.6888ms 93.5561 Ops/s 91.8118 Ops/s $\color{#35bf28}+1.90\%$
test_redq_deprec_speed 12.1650ms 11.6389ms 85.9190 Ops/s 84.7727 Ops/s $\color{#35bf28}+1.35\%$
test_td3_speed 8.5237ms 8.3452ms 119.8292 Ops/s 117.1587 Ops/s $\color{#35bf28}+2.28\%$
test_cql_speed 27.3248ms 26.2484ms 38.0975 Ops/s 37.3405 Ops/s $\color{#35bf28}+2.03\%$
test_a2c_speed 5.9676ms 5.7438ms 174.0995 Ops/s 179.3030 Ops/s $\color{#d91a1a}-2.90\%$
test_ppo_speed 6.3082ms 6.0227ms 166.0385 Ops/s 166.8977 Ops/s $\color{#d91a1a}-0.51\%$
test_reinforce_speed 5.2858ms 4.6595ms 214.6151 Ops/s 215.4919 Ops/s $\color{#d91a1a}-0.41\%$
test_iql_speed 20.8993ms 20.3515ms 49.1365 Ops/s 50.0943 Ops/s $\color{#d91a1a}-1.91\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.9259ms 3.7294ms 268.1400 Ops/s 265.4043 Ops/s $\color{#35bf28}+1.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7506ms 0.5767ms 1.7340 KOps/s 1.7492 KOps/s $\color{#d91a1a}-0.87\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7350ms 0.5489ms 1.8218 KOps/s 1.8195 KOps/s $\color{#35bf28}+0.13\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.0404ms 3.7503ms 266.6438 Ops/s 262.7654 Ops/s $\color{#35bf28}+1.48\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7413ms 0.5649ms 1.7701 KOps/s 1.7743 KOps/s $\color{#d91a1a}-0.24\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6814ms 0.5403ms 1.8507 KOps/s 1.8494 KOps/s $\color{#35bf28}+0.07\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.0020ms 3.8401ms 260.4108 Ops/s 257.2665 Ops/s $\color{#35bf28}+1.22\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8567ms 0.7003ms 1.4280 KOps/s 1.2664 KOps/s $\textbf{\color{#35bf28}+12.76\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8535ms 0.6722ms 1.4877 KOps/s 1.4934 KOps/s $\color{#d91a1a}-0.38\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.9058ms 3.7277ms 268.2627 Ops/s 264.6192 Ops/s $\color{#35bf28}+1.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7435ms 0.5731ms 1.7450 KOps/s 1.7483 KOps/s $\color{#d91a1a}-0.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7136ms 0.5486ms 1.8228 KOps/s 1.8345 KOps/s $\color{#d91a1a}-0.64\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.0378ms 3.7644ms 265.6460 Ops/s 263.1276 Ops/s $\color{#35bf28}+0.96\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8340ms 0.5700ms 1.7545 KOps/s 1.7603 KOps/s $\color{#d91a1a}-0.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6980ms 0.5402ms 1.8513 KOps/s 1.5489 KOps/s $\textbf{\color{#35bf28}+19.52\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.0335ms 3.8509ms 259.6809 Ops/s 255.6728 Ops/s $\color{#35bf28}+1.57\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9397ms 0.7040ms 1.4204 KOps/s 1.4234 KOps/s $\color{#d91a1a}-0.21\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8560ms 0.6738ms 1.4842 KOps/s 1.4790 KOps/s $\color{#35bf28}+0.35\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1374s 10.3144ms 96.9517 Ops/s 85.3858 Ops/s $\textbf{\color{#35bf28}+13.55\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.1111ms 16.3934ms 61.0002 Ops/s 60.5704 Ops/s $\color{#35bf28}+0.71\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.4196ms 3.2065ms 311.8643 Ops/s 310.5883 Ops/s $\color{#35bf28}+0.41\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1097s 11.5898ms 86.2831 Ops/s 86.3065 Ops/s $\color{#d91a1a}-0.03\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 19.3367ms 16.2126ms 61.6803 Ops/s 60.6359 Ops/s $\color{#35bf28}+1.72\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 8.5948ms 3.2081ms 311.7142 Ops/s 306.2161 Ops/s $\color{#35bf28}+1.80\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1143s 12.1476ms 82.3206 Ops/s 101.1480 Ops/s $\textbf{\color{#d91a1a}-18.61\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 19.1227ms 16.5886ms 60.2825 Ops/s 52.8218 Ops/s $\textbf{\color{#35bf28}+14.12\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 8.7636ms 3.5346ms 282.9214 Ops/s 282.9522 Ops/s $\color{#d91a1a}-0.01\%$

vmoens added a commit that referenced this pull request Feb 28, 2024
@vmoens
[Minor] Add env.shape attribute (#1938)
4723747
@vmoens vmoens mentioned this pull request Feb 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. documentation Improvements or additions to documentation Suitable for minor Suitable to be integrated in minor release (no new feature)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] Expected a tensordict with shape==env.shape, got torch.Size([1]) and torch.Size([])
2 participants
@vmoens @facebook-github-bot