Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] A PixelRenderTransform #2099

Merged
merged 11 commits into from
Apr 23, 2024
Merged

[Feature] A PixelRenderTransform #2099

merged 11 commits into from
Apr 23, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Apr 23, 2024 •
edited
Loading

Adds a transform to record rgb_arrays coming from env.render()

cc @matteobettini @vikashplus

@Miffyli: I also added a bit of doc RE recording videos in torchrl, LMK if you see anything missing.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Apr 23, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2099

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 16 Unrelated Failures

As of commit e7986ad with merge base 7dd0128 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 23, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 23, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 53.6816ms 53.2092ms 18.7937 Ops/s 17.7813 Ops/s $\textbf{\color{#35bf28}+5.69\%}$
test_sync 34.0465ms 29.3941ms 34.0204 Ops/s 34.4575 Ops/s $\color{#d91a1a}-1.27\%$
test_async 62.3865ms 29.0366ms 34.4393 Ops/s 36.6387 Ops/s $\textbf{\color{#d91a1a}-6.00\%}$
test_simple 0.3403s 0.3372s 2.9656 Ops/s 2.9660 Ops/s $\color{#d91a1a}-0.01\%$
test_transformed 0.4873s 0.4836s 2.0677 Ops/s 2.0153 Ops/s $\color{#35bf28}+2.60\%$
test_serial 1.2722s 1.2210s 0.8190 Ops/s 0.8335 Ops/s $\color{#d91a1a}-1.73\%$
test_parallel 1.0577s 1.0055s 0.9945 Ops/s 0.9969 Ops/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[True-True-True-True-True] 0.1720ms 21.2280μs 47.1075 KOps/s 46.0317 KOps/s $\color{#35bf28}+2.34\%$
test_step_mdp_speed[True-True-True-True-False] 74.6290μs 12.9540μs 77.1960 KOps/s 76.9615 KOps/s $\color{#35bf28}+0.30\%$
test_step_mdp_speed[True-True-True-False-True] 34.1240μs 12.4975μs 80.0161 KOps/s 78.7046 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[True-True-True-False-False] 29.4550μs 7.6106μs 131.3962 KOps/s 130.7048 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[True-True-False-True-True] 69.5200μs 22.7825μs 43.8932 KOps/s 43.9095 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-True-False-True-False] 41.0070μs 14.3477μs 69.6978 KOps/s 69.8392 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[True-True-False-False-True] 56.7460μs 13.8305μs 72.3042 KOps/s 72.4439 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[True-True-False-False-False] 32.0900μs 8.7425μs 114.3835 KOps/s 111.3041 KOps/s $\color{#35bf28}+2.77\%$
test_step_mdp_speed[True-False-True-True-True] 77.7850μs 23.7964μs 42.0231 KOps/s 40.9701 KOps/s $\color{#35bf28}+2.57\%$
test_step_mdp_speed[True-False-True-True-False] 51.9270μs 15.6695μs 63.8184 KOps/s 63.0521 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[True-False-True-False-True] 37.4300μs 13.6574μs 73.2202 KOps/s 71.8658 KOps/s $\color{#35bf28}+1.88\%$
test_step_mdp_speed[True-False-True-False-False] 60.1050μs 8.7753μs 113.9568 KOps/s 112.4567 KOps/s $\color{#35bf28}+1.33\%$
test_step_mdp_speed[True-False-False-True-True] 75.8110μs 25.0979μs 39.8440 KOps/s 39.2254 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[True-False-False-True-False] 43.5410μs 16.6455μs 60.0761 KOps/s 59.2527 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-False-False-False-True] 43.4910μs 14.8472μs 67.3528 KOps/s 66.4361 KOps/s $\color{#35bf28}+1.38\%$
test_step_mdp_speed[True-False-False-False-False] 52.6580μs 9.8431μs 101.5941 KOps/s 97.6965 KOps/s $\color{#35bf28}+3.99\%$
test_step_mdp_speed[False-True-True-True-True] 50.1830μs 23.9181μs 41.8093 KOps/s 41.1695 KOps/s $\color{#35bf28}+1.55\%$
test_step_mdp_speed[False-True-True-True-False] 50.9050μs 15.5670μs 64.2386 KOps/s 63.4951 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-True-True-False-True] 54.7920μs 16.0821μs 62.1808 KOps/s 61.7488 KOps/s $\color{#35bf28}+0.70\%$
test_step_mdp_speed[False-True-True-False-False] 80.2060μs 9.8388μs 101.6383 KOps/s 97.8638 KOps/s $\color{#35bf28}+3.86\%$
test_step_mdp_speed[False-True-False-True-True] 37.8110μs 25.5233μs 39.1798 KOps/s 38.7069 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[False-True-False-True-False] 51.0650μs 16.7753μs 59.6113 KOps/s 58.6080 KOps/s $\color{#35bf28}+1.71\%$
test_step_mdp_speed[False-True-False-False-True] 38.0010μs 17.1616μs 58.2697 KOps/s 56.9805 KOps/s $\color{#35bf28}+2.26\%$
test_step_mdp_speed[False-True-False-False-False] 0.1057ms 11.1868μs 89.3912 KOps/s 87.2226 KOps/s $\color{#35bf28}+2.49\%$
test_step_mdp_speed[False-False-True-True-True] 0.1966ms 26.9562μs 37.0972 KOps/s 36.9539 KOps/s $\color{#35bf28}+0.39\%$
test_step_mdp_speed[False-False-True-True-False] 68.5780μs 17.9138μs 55.8227 KOps/s 54.3562 KOps/s $\color{#35bf28}+2.70\%$
test_step_mdp_speed[False-False-True-False-True] 41.8480μs 17.0075μs 58.7975 KOps/s 57.1254 KOps/s $\color{#35bf28}+2.93\%$
test_step_mdp_speed[False-False-True-False-False] 65.7960μs 11.1654μs 89.5628 KOps/s 87.3646 KOps/s $\color{#35bf28}+2.52\%$
test_step_mdp_speed[False-False-False-True-True] 61.0940μs 27.3662μs 36.5415 KOps/s 36.3703 KOps/s $\color{#35bf28}+0.47\%$
test_step_mdp_speed[False-False-False-True-False] 67.3020μs 18.9454μs 52.7832 KOps/s 51.3563 KOps/s $\color{#35bf28}+2.78\%$
test_step_mdp_speed[False-False-False-False-True] 0.1288ms 18.1233μs 55.1775 KOps/s 54.3137 KOps/s $\color{#35bf28}+1.59\%$
test_step_mdp_speed[False-False-False-False-False] 0.1071ms 12.2879μs 81.3810 KOps/s 79.6908 KOps/s $\color{#35bf28}+2.12\%$
test_values[generalized_advantage_estimate-True-True] 10.1316ms 9.1236ms 109.6060 Ops/s 107.7683 Ops/s $\color{#35bf28}+1.71\%$
test_values[vec_generalized_advantage_estimate-True-True] 35.6070ms 33.3254ms 30.0071 Ops/s 29.4166 Ops/s $\color{#35bf28}+2.01\%$
test_values[td0_return_estimate-False-False] 0.2420ms 0.1666ms 6.0030 KOps/s 6.1205 KOps/s $\color{#d91a1a}-1.92\%$
test_values[td1_return_estimate-False-False] 25.8393ms 22.6626ms 44.1255 Ops/s 42.6923 Ops/s $\color{#35bf28}+3.36\%$
test_values[vec_td1_return_estimate-False-False] 34.6188ms 33.3340ms 29.9994 Ops/s 29.7391 Ops/s $\color{#35bf28}+0.88\%$
test_values[td_lambda_return_estimate-True-False] 36.1228ms 32.9824ms 30.3192 Ops/s 29.8696 Ops/s $\color{#35bf28}+1.51\%$
test_values[vec_td_lambda_return_estimate-True-False] 34.8634ms 33.4923ms 29.8576 Ops/s 29.6690 Ops/s $\color{#35bf28}+0.64\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.2702ms 8.0452ms 124.2974 Ops/s 121.6858 Ops/s $\color{#35bf28}+2.15\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2978ms 1.8561ms 538.7518 Ops/s 495.4111 Ops/s $\textbf{\color{#35bf28}+8.75\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5332ms 0.3585ms 2.7893 KOps/s 2.8467 KOps/s $\color{#d91a1a}-2.02\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 49.7044ms 43.5913ms 22.9404 Ops/s 21.5079 Ops/s $\textbf{\color{#35bf28}+6.66\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.5959ms 3.0271ms 330.3543 Ops/s 331.4751 Ops/s $\color{#d91a1a}-0.34\%$
test_dqn_speed 1.9877ms 1.3974ms 715.5928 Ops/s 729.8663 Ops/s $\color{#d91a1a}-1.96\%$
test_ddpg_speed 3.0414ms 2.7430ms 364.5675 Ops/s 370.8667 Ops/s $\color{#d91a1a}-1.70\%$
test_sac_speed 9.1507ms 8.3803ms 119.3273 Ops/s 109.7101 Ops/s $\textbf{\color{#35bf28}+8.77\%}$
test_redq_speed 15.0343ms 13.3648ms 74.8235 Ops/s 75.9064 Ops/s $\color{#d91a1a}-1.43\%$
test_redq_deprec_speed 14.2999ms 13.3644ms 74.8257 Ops/s 74.5300 Ops/s $\color{#35bf28}+0.40\%$
test_td3_speed 9.2750ms 8.3293ms 120.0582 Ops/s 122.0391 Ops/s $\color{#d91a1a}-1.62\%$
test_cql_speed 37.8396ms 36.4862ms 27.4076 Ops/s 27.5469 Ops/s $\color{#d91a1a}-0.51\%$
test_a2c_speed 8.6219ms 7.4484ms 134.2570 Ops/s 134.3951 Ops/s $\color{#d91a1a}-0.10\%$
test_ppo_speed 8.2867ms 7.6449ms 130.8065 Ops/s 131.2059 Ops/s $\color{#d91a1a}-0.30\%$
test_reinforce_speed 7.5818ms 6.5806ms 151.9622 Ops/s 152.5340 Ops/s $\color{#d91a1a}-0.37\%$
test_iql_speed 33.9566ms 32.7396ms 30.5441 Ops/s 30.6230 Ops/s $\color{#d91a1a}-0.26\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3466ms 2.1305ms 469.3798 Ops/s 466.7583 Ops/s $\color{#35bf28}+0.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7550ms 0.5046ms 1.9819 KOps/s 1.9798 KOps/s $\color{#35bf28}+0.10\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7849ms 0.4761ms 2.1005 KOps/s 2.1053 KOps/s $\color{#d91a1a}-0.23\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.4388ms 2.1258ms 470.4150 Ops/s 464.2539 Ops/s $\color{#35bf28}+1.33\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7959ms 0.4954ms 2.0185 KOps/s 2.0111 KOps/s $\color{#35bf28}+0.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.4220ms 0.4696ms 2.1293 KOps/s 2.1238 KOps/s $\color{#35bf28}+0.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6990ms 1.2357ms 809.2683 Ops/s 814.8469 Ops/s $\color{#d91a1a}-0.68\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.7780ms 1.1667ms 857.1148 Ops/s 869.5917 Ops/s $\color{#d91a1a}-1.43\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.6857ms 2.2527ms 443.9099 Ops/s 444.1562 Ops/s $\color{#d91a1a}-0.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9857ms 0.6227ms 1.6059 KOps/s 1.6139 KOps/s $\color{#d91a1a}-0.50\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8451ms 0.5938ms 1.6839 KOps/s 1.6656 KOps/s $\color{#35bf28}+1.10\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.3010ms 2.1127ms 473.3229 Ops/s 472.3612 Ops/s $\color{#35bf28}+0.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8675ms 0.5057ms 1.9776 KOps/s 1.9927 KOps/s $\color{#d91a1a}-0.75\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6410ms 0.4790ms 2.0875 KOps/s 2.0782 KOps/s $\color{#35bf28}+0.45\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.6098ms 2.1901ms 456.6006 Ops/s 456.7943 Ops/s $\color{#d91a1a}-0.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0595ms 0.5036ms 1.9859 KOps/s 2.0190 KOps/s $\color{#d91a1a}-1.64\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6274ms 0.4701ms 2.1274 KOps/s 2.1263 KOps/s $\color{#35bf28}+0.05\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0941ms 2.2345ms 447.5196 Ops/s 440.6557 Ops/s $\color{#35bf28}+1.56\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7426ms 0.6200ms 1.6128 KOps/s 1.5992 KOps/s $\color{#35bf28}+0.86\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.5222ms 0.5975ms 1.6737 KOps/s 1.6668 KOps/s $\color{#35bf28}+0.41\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1065s 7.7209ms 129.5191 Ops/s 126.3402 Ops/s $\color{#35bf28}+2.52\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.5973ms 13.3818ms 74.7283 Ops/s 81.2091 Ops/s $\textbf{\color{#d91a1a}-7.98\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.1808ms 1.5588ms 641.5133 Ops/s 914.2058 Ops/s $\textbf{\color{#d91a1a}-29.83\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1049s 5.6744ms 176.2301 Ops/s 179.2200 Ops/s $\color{#d91a1a}-1.67\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1185s 14.2999ms 69.9307 Ops/s 68.7074 Ops/s $\color{#35bf28}+1.78\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.2458ms 1.0655ms 938.4906 Ops/s 961.6593 Ops/s $\color{#d91a1a}-2.41\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1114s 6.1297ms 163.1388 Ops/s 166.8802 Ops/s $\color{#d91a1a}-2.24\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.2888ms 12.6102ms 79.3009 Ops/s 76.1531 Ops/s $\color{#35bf28}+4.13\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.7028ms 1.3694ms 730.2688 Ops/s 706.6737 Ops/s $\color{#35bf28}+3.34\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 23, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1011s 99.8512ms 10.0149 Ops/s 9.4887 Ops/s $\textbf{\color{#35bf28}+5.55\%}$
test_sync 87.5429ms 86.7895ms 11.5221 Ops/s 11.3916 Ops/s $\color{#35bf28}+1.15\%$
test_async 0.1647s 71.4710ms 13.9917 Ops/s 11.8748 Ops/s $\textbf{\color{#35bf28}+17.83\%}$
test_single_pixels 0.1112s 0.1110s 9.0094 Ops/s 9.0985 Ops/s $\color{#d91a1a}-0.98\%$
test_sync_pixels 68.3056ms 66.2751ms 15.0886 Ops/s 14.9212 Ops/s $\color{#35bf28}+1.12\%$
test_async_pixels 65.9040ms 62.5838ms 15.9786 Ops/s 16.1243 Ops/s $\color{#d91a1a}-0.90\%$
test_simple 0.6699s 0.6696s 1.4934 Ops/s 1.4632 Ops/s $\color{#35bf28}+2.06\%$
test_transformed 0.8909s 0.8897s 1.1240 Ops/s 1.1118 Ops/s $\color{#35bf28}+1.09\%$
test_serial 2.1487s 2.0888s 0.4788 Ops/s 0.4845 Ops/s $\color{#d91a1a}-1.18\%$
test_parallel 1.7919s 1.7486s 0.5719 Ops/s 0.5718 Ops/s $\color{#35bf28}+0.01\%$
test_step_mdp_speed[True-True-True-True-True] 69.8730μs 32.1384μs 31.1154 KOps/s 30.4323 KOps/s $\color{#35bf28}+2.24\%$
test_step_mdp_speed[True-True-True-True-False] 40.1720μs 19.1196μs 52.3024 KOps/s 51.4804 KOps/s $\color{#35bf28}+1.60\%$
test_step_mdp_speed[True-True-True-False-True] 39.7220μs 18.4007μs 54.3457 KOps/s 53.3300 KOps/s $\color{#35bf28}+1.90\%$
test_step_mdp_speed[True-True-True-False-False] 28.4220μs 11.1775μs 89.4651 KOps/s 90.3688 KOps/s $\color{#d91a1a}-1.00\%$
test_step_mdp_speed[True-True-False-True-True] 49.0630μs 33.9769μs 29.4318 KOps/s 29.1069 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[True-True-False-True-False] 39.3520μs 21.2268μs 47.1102 KOps/s 46.8904 KOps/s $\color{#35bf28}+0.47\%$
test_step_mdp_speed[True-True-False-False-True] 38.3120μs 20.2002μs 49.5044 KOps/s 49.1147 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[True-True-False-False-False] 33.2620μs 13.0196μs 76.8070 KOps/s 77.0908 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-False-True-True-True] 0.1997ms 35.9825μs 27.7913 KOps/s 27.3219 KOps/s $\color{#35bf28}+1.72\%$
test_step_mdp_speed[True-False-True-True-False] 46.6030μs 22.9155μs 43.6385 KOps/s 43.0449 KOps/s $\color{#35bf28}+1.38\%$
test_step_mdp_speed[True-False-True-False-True] 39.0920μs 20.0970μs 49.7586 KOps/s 49.8219 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-False-True-False-False] 30.2410μs 12.7699μs 78.3092 KOps/s 77.3521 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-False-False-True-True] 64.9030μs 37.0129μs 27.0176 KOps/s 26.1835 KOps/s $\color{#35bf28}+3.19\%$
test_step_mdp_speed[True-False-False-True-False] 50.4630μs 24.7422μs 40.4168 KOps/s 40.0776 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[True-False-False-False-True] 45.1820μs 22.0089μs 45.4361 KOps/s 45.7213 KOps/s $\color{#d91a1a}-0.62\%$
test_step_mdp_speed[True-False-False-False-False] 37.6320μs 14.5273μs 68.8361 KOps/s 68.0971 KOps/s $\color{#35bf28}+1.09\%$
test_step_mdp_speed[False-True-True-True-True] 59.4730μs 35.7658μs 27.9597 KOps/s 27.5314 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[False-True-True-True-False] 51.5920μs 23.1998μs 43.1038 KOps/s 42.8983 KOps/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[False-True-True-False-True] 44.0420μs 24.0449μs 41.5889 KOps/s 41.7407 KOps/s $\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-True-True-False-False] 31.8920μs 14.5798μs 68.5881 KOps/s 67.8168 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[False-True-False-True-True] 58.5630μs 38.4154μs 26.0312 KOps/s 25.4776 KOps/s $\color{#35bf28}+2.17\%$
test_step_mdp_speed[False-True-False-True-False] 44.4720μs 24.7094μs 40.4704 KOps/s 39.9582 KOps/s $\color{#35bf28}+1.28\%$
test_step_mdp_speed[False-True-False-False-True] 45.5220μs 25.6930μs 38.9210 KOps/s 38.7307 KOps/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[False-True-False-False-False] 34.9110μs 16.2393μs 61.5790 KOps/s 60.5561 KOps/s $\color{#35bf28}+1.69\%$
test_step_mdp_speed[False-False-True-True-True] 63.7930μs 39.4289μs 25.3621 KOps/s 24.8948 KOps/s $\color{#35bf28}+1.88\%$
test_step_mdp_speed[False-False-True-True-False] 43.4820μs 26.8364μs 37.2628 KOps/s 36.8977 KOps/s $\color{#35bf28}+0.99\%$
test_step_mdp_speed[False-False-True-False-True] 48.0320μs 25.4649μs 39.2698 KOps/s 38.9657 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[False-False-True-False-False] 34.0920μs 16.2962μs 61.3639 KOps/s 60.5194 KOps/s $\color{#35bf28}+1.40\%$
test_step_mdp_speed[False-False-False-True-True] 67.8730μs 41.0110μs 24.3837 KOps/s 24.1408 KOps/s $\color{#35bf28}+1.01\%$
test_step_mdp_speed[False-False-False-True-False] 49.3320μs 28.2482μs 35.4005 KOps/s 34.7668 KOps/s $\color{#35bf28}+1.82\%$
test_step_mdp_speed[False-False-False-False-True] 43.0020μs 26.7370μs 37.4014 KOps/s 36.3643 KOps/s $\color{#35bf28}+2.85\%$
test_step_mdp_speed[False-False-False-False-False] 38.1020μs 17.9508μs 55.7079 KOps/s 55.1400 KOps/s $\color{#35bf28}+1.03\%$
test_values[generalized_advantage_estimate-True-True] 24.9892ms 24.0655ms 41.5533 Ops/s 41.4968 Ops/s $\color{#35bf28}+0.14\%$
test_values[vec_generalized_advantage_estimate-True-True] 81.9855ms 3.2050ms 312.0104 Ops/s 313.9494 Ops/s $\color{#d91a1a}-0.62\%$
test_values[td0_return_estimate-False-False] 94.9350μs 64.3197μs 15.5473 KOps/s 15.4386 KOps/s $\color{#35bf28}+0.70\%$
test_values[td1_return_estimate-False-False] 52.1515ms 51.5098ms 19.4138 Ops/s 19.3410 Ops/s $\color{#35bf28}+0.38\%$
test_values[vec_td1_return_estimate-False-False] 1.9867ms 1.7554ms 569.6793 Ops/s 570.2427 Ops/s $\color{#d91a1a}-0.10\%$
test_values[td_lambda_return_estimate-True-False] 82.7564ms 82.0365ms 12.1897 Ops/s 12.1110 Ops/s $\color{#35bf28}+0.65\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.1003ms 1.7547ms 569.8924 Ops/s 570.5043 Ops/s $\color{#d91a1a}-0.11\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.9088ms 22.6977ms 44.0573 Ops/s 43.5979 Ops/s $\color{#35bf28}+1.05\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8965ms 0.6979ms 1.4328 KOps/s 1.4398 KOps/s $\color{#d91a1a}-0.49\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7874ms 0.6442ms 1.5522 KOps/s 1.5599 KOps/s $\color{#d91a1a}-0.49\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6011ms 1.4507ms 689.3303 Ops/s 690.7809 Ops/s $\color{#d91a1a}-0.21\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9500ms 0.6669ms 1.4994 KOps/s 1.5027 KOps/s $\color{#d91a1a}-0.22\%$
test_dqn_speed 76.4413ms 1.6134ms 619.7952 Ops/s 709.5023 Ops/s $\textbf{\color{#d91a1a}-12.64\%}$
test_ddpg_speed 3.6604ms 2.8032ms 356.7294 Ops/s 366.7624 Ops/s $\color{#d91a1a}-2.74\%$
test_sac_speed 11.1644ms 8.4258ms 118.6830 Ops/s 123.6743 Ops/s $\color{#d91a1a}-4.04\%$
test_redq_speed 12.4228ms 10.5345ms 94.9264 Ops/s 96.4835 Ops/s $\color{#d91a1a}-1.61\%$
test_redq_deprec_speed 12.8321ms 11.4692ms 87.1897 Ops/s 91.5308 Ops/s $\color{#d91a1a}-4.74\%$
test_td3_speed 8.4998ms 8.1313ms 122.9814 Ops/s 124.9177 Ops/s $\color{#d91a1a}-1.55\%$
test_cql_speed 26.8523ms 25.6571ms 38.9755 Ops/s 39.1617 Ops/s $\color{#d91a1a}-0.48\%$
test_a2c_speed 5.8272ms 5.5838ms 179.0908 Ops/s 180.1056 Ops/s $\color{#d91a1a}-0.56\%$
test_ppo_speed 6.4375ms 5.9396ms 168.3603 Ops/s 171.4914 Ops/s $\color{#d91a1a}-1.83\%$
test_reinforce_speed 4.8000ms 4.5062ms 221.9155 Ops/s 222.8599 Ops/s $\color{#d91a1a}-0.42\%$
test_iql_speed 20.8149ms 19.9477ms 50.1312 Ops/s 51.4459 Ops/s $\color{#d91a1a}-2.56\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0089ms 2.8135ms 355.4289 Ops/s 356.5074 Ops/s $\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1662ms 0.5456ms 1.8328 KOps/s 1.8511 KOps/s $\color{#d91a1a}-0.99\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6998ms 0.5207ms 1.9203 KOps/s 1.9400 KOps/s $\color{#d91a1a}-1.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0838ms 2.8282ms 353.5842 Ops/s 355.6122 Ops/s $\color{#d91a1a}-0.57\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1390ms 0.5419ms 1.8452 KOps/s 1.8834 KOps/s $\color{#d91a1a}-2.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7065ms 0.5191ms 1.9262 KOps/s 1.9706 KOps/s $\color{#d91a1a}-2.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6802ms 1.4497ms 689.7854 Ops/s 700.6565 Ops/s $\color{#d91a1a}-1.55\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 5.3031ms 1.3816ms 723.7818 Ops/s 708.0337 Ops/s $\color{#35bf28}+2.22\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1102ms 2.9057ms 344.1459 Ops/s 342.5923 Ops/s $\color{#35bf28}+0.45\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8402ms 0.6688ms 1.4952 KOps/s 1.5122 KOps/s $\color{#d91a1a}-1.12\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1210s 0.7451ms 1.3422 KOps/s 1.5721 KOps/s $\textbf{\color{#d91a1a}-14.63\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0591ms 2.8236ms 354.1621 Ops/s 358.5619 Ops/s $\color{#d91a1a}-1.23\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7096ms 0.5487ms 1.8224 KOps/s 1.8441 KOps/s $\color{#d91a1a}-1.18\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7142ms 0.5224ms 1.9144 KOps/s 1.9175 KOps/s $\color{#d91a1a}-0.16\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0935ms 2.8443ms 351.5779 Ops/s 353.8337 Ops/s $\color{#d91a1a}-0.64\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.2498ms 0.5418ms 1.8457 KOps/s 1.8687 KOps/s $\color{#d91a1a}-1.23\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.1231s 0.7018ms 1.4250 KOps/s 1.9609 KOps/s $\textbf{\color{#d91a1a}-27.33\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2196ms 2.9612ms 337.7026 Ops/s 343.8153 Ops/s $\color{#d91a1a}-1.78\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8253ms 0.6810ms 1.4685 KOps/s 1.5018 KOps/s $\color{#d91a1a}-2.22\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8779ms 0.6502ms 1.5379 KOps/s 1.5502 KOps/s $\color{#d91a1a}-0.79\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1189s 7.2687ms 137.5764 Ops/s 104.3132 Ops/s $\textbf{\color{#35bf28}+31.89\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1286s 16.9552ms 58.9788 Ops/s 67.9319 Ops/s $\textbf{\color{#d91a1a}-13.18\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.2140ms 1.1171ms 895.2033 Ops/s 886.7232 Ops/s $\color{#35bf28}+0.96\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1170s 7.1762ms 139.3497 Ops/s 140.9480 Ops/s $\color{#d91a1a}-1.13\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.9658ms 14.7476ms 67.8074 Ops/s 68.0269 Ops/s $\color{#d91a1a}-0.32\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.4468ms 1.2456ms 802.7973 Ops/s 877.9307 Ops/s $\textbf{\color{#d91a1a}-8.56\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1174s 7.5264ms 132.8651 Ops/s 133.0720 Ops/s $\color{#d91a1a}-0.16\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.3929ms 15.1042ms 66.2067 Ops/s 57.0588 Ops/s $\textbf{\color{#35bf28}+16.03\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.8432ms 1.5702ms 636.8638 Ops/s 670.2429 Ops/s $\color{#d91a1a}-4.98\%$

@vmoens vmoens added the enhancement New feature or request label Apr 23, 2024
Miffyli
Miffyli reviewed Apr 23, 2024
View reviewed changes
Copy link

@Miffyli Miffyli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this stuff! I don't dare to give LGTM, being still relatively fresh to TorchRL, but just left some generic comments :)

docs/source/reference/envs.rst
Recording data during environment rollout execution is crucial to keep an eye on the algorithm performance as well as
reporting results after training.

TorchRL offers several tools to interact with the environment output: first and foremost, a ``callback`` callable
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BIt of a sidenote, but at least in current public doc version the signature of the callback is not very clear. Maybe clarify here (or ideally in rollout's doc) that the callback is called with callback(self, tensordict)?

docs/source/reference/envs.rst Outdated
(some libraries only allow one environment instance per workspace).
In these cases, assuming that a `render` method is available in the environment, the :class:`~torchrl.record.PixelRenderTransform`
can be used to call `render` on the parent environment and save the images in the rollout data stream.
This class should only be used within the same process as the environment that is being rendered (remote calls to `render`
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class should only be used within the same process as the environment that is being rendered

This part is bit unclear. Is it saying that, for example, using ParallelEnv with multiprocessing won't work, and the environments need to be serial/in the same process for things to work?

torchrl/record/recorder.py
This transform offers an alternative to the ``from_pixels`` syntatic sugar when instantiating an environment
that offers rendering is expensive, or when ``from_pixels`` is not implemented.

Args:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a chance to add some way to disable/enable this transform, so that you can control when you record videos (if calls to render are expensive)? I am thinking of just having transform.enable() and transform.disable() type of things, but I have no idea how it collides with style of TorchRL or functionality :D

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right we talked about that! Sure that makes sense
then you could use env.apply to enable/disable it

@vmoens vmoens merged commit df749a3 into main Apr 23, 2024
2 of 3 checks passed
@vmoens vmoens deleted the render-transform branch April 23, 2024 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Miffyli Miffyli Miffyli left review comments

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vmoens @Miffyli @facebook-github-bot