Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Temporarily remove snapshot from CI #2000

Merged
merged 1 commit into from
Mar 7, 2024
Merged

[CI] Temporarily remove snapshot from CI #2000

merged 1 commit into from
Mar 7, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Mar 7, 2024 •
edited
Loading

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 7, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2000

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit cf8a9af with merge base 535bd63 (image):

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 7, 2024
@vmoens vmoens marked this pull request as ready for review March 7, 2024 12:00
@vmoens vmoens added the CI Has to do with CI setup (e.g. wheels & builds, tests...) label Mar 7, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 7, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 60.2092ms 59.8973ms 16.6952 Ops/s 16.4854 Ops/s $\color{#35bf28}+1.27\%$
test_sync 39.3670ms 32.6446ms 30.6329 Ops/s 29.1781 Ops/s $\color{#35bf28}+4.99\%$
test_async 60.3904ms 29.0731ms 34.3961 Ops/s 32.1707 Ops/s $\textbf{\color{#35bf28}+6.92\%}$
test_simple 0.4660s 0.4140s 2.4152 Ops/s 2.3653 Ops/s $\color{#35bf28}+2.11\%$
test_transformed 0.6197s 0.5694s 1.7562 Ops/s 1.7403 Ops/s $\color{#35bf28}+0.91\%$
test_serial 1.4211s 1.3745s 0.7275 Ops/s 0.7072 Ops/s $\color{#35bf28}+2.88\%$
test_parallel 1.4466s 1.3628s 0.7338 Ops/s 0.7134 Ops/s $\color{#35bf28}+2.85\%$
test_step_mdp_speed[True-True-True-True-True] 0.1329ms 20.8564μs 47.9470 KOps/s 47.0534 KOps/s $\color{#35bf28}+1.90\%$
test_step_mdp_speed[True-True-True-True-False] 52.9990μs 12.8736μs 77.6783 KOps/s 76.4300 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-True-True-False-True] 38.1120μs 12.2208μs 81.8280 KOps/s 80.1194 KOps/s $\color{#35bf28}+2.13\%$
test_step_mdp_speed[True-True-True-False-False] 46.9470μs 7.4759μs 133.7638 KOps/s 130.7718 KOps/s $\color{#35bf28}+2.29\%$
test_step_mdp_speed[True-True-False-True-True] 59.4410μs 22.1964μs 45.0524 KOps/s 43.9867 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[True-True-False-True-False] 0.1312ms 14.2615μs 70.1188 KOps/s 69.7157 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[True-True-False-False-True] 49.6130μs 13.5337μs 73.8896 KOps/s 72.8254 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[True-True-False-False-False] 43.5220μs 8.7460μs 114.3381 KOps/s 112.0171 KOps/s $\color{#35bf28}+2.07\%$
test_step_mdp_speed[True-False-True-True-True] 47.6290μs 23.7113μs 42.1741 KOps/s 41.1198 KOps/s $\color{#35bf28}+2.56\%$
test_step_mdp_speed[True-False-True-True-False] 52.6080μs 15.5224μs 64.4231 KOps/s 62.9806 KOps/s $\color{#35bf28}+2.29\%$
test_step_mdp_speed[True-False-True-False-True] 46.1060μs 13.4423μs 74.3922 KOps/s 72.5213 KOps/s $\color{#35bf28}+2.58\%$
test_step_mdp_speed[True-False-True-False-False] 53.1190μs 8.6251μs 115.9400 KOps/s 112.4384 KOps/s $\color{#35bf28}+3.11\%$
test_step_mdp_speed[True-False-False-True-True] 93.4450μs 24.9751μs 40.0398 KOps/s 39.5541 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[True-False-False-True-False] 62.9470μs 16.5528μs 60.4127 KOps/s 58.7851 KOps/s $\color{#35bf28}+2.77\%$
test_step_mdp_speed[True-False-False-False-True] 41.3580μs 14.6068μs 68.4613 KOps/s 67.2407 KOps/s $\color{#35bf28}+1.82\%$
test_step_mdp_speed[True-False-False-False-False] 0.1214ms 9.9273μs 100.7319 KOps/s 98.9401 KOps/s $\color{#35bf28}+1.81\%$
test_step_mdp_speed[False-True-True-True-True] 74.9270μs 23.4617μs 42.6227 KOps/s 41.3201 KOps/s $\color{#35bf28}+3.15\%$
test_step_mdp_speed[False-True-True-True-False] 40.6160μs 15.4373μs 64.7782 KOps/s 63.4487 KOps/s $\color{#35bf28}+2.10\%$
test_step_mdp_speed[False-True-True-False-True] 41.9780μs 15.7647μs 63.4329 KOps/s 63.0581 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[False-True-True-False-False] 36.9800μs 9.9537μs 100.4651 KOps/s 99.2089 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[False-True-False-True-True] 38.8720μs 25.4580μs 39.2804 KOps/s 39.0781 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[False-True-False-True-False] 43.7020μs 16.5987μs 60.2457 KOps/s 59.5257 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[False-True-False-False-True] 99.0550μs 17.1039μs 58.4662 KOps/s 58.2805 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[False-True-False-False-False] 43.3100μs 11.1973μs 89.3074 KOps/s 88.9026 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-False-True-True-True] 60.3030μs 26.1496μs 38.2415 KOps/s 37.8289 KOps/s $\color{#35bf28}+1.09\%$
test_step_mdp_speed[False-False-True-True-False] 47.6600μs 17.9894μs 55.5884 KOps/s 54.2694 KOps/s $\color{#35bf28}+2.43\%$
test_step_mdp_speed[False-False-True-False-True] 55.5630μs 16.8439μs 59.3687 KOps/s 58.2269 KOps/s $\color{#35bf28}+1.96\%$
test_step_mdp_speed[False-False-True-False-False] 37.4700μs 11.1221μs 89.9111 KOps/s 87.5739 KOps/s $\color{#35bf28}+2.67\%$
test_step_mdp_speed[False-False-False-True-True] 85.7300μs 27.5360μs 36.3161 KOps/s 36.0530 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[False-False-False-True-False] 44.2830μs 19.0142μs 52.5924 KOps/s 51.5141 KOps/s $\color{#35bf28}+2.09\%$
test_step_mdp_speed[False-False-False-False-True] 41.2470μs 18.0522μs 55.3949 KOps/s 54.9425 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[False-False-False-False-False] 42.6990μs 12.2865μs 81.3898 KOps/s 80.1226 KOps/s $\color{#35bf28}+1.58\%$
test_values[generalized_advantage_estimate-True-True] 9.9562ms 9.4639ms 105.6641 Ops/s 109.6968 Ops/s $\color{#d91a1a}-3.68\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.2503ms 35.3204ms 28.3123 Ops/s 28.3097 Ops/s $+0.01\%$
test_values[td0_return_estimate-False-False] 0.2491ms 0.1738ms 5.7535 KOps/s 5.8168 KOps/s $\color{#d91a1a}-1.09\%$
test_values[td1_return_estimate-False-False] 26.4493ms 23.2603ms 42.9916 Ops/s 43.8592 Ops/s $\color{#d91a1a}-1.98\%$
test_values[vec_td1_return_estimate-False-False] 36.3849ms 35.2958ms 28.3320 Ops/s 28.1896 Ops/s $\color{#35bf28}+0.50\%$
test_values[td_lambda_return_estimate-True-False] 36.9979ms 33.4756ms 29.8725 Ops/s 30.3756 Ops/s $\color{#d91a1a}-1.66\%$
test_values[vec_td_lambda_return_estimate-True-False] 38.9975ms 35.3767ms 28.2672 Ops/s 28.2329 Ops/s $\color{#35bf28}+0.12\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.5527ms 8.2002ms 121.9478 Ops/s 124.3080 Ops/s $\color{#d91a1a}-1.90\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3099ms 1.9658ms 508.6867 Ops/s 495.7173 Ops/s $\color{#35bf28}+2.62\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5160ms 0.3497ms 2.8599 KOps/s 2.8595 KOps/s $\color{#35bf28}+0.02\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 48.2098ms 45.9712ms 21.7527 Ops/s 21.5485 Ops/s $\color{#35bf28}+0.95\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.6603ms 3.0354ms 329.4450 Ops/s 328.3841 Ops/s $\color{#35bf28}+0.32\%$
test_dqn_speed 6.9384ms 1.3469ms 742.4623 Ops/s 735.8136 Ops/s $\color{#35bf28}+0.90\%$
test_ddpg_speed 3.2207ms 2.6682ms 374.7871 Ops/s 371.7809 Ops/s $\color{#35bf28}+0.81\%$
test_sac_speed 9.7102ms 8.2505ms 121.2054 Ops/s 120.6109 Ops/s $\color{#35bf28}+0.49\%$
test_redq_speed 13.9236ms 13.1004ms 76.3333 Ops/s 75.3616 Ops/s $\color{#35bf28}+1.29\%$
test_redq_deprec_speed 15.4603ms 13.3954ms 74.6523 Ops/s 75.2163 Ops/s $\color{#d91a1a}-0.75\%$
test_td3_speed 8.4537ms 8.0990ms 123.4719 Ops/s 123.7757 Ops/s $\color{#d91a1a}-0.25\%$
test_cql_speed 40.2371ms 36.3165ms 27.5357 Ops/s 27.4912 Ops/s $\color{#35bf28}+0.16\%$
test_a2c_speed 8.5688ms 7.3562ms 135.9401 Ops/s 132.6684 Ops/s $\color{#35bf28}+2.47\%$
test_ppo_speed 8.3844ms 7.7149ms 129.6188 Ops/s 131.3102 Ops/s $\color{#d91a1a}-1.29\%$
test_reinforce_speed 7.6798ms 6.6109ms 151.2651 Ops/s 146.8495 Ops/s $\color{#35bf28}+3.01\%$
test_iql_speed 34.0122ms 32.7789ms 30.5074 Ops/s 30.6626 Ops/s $\color{#d91a1a}-0.51\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.4603ms 2.2598ms 442.5075 Ops/s 446.3300 Ops/s $\color{#d91a1a}-0.86\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0016ms 0.5033ms 1.9867 KOps/s 1.8655 KOps/s $\textbf{\color{#35bf28}+6.50\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8440ms 0.4750ms 2.1051 KOps/s 2.1022 KOps/s $\color{#35bf28}+0.14\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.6239ms 2.3453ms 426.3833 Ops/s 449.6263 Ops/s $\textbf{\color{#d91a1a}-5.17\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0452ms 0.4983ms 2.0070 KOps/s 2.0345 KOps/s $\color{#d91a1a}-1.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.1266ms 0.5072ms 1.9716 KOps/s 2.1246 KOps/s $\textbf{\color{#d91a1a}-7.20\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.3875ms 1.2578ms 795.0579 Ops/s 697.4544 Ops/s $\textbf{\color{#35bf28}+13.99\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.3876ms 1.1897ms 840.5359 Ops/s 840.7509 Ops/s $\color{#d91a1a}-0.03\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.3970ms 2.4222ms 412.8557 Ops/s 401.8035 Ops/s $\color{#35bf28}+2.75\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0274ms 0.6266ms 1.5958 KOps/s 1.5860 KOps/s $\color{#35bf28}+0.62\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9573ms 0.6063ms 1.6494 KOps/s 1.6596 KOps/s $\color{#d91a1a}-0.61\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.6498ms 2.3609ms 423.5654 Ops/s 425.1240 Ops/s $\color{#d91a1a}-0.37\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6596ms 0.5126ms 1.9509 KOps/s 1.9627 KOps/s $\color{#d91a1a}-0.60\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7670ms 0.4960ms 2.0160 KOps/s 1.9942 KOps/s $\color{#35bf28}+1.10\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0056ms 2.3966ms 417.2541 Ops/s 395.8498 Ops/s $\textbf{\color{#35bf28}+5.41\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6154ms 0.5064ms 1.9746 KOps/s 1.9365 KOps/s $\color{#35bf28}+1.96\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7946ms 0.4842ms 2.0654 KOps/s 2.0955 KOps/s $\color{#d91a1a}-1.43\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.7156ms 2.5330ms 394.7869 Ops/s 394.7190 Ops/s $\color{#35bf28}+0.02\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1283ms 0.6375ms 1.5687 KOps/s 1.5840 KOps/s $\color{#d91a1a}-0.96\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7534ms 0.6054ms 1.6519 KOps/s 1.6940 KOps/s $\color{#d91a1a}-2.49\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1117s 7.7240ms 129.4673 Ops/s 179.3943 Ops/s $\textbf{\color{#d91a1a}-27.83\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.5426ms 12.1101ms 82.5759 Ops/s 85.3954 Ops/s $\color{#d91a1a}-3.30\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.8502ms 1.0685ms 935.8894 Ops/s 970.1273 Ops/s $\color{#d91a1a}-3.53\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1005s 7.5115ms 133.1292 Ops/s 141.6473 Ops/s $\textbf{\color{#d91a1a}-6.01\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.6282ms 12.0955ms 82.6756 Ops/s 85.3431 Ops/s $\color{#d91a1a}-3.13\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.8684ms 1.0674ms 936.8181 Ops/s 972.8046 Ops/s $\color{#d91a1a}-3.70\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 90.4069ms 5.6996ms 175.4496 Ops/s 177.6563 Ops/s $\color{#d91a1a}-1.24\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1057s 14.1625ms 70.6088 Ops/s 83.3432 Ops/s $\textbf{\color{#d91a1a}-15.28\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.0460ms 1.3713ms 729.2192 Ops/s 736.8459 Ops/s $\color{#d91a1a}-1.04\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 7, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1120s 0.1118s 8.9435 Ops/s 8.9386 Ops/s $\color{#35bf28}+0.05\%$
test_sync 94.8736ms 94.6927ms 10.5605 Ops/s 10.5430 Ops/s $\color{#35bf28}+0.17\%$
test_async 0.1793s 90.7783ms 11.0158 Ops/s 11.0202 Ops/s $\color{#d91a1a}-0.04\%$
test_single_pixels 0.1223s 0.1216s 8.2230 Ops/s 8.1649 Ops/s $\color{#35bf28}+0.71\%$
test_sync_pixels 82.0587ms 78.7351ms 12.7008 Ops/s 12.4205 Ops/s $\color{#35bf28}+2.26\%$
test_async_pixels 0.1455s 75.0131ms 13.3310 Ops/s 15.7693 Ops/s $\textbf{\color{#d91a1a}-15.46\%}$
test_simple 0.8837s 0.8181s 1.2223 Ops/s 1.2282 Ops/s $\color{#d91a1a}-0.47\%$
test_transformed 1.0945s 1.0365s 0.9648 Ops/s 0.9680 Ops/s $\color{#d91a1a}-0.33\%$
test_serial 2.3938s 2.3392s 0.4275 Ops/s 0.4236 Ops/s $\color{#35bf28}+0.92\%$
test_parallel 2.0731s 1.9988s 0.5003 Ops/s 0.4884 Ops/s $\color{#35bf28}+2.43\%$
test_step_mdp_speed[True-True-True-True-True] 0.1045ms 33.5471μs 29.8089 KOps/s 29.4415 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-True-True-True-False] 35.3110μs 20.3792μs 49.0696 KOps/s 49.7688 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[True-True-True-False-True] 36.7010μs 18.9904μs 52.6582 KOps/s 52.1460 KOps/s $\color{#35bf28}+0.98\%$
test_step_mdp_speed[True-True-True-False-False] 31.0000μs 11.3388μs 88.1928 KOps/s 89.7235 KOps/s $\color{#d91a1a}-1.71\%$
test_step_mdp_speed[True-True-False-True-True] 49.8210μs 35.2165μs 28.3958 KOps/s 27.9278 KOps/s $\color{#35bf28}+1.68\%$
test_step_mdp_speed[True-True-False-True-False] 38.6510μs 21.8973μs 45.6677 KOps/s 46.0246 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[True-True-False-False-True] 47.1810μs 20.7207μs 48.2609 KOps/s 48.3105 KOps/s $\color{#d91a1a}-0.10\%$
test_step_mdp_speed[True-True-False-False-False] 30.5310μs 13.2786μs 75.3094 KOps/s 76.5721 KOps/s $\color{#d91a1a}-1.65\%$
test_step_mdp_speed[True-False-True-True-True] 52.5610μs 37.1419μs 26.9238 KOps/s 26.6395 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[True-False-True-True-False] 53.1710μs 23.9019μs 41.8376 KOps/s 41.7615 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[True-False-True-False-True] 38.4710μs 20.8289μs 48.0103 KOps/s 47.6960 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[True-False-True-False-False] 38.4010μs 13.2437μs 75.5077 KOps/s 76.1321 KOps/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[True-False-False-True-True] 57.5910μs 38.9024μs 25.7053 KOps/s 25.4193 KOps/s $\color{#35bf28}+1.13\%$
test_step_mdp_speed[True-False-False-True-False] 45.2210μs 26.1460μs 38.2468 KOps/s 38.3463 KOps/s $\color{#d91a1a}-0.26\%$
test_step_mdp_speed[True-False-False-False-True] 47.9410μs 22.6839μs 44.0842 KOps/s 44.3538 KOps/s $\color{#d91a1a}-0.61\%$
test_step_mdp_speed[True-False-False-False-False] 28.7200μs 15.0652μs 66.3782 KOps/s 66.7840 KOps/s $\color{#d91a1a}-0.61\%$
test_step_mdp_speed[False-True-True-True-True] 63.0310μs 37.4864μs 26.6763 KOps/s 26.6894 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[False-True-True-True-False] 40.4310μs 23.9229μs 41.8009 KOps/s 41.6493 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-True-True-False-True] 42.3800μs 24.8667μs 40.2144 KOps/s 40.3430 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-True-True-False-False] 30.8500μs 15.1017μs 66.2178 KOps/s 66.3742 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[False-True-False-True-True] 74.8310μs 39.5009μs 25.3159 KOps/s 25.2045 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[False-True-False-True-False] 49.5220μs 26.1888μs 38.1842 KOps/s 38.3386 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-True-False-False-True] 59.0210μs 27.0681μs 36.9439 KOps/s 37.2392 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[False-True-False-False-False] 35.8700μs 17.1040μs 58.4659 KOps/s 58.6330 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[False-False-True-True-True] 60.0110μs 41.4007μs 24.1542 KOps/s 24.2942 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-False-True-True-False] 54.4710μs 27.9428μs 35.7874 KOps/s 35.8044 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[False-False-True-False-True] 53.6910μs 27.0033μs 37.0325 KOps/s 37.3672 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-False-True-False-False] 42.0810μs 16.9349μs 59.0495 KOps/s 58.9184 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[False-False-False-True-True] 61.5610μs 42.5838μs 23.4831 KOps/s 23.3253 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[False-False-False-True-False] 48.1110μs 30.1476μs 33.1701 KOps/s 33.5909 KOps/s $\color{#d91a1a}-1.25\%$
test_step_mdp_speed[False-False-False-False-True] 59.8510μs 28.4771μs 35.1160 KOps/s 35.5441 KOps/s $\color{#d91a1a}-1.20\%$
test_step_mdp_speed[False-False-False-False-False] 35.1000μs 18.8280μs 53.1123 KOps/s 53.5363 KOps/s $\color{#d91a1a}-0.79\%$
test_values[generalized_advantage_estimate-True-True] 25.0606ms 24.3158ms 41.1255 Ops/s 41.9952 Ops/s $\color{#d91a1a}-2.07\%$
test_values[vec_generalized_advantage_estimate-True-True] 94.6623ms 3.4511ms 289.7592 Ops/s 297.3334 Ops/s $\color{#d91a1a}-2.55\%$
test_values[td0_return_estimate-False-False] 0.1049ms 63.7829μs 15.6782 KOps/s 15.9491 KOps/s $\color{#d91a1a}-1.70\%$
test_values[td1_return_estimate-False-False] 52.9316ms 50.2822ms 19.8878 Ops/s 19.5069 Ops/s $\color{#35bf28}+1.95\%$
test_values[vec_td1_return_estimate-False-False] 2.1933ms 1.7555ms 569.6263 Ops/s 571.5467 Ops/s $\color{#d91a1a}-0.34\%$
test_values[td_lambda_return_estimate-True-False] 86.3632ms 83.5200ms 11.9732 Ops/s 12.0259 Ops/s $\color{#d91a1a}-0.44\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.1129ms 1.7550ms 569.8050 Ops/s 571.9900 Ops/s $\color{#d91a1a}-0.38\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.1588ms 23.8846ms 41.8680 Ops/s 43.0726 Ops/s $\color{#d91a1a}-2.80\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8638ms 0.6864ms 1.4568 KOps/s 1.4672 KOps/s $\color{#d91a1a}-0.71\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7077ms 0.6556ms 1.5254 KOps/s 1.5284 KOps/s $\color{#d91a1a}-0.19\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4682ms 1.4407ms 694.0885 Ops/s 695.1915 Ops/s $\color{#d91a1a}-0.16\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9230ms 0.6542ms 1.5286 KOps/s 1.5267 KOps/s $\color{#35bf28}+0.12\%$
test_dqn_speed 8.9858ms 1.4111ms 708.6452 Ops/s 667.7849 Ops/s $\textbf{\color{#35bf28}+6.12\%}$
test_ddpg_speed 2.9486ms 2.6744ms 373.9201 Ops/s 378.9513 Ops/s $\color{#d91a1a}-1.33\%$
test_sac_speed 8.5205ms 7.9896ms 125.1621 Ops/s 128.2068 Ops/s $\color{#d91a1a}-2.37\%$
test_redq_speed 10.9007ms 10.0628ms 99.3756 Ops/s 100.4932 Ops/s $\color{#d91a1a}-1.11\%$
test_redq_deprec_speed 11.4183ms 11.0090ms 90.8351 Ops/s 90.6039 Ops/s $\color{#35bf28}+0.26\%$
test_td3_speed 15.6876ms 7.9300ms 126.1030 Ops/s 128.2270 Ops/s $\color{#d91a1a}-1.66\%$
test_cql_speed 26.0650ms 24.9498ms 40.0804 Ops/s 40.3461 Ops/s $\color{#d91a1a}-0.66\%$
test_a2c_speed 5.7131ms 5.4231ms 184.3950 Ops/s 182.2518 Ops/s $\color{#35bf28}+1.18\%$
test_ppo_speed 6.1483ms 5.7300ms 174.5192 Ops/s 174.2128 Ops/s $\color{#35bf28}+0.18\%$
test_reinforce_speed 4.6215ms 4.3785ms 228.3912 Ops/s 221.5577 Ops/s $\color{#35bf28}+3.08\%$
test_iql_speed 19.7634ms 18.9401ms 52.7981 Ops/s 52.3286 Ops/s $\color{#35bf28}+0.90\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9761ms 2.8548ms 350.2850 Ops/s 344.4019 Ops/s $\color{#35bf28}+1.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 87.7634ms 0.6109ms 1.6369 KOps/s 1.8610 KOps/s $\textbf{\color{#d91a1a}-12.04\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7813ms 0.5130ms 1.9493 KOps/s 1.9568 KOps/s $\color{#d91a1a}-0.38\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0808ms 2.8727ms 348.1054 Ops/s 347.2099 Ops/s $\color{#35bf28}+0.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3257ms 0.5309ms 1.8835 KOps/s 1.8840 KOps/s $\color{#d91a1a}-0.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7204ms 0.5067ms 1.9734 KOps/s 1.9827 KOps/s $\color{#d91a1a}-0.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 4.1001ms 1.4174ms 705.5139 Ops/s 709.8336 Ops/s $\color{#d91a1a}-0.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5300ms 1.3419ms 745.2052 Ops/s 743.7960 Ops/s $\color{#35bf28}+0.19\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0667ms 2.9528ms 338.6584 Ops/s 332.9248 Ops/s $\color{#35bf28}+1.72\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8030ms 0.6608ms 1.5133 KOps/s 1.5140 KOps/s $\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.4603ms 0.6388ms 1.5655 KOps/s 1.5700 KOps/s $\color{#d91a1a}-0.29\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9667ms 2.8581ms 349.8859 Ops/s 349.5010 Ops/s $\color{#35bf28}+0.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7849ms 0.5389ms 1.8555 KOps/s 1.8509 KOps/s $\color{#35bf28}+0.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.5944ms 0.5180ms 1.9306 KOps/s 1.5835 KOps/s $\textbf{\color{#35bf28}+21.92\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0249ms 2.8682ms 348.6452 Ops/s 343.5134 Ops/s $\color{#35bf28}+1.49\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3280ms 0.5326ms 1.8776 KOps/s 1.8736 KOps/s $\color{#35bf28}+0.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7777ms 0.5084ms 1.9671 KOps/s 1.9747 KOps/s $\color{#d91a1a}-0.39\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0624ms 2.9790ms 335.6777 Ops/s 329.8145 Ops/s $\color{#35bf28}+1.78\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7765ms 0.6588ms 1.5179 KOps/s 1.5047 KOps/s $\color{#35bf28}+0.88\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.7407ms 0.6405ms 1.5612 KOps/s 1.2823 KOps/s $\textbf{\color{#35bf28}+21.75\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1126s 8.7486ms 114.3044 Ops/s 148.8395 Ops/s $\textbf{\color{#d91a1a}-23.20\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.5515ms 14.2879ms 69.9891 Ops/s 67.8896 Ops/s $\color{#35bf28}+3.09\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.9259ms 1.0556ms 947.3248 Ops/s 914.4342 Ops/s $\color{#35bf28}+3.60\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1044s 6.7288ms 148.6156 Ops/s 116.3331 Ops/s $\textbf{\color{#35bf28}+27.75\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1092s 16.1229ms 62.0235 Ops/s 69.5291 Ops/s $\textbf{\color{#d91a1a}-10.79\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1925ms 1.0788ms 926.9512 Ops/s 924.6096 Ops/s $\color{#35bf28}+0.25\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 98.6785ms 7.0277ms 142.2941 Ops/s 142.1227 Ops/s $\color{#35bf28}+0.12\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.0418ms 14.6067ms 68.4619 Ops/s 68.6720 Ops/s $\color{#d91a1a}-0.31\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.5559ms 1.4052ms 711.6434 Ops/s 694.1471 Ops/s $\color{#35bf28}+2.52\%$

@vmoens vmoens merged commit ad73733 into main Mar 7, 2024
66 of 67 checks passed
@vmoens vmoens deleted the deprec-snapshot-tests branch March 7, 2024 18:03
vmoens added a commit that referenced this pull request Mar 25, 2024
@vmoens
[CI] Temporarily remove snapshot from CI (#2000)
cca217e 
(cherry picked from commit ad73733)
vmoens added a commit that referenced this pull request Apr 7, 2024
@vmoens
[CI] Temporarily remove snapshot from CI (#2000)
a4795aa 
(cherry picked from commit ad73733)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot