Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Schedule workflow for release branches #759

Merged
merged 1 commit into from
Apr 25, 2024
Merged

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Apr 25, 2024

No description provided.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 25, 2024
@vmoens vmoens added the CI label Apr 25, 2024
@vmoens vmoens merged commit d244c13 into main Apr 25, 2024
20 of 31 checks passed
@vmoens vmoens deleted the worflows-releases branch April 25, 2024 13:07
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 127. Improved: $\large\color{#35bf28}34$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_plain_set_nested 39.5030μs 16.0615μs 62.2607 KOps/s 56.6528 KOps/s $\textbf{\color{#35bf28}+9.90\%}$
test_plain_set_stack_nested 45.3540μs 16.1817μs 61.7980 KOps/s 55.6117 KOps/s $\textbf{\color{#35bf28}+11.12\%}$
test_plain_set_nested_inplace 72.1550μs 18.1159μs 55.2000 KOps/s 49.8285 KOps/s $\textbf{\color{#35bf28}+10.78\%}$
test_plain_set_stack_nested_inplace 62.8570μs 18.1555μs 55.0799 KOps/s 49.3837 KOps/s $\textbf{\color{#35bf28}+11.53\%}$
test_items 22.2420μs 2.4935μs 401.0401 KOps/s 400.6938 KOps/s $\color{#35bf28}+0.09\%$
test_items_nested 1.0582ms 0.2722ms 3.6736 KOps/s 3.6932 KOps/s $\color{#d91a1a}-0.53\%$
test_items_nested_locked 0.5932ms 0.2695ms 3.7106 KOps/s 3.7872 KOps/s $\color{#d91a1a}-2.02\%$
test_items_nested_leaf 0.1305ms 77.6528μs 12.8778 KOps/s 12.7498 KOps/s $\color{#35bf28}+1.00\%$
test_items_stack_nested 0.4924ms 0.2688ms 3.7203 KOps/s 3.7819 KOps/s $\color{#d91a1a}-1.63\%$
test_items_stack_nested_leaf 0.2400ms 80.9057μs 12.3601 KOps/s 12.4835 KOps/s $\color{#d91a1a}-0.99\%$
test_items_stack_nested_locked 0.4917ms 0.2690ms 3.7174 KOps/s 3.7549 KOps/s $\color{#d91a1a}-1.00\%$
test_keys 41.8280μs 3.8397μs 260.4369 KOps/s 258.1576 KOps/s $\color{#35bf28}+0.88\%$
test_keys_nested 0.2823ms 0.1386ms 7.2132 KOps/s 7.1611 KOps/s $\color{#35bf28}+0.73\%$
test_keys_nested_locked 2.1155ms 0.1439ms 6.9478 KOps/s 6.8958 KOps/s $\color{#35bf28}+0.75\%$
test_keys_nested_leaf 0.2059ms 0.1189ms 8.4099 KOps/s 8.4786 KOps/s $\color{#d91a1a}-0.81\%$
test_keys_stack_nested 0.2637ms 0.1402ms 7.1304 KOps/s 7.1685 KOps/s $\color{#d91a1a}-0.53\%$
test_keys_stack_nested_leaf 0.2002ms 0.1189ms 8.4128 KOps/s 8.4500 KOps/s $\color{#d91a1a}-0.44\%$
test_keys_stack_nested_locked 0.2751ms 0.1441ms 6.9398 KOps/s 7.0113 KOps/s $\color{#d91a1a}-1.02\%$
test_values 10.8253μs 1.1752μs 850.8861 KOps/s 853.7306 KOps/s $\color{#d91a1a}-0.33\%$
test_values_nested 0.1336ms 51.3601μs 19.4704 KOps/s 19.2511 KOps/s $\color{#35bf28}+1.14\%$
test_values_nested_locked 98.1740μs 51.1717μs 19.5421 KOps/s 19.2862 KOps/s $\color{#35bf28}+1.33\%$
test_values_nested_leaf 0.1016ms 46.6211μs 21.4495 KOps/s 21.2542 KOps/s $\color{#35bf28}+0.92\%$
test_values_stack_nested 0.1098ms 51.9489μs 19.2497 KOps/s 18.9745 KOps/s $\color{#35bf28}+1.45\%$
test_values_stack_nested_leaf 0.1009ms 46.8902μs 21.3264 KOps/s 21.4707 KOps/s $\color{#d91a1a}-0.67\%$
test_values_stack_nested_locked 99.1450μs 51.5552μs 19.3967 KOps/s 18.9637 KOps/s $\color{#35bf28}+2.28\%$
test_membership 28.0230μs 1.3591μs 735.8007 KOps/s 730.8762 KOps/s $\color{#35bf28}+0.67\%$
test_membership_nested 23.6040μs 3.4437μs 290.3884 KOps/s 294.0731 KOps/s $\color{#d91a1a}-1.25\%$
test_membership_nested_leaf 42.5300μs 3.4620μs 288.8494 KOps/s 290.8492 KOps/s $\color{#d91a1a}-0.69\%$
test_membership_stacked_nested 42.8300μs 3.4762μs 287.6686 KOps/s 293.5747 KOps/s $\color{#d91a1a}-2.01\%$
test_membership_stacked_nested_leaf 43.2410μs 3.4473μs 290.0802 KOps/s 210.1209 KOps/s $\textbf{\color{#35bf28}+38.05\%}$
test_membership_nested_last 28.9840μs 4.2288μs 236.4739 KOps/s 236.2273 KOps/s $\color{#35bf28}+0.10\%$
test_membership_nested_leaf_last 45.5650μs 4.2243μs 236.7231 KOps/s 235.9463 KOps/s $\color{#35bf28}+0.33\%$
test_membership_stacked_nested_last 28.8840μs 4.2567μs 234.9223 KOps/s 206.6886 KOps/s $\textbf{\color{#35bf28}+13.66\%}$
test_membership_stacked_nested_leaf_last 28.4040μs 4.2366μs 236.0408 KOps/s 210.0775 KOps/s $\textbf{\color{#35bf28}+12.36\%}$
test_nested_getleaf 55.6640μs 10.7552μs 92.9783 KOps/s 91.7279 KOps/s $\color{#35bf28}+1.36\%$
test_nested_get 53.2490μs 10.1671μs 98.3563 KOps/s 97.3832 KOps/s $\color{#35bf28}+1.00\%$
test_stacked_getleaf 47.8190μs 10.7727μs 92.8275 KOps/s 93.1925 KOps/s $\color{#d91a1a}-0.39\%$
test_stacked_get 34.8960μs 10.1757μs 98.2731 KOps/s 97.8411 KOps/s $\color{#35bf28}+0.44\%$
test_nested_getitemleaf 55.5940μs 11.2495μs 88.8932 KOps/s 88.3726 KOps/s $\color{#35bf28}+0.59\%$
test_nested_getitem 39.6940μs 10.3167μs 96.9299 KOps/s 94.8422 KOps/s $\color{#35bf28}+2.20\%$
test_stacked_getitemleaf 39.4140μs 11.2693μs 88.7364 KOps/s 88.0173 KOps/s $\color{#35bf28}+0.82\%$
test_stacked_getitem 54.8820μs 10.3975μs 96.1767 KOps/s 95.8419 KOps/s $\color{#35bf28}+0.35\%$
test_lock_nested 51.3883ms 0.4013ms 2.4922 KOps/s 2.8282 KOps/s $\textbf{\color{#d91a1a}-11.88\%}$
test_lock_stack_nested 0.5938ms 0.3126ms 3.1993 KOps/s 3.2483 KOps/s $\color{#d91a1a}-1.51\%$
test_unlock_nested 0.7565ms 0.3507ms 2.8518 KOps/s 2.5228 KOps/s $\textbf{\color{#35bf28}+13.04\%}$
test_unlock_stack_nested 0.3896ms 0.3188ms 3.1365 KOps/s 3.1666 KOps/s $\color{#d91a1a}-0.95\%$
test_flatten_speed 0.1905ms 96.7666μs 10.3341 KOps/s 10.4279 KOps/s $\color{#d91a1a}-0.90\%$
test_unflatten_speed 0.6073ms 0.4183ms 2.3908 KOps/s 2.3767 KOps/s $\color{#35bf28}+0.59\%$
test_common_ops 4.5921ms 0.6680ms 1.4970 KOps/s 1.3583 KOps/s $\textbf{\color{#35bf28}+10.21\%}$
test_creation 23.2330μs 1.9247μs 519.5525 KOps/s 512.0625 KOps/s $\color{#35bf28}+1.46\%$
test_creation_empty 24.1350μs 8.2534μs 121.1623 KOps/s 82.8480 KOps/s $\textbf{\color{#35bf28}+46.25\%}$
test_creation_nested_1 58.7600μs 10.9579μs 91.2583 KOps/s 68.0922 KOps/s $\textbf{\color{#35bf28}+34.02\%}$
test_creation_nested_2 34.1730μs 14.2625μs 70.1139 KOps/s 54.2408 KOps/s $\textbf{\color{#35bf28}+29.26\%}$
test_clone 83.8870μs 13.3835μs 74.7187 KOps/s 74.3451 KOps/s $\color{#35bf28}+0.50\%$
test_getitem[int] 28.2330μs 11.5352μs 86.6908 KOps/s 86.2960 KOps/s $\color{#35bf28}+0.46\%$
test_getitem[slice_int] 0.5576ms 23.4440μs 42.6548 KOps/s 44.1971 KOps/s $\color{#d91a1a}-3.49\%$
test_getitem[range] 79.6180μs 59.2382μs 16.8810 KOps/s 16.7415 KOps/s $\color{#35bf28}+0.83\%$
test_getitem[tuple] 55.9750μs 19.1345μs 52.2616 KOps/s 52.6271 KOps/s $\color{#d91a1a}-0.69\%$
test_getitem[list] 0.1215ms 41.5111μs 24.0900 KOps/s 24.0046 KOps/s $\color{#35bf28}+0.36\%$
test_setitem_dim[int] 93.2150μs 30.3073μs 32.9953 KOps/s 28.4951 KOps/s $\textbf{\color{#35bf28}+15.79\%}$
test_setitem_dim[slice_int] 0.1108ms 56.9490μs 17.5596 KOps/s 16.3450 KOps/s $\textbf{\color{#35bf28}+7.43\%}$
test_setitem_dim[range] 0.1703ms 79.9530μs 12.5074 KOps/s 11.7595 KOps/s $\textbf{\color{#35bf28}+6.36\%}$
test_setitem_dim[tuple] 72.1550μs 45.6491μs 21.9063 KOps/s 19.9809 KOps/s $\textbf{\color{#35bf28}+9.64\%}$
test_setitem 57.2770μs 18.7627μs 53.2972 KOps/s 47.4854 KOps/s $\textbf{\color{#35bf28}+12.24\%}$
test_set 75.5320μs 18.1026μs 55.2406 KOps/s 48.5890 KOps/s $\textbf{\color{#35bf28}+13.69\%}$
test_set_shared 1.5796ms 0.1451ms 6.8911 KOps/s 6.9071 KOps/s $\color{#d91a1a}-0.23\%$
test_update 0.1417ms 18.9300μs 52.8262 KOps/s 43.1621 KOps/s $\textbf{\color{#35bf28}+22.39\%}$
test_update_nested 88.5960μs 27.0309μs 36.9947 KOps/s 32.1003 KOps/s $\textbf{\color{#35bf28}+15.25\%}$
test_update__nested 0.1140ms 25.2173μs 39.6553 KOps/s 38.8515 KOps/s $\color{#35bf28}+2.07\%$
test_set_nested 79.8700μs 20.1206μs 49.7003 KOps/s 44.2563 KOps/s $\textbf{\color{#35bf28}+12.30\%}$
test_set_nested_new 67.9470μs 24.0792μs 41.5297 KOps/s 37.7138 KOps/s $\textbf{\color{#35bf28}+10.12\%}$
test_select 0.1036ms 39.2800μs 25.4583 KOps/s 23.8651 KOps/s $\textbf{\color{#35bf28}+6.68\%}$
test_select_nested 0.1268ms 61.5803μs 16.2390 KOps/s 16.2902 KOps/s $\color{#d91a1a}-0.31\%$
test_exclude_nested 0.2287ms 0.1212ms 8.2532 KOps/s 8.2253 KOps/s $\color{#35bf28}+0.34\%$
test_empty[True] 0.5403ms 0.3989ms 2.5066 KOps/s 2.5056 KOps/s $\color{#35bf28}+0.04\%$
test_empty[False] 10.4848μs 1.1090μs 901.6979 KOps/s 918.7344 KOps/s $\color{#d91a1a}-1.85\%$
test_unbind_speed 0.3289ms 0.2634ms 3.7960 KOps/s 3.7813 KOps/s $\color{#35bf28}+0.39\%$
test_unbind_speed_stack0 0.3799ms 0.2590ms 3.8609 KOps/s 3.8941 KOps/s $\color{#d91a1a}-0.85\%$
test_unbind_speed_stack1 66.4679ms 0.7384ms 1.3542 KOps/s 1.3020 KOps/s $\color{#35bf28}+4.01\%$
test_split 70.4649ms 1.6257ms 615.1107 Ops/s 614.9678 Ops/s $\color{#35bf28}+0.02\%$
test_chunk 65.5940ms 1.6150ms 619.2106 Ops/s 613.0644 Ops/s $\color{#35bf28}+1.00\%$
test_creation[device0] 3.2857ms 0.1073ms 9.3170 KOps/s 9.4981 KOps/s $\color{#d91a1a}-1.91\%$
test_creation_from_tensor 0.1599ms 84.0876μs 11.8924 KOps/s 12.1817 KOps/s $\color{#d91a1a}-2.38\%$
test_add_one[memmap_tensor0] 78.7980μs 5.4080μs 184.9125 KOps/s 175.5628 KOps/s $\textbf{\color{#35bf28}+5.33\%}$
test_contiguous[memmap_tensor0] 22.4120μs 0.6538μs 1.5296 MOps/s 1.5687 MOps/s $\color{#d91a1a}-2.49\%$
test_stack[memmap_tensor0] 32.9420μs 3.6499μs 273.9798 KOps/s 269.4000 KOps/s $\color{#35bf28}+1.70\%$
test_memmaptd_index 1.0122ms 0.2377ms 4.2070 KOps/s 4.1725 KOps/s $\color{#35bf28}+0.83\%$
test_memmaptd_index_astensor 0.8063ms 0.3114ms 3.2114 KOps/s 3.1647 KOps/s $\color{#35bf28}+1.48\%$
test_memmaptd_index_op 0.8931ms 0.5479ms 1.8252 KOps/s 1.6021 KOps/s $\textbf{\color{#35bf28}+13.93\%}$
test_serialize_model 0.1724s 0.1118s 8.9473 Ops/s 8.4849 Ops/s $\textbf{\color{#35bf28}+5.45\%}$
test_serialize_model_pickle 0.4669s 0.3794s 2.6356 Ops/s 2.6171 Ops/s $\color{#35bf28}+0.70\%$
test_serialize_weights 0.1629s 0.1066s 9.3799 Ops/s 9.0268 Ops/s $\color{#35bf28}+3.91\%$
test_serialize_weights_returnearly 0.1344s 0.1241s 8.0569 Ops/s 7.4183 Ops/s $\textbf{\color{#35bf28}+8.61\%}$
test_serialize_weights_pickle 1.1014s 0.5762s 1.7355 Ops/s 2.3762 Ops/s $\textbf{\color{#d91a1a}-26.96\%}$
test_serialize_weights_filesystem 92.5066ms 89.6492ms 11.1546 Ops/s 10.7708 Ops/s $\color{#35bf28}+3.56\%$
test_serialize_model_filesystem 98.4280ms 93.9383ms 10.6453 Ops/s 9.8373 Ops/s $\textbf{\color{#35bf28}+8.21\%}$
test_reshape_pytree 53.0490μs 25.9492μs 38.5369 KOps/s 38.7008 KOps/s $\color{#d91a1a}-0.42\%$
test_reshape_td 90.7700μs 33.5063μs 29.8452 KOps/s 29.2499 KOps/s $\color{#35bf28}+2.04\%$
test_view_pytree 75.5010μs 25.9753μs 38.4981 KOps/s 39.0544 KOps/s $\color{#d91a1a}-1.42\%$
test_view_td 89.1070μs 37.5085μs 26.6606 KOps/s 25.9938 KOps/s $\color{#35bf28}+2.57\%$
test_unbind_pytree 91.6820μs 30.2013μs 33.1112 KOps/s 33.4720 KOps/s $\color{#d91a1a}-1.08\%$
test_unbind_td 0.3551ms 39.0007μs 25.6406 KOps/s 25.5315 KOps/s $\color{#35bf28}+0.43\%$
test_split_pytree 87.1030μs 29.5837μs 33.8024 KOps/s 33.2097 KOps/s $\color{#35bf28}+1.78\%$
test_split_td 0.4562ms 41.6238μs 24.0247 KOps/s 24.1486 KOps/s $\color{#d91a1a}-0.51\%$
test_add_pytree 77.5160μs 35.2915μs 28.3355 KOps/s 27.8221 KOps/s $\color{#35bf28}+1.85\%$
test_add_td 0.1255ms 49.5467μs 20.1830 KOps/s 17.3561 KOps/s $\textbf{\color{#35bf28}+16.29\%}$
test_distributed 0.1996ms 0.1005ms 9.9479 KOps/s 9.8242 KOps/s $\color{#35bf28}+1.26\%$
test_tdmodule 78.3860μs 16.3199μs 61.2747 KOps/s 53.2164 KOps/s $\textbf{\color{#35bf28}+15.14\%}$
test_tdmodule_dispatch 62.5470μs 32.0654μs 31.1863 KOps/s 26.7299 KOps/s $\textbf{\color{#35bf28}+16.67\%}$
test_tdseq 34.8850μs 18.7935μs 53.2099 KOps/s 46.4581 KOps/s $\textbf{\color{#35bf28}+14.53\%}$
test_tdseq_dispatch 71.4240μs 36.3180μs 27.5346 KOps/s 23.6137 KOps/s $\textbf{\color{#35bf28}+16.60\%}$
test_instantiation_functorch 1.5816ms 1.3319ms 750.7834 Ops/s 752.9221 Ops/s $\color{#d91a1a}-0.28\%$
test_instantiation_td 1.5533ms 1.0154ms 984.7929 Ops/s 978.2473 Ops/s $\color{#35bf28}+0.67\%$
test_exec_functorch 0.2571ms 0.1639ms 6.1019 KOps/s 6.1083 KOps/s $\color{#d91a1a}-0.10\%$
test_exec_functional_call 0.2688ms 0.1511ms 6.6164 KOps/s 6.4810 KOps/s $\color{#35bf28}+2.09\%$
test_exec_td 0.2283ms 0.1463ms 6.8365 KOps/s 6.6605 KOps/s $\color{#35bf28}+2.64\%$
test_exec_td_decorator 0.3612ms 0.2200ms 4.5448 KOps/s 4.4840 KOps/s $\color{#35bf28}+1.36\%$
test_vmap_mlp_speed[True-True] 0.6020ms 0.4900ms 2.0409 KOps/s 2.0162 KOps/s $\color{#35bf28}+1.23\%$
test_vmap_mlp_speed[True-False] 0.7280ms 0.4903ms 2.0397 KOps/s 2.0254 KOps/s $\color{#35bf28}+0.70\%$
test_vmap_mlp_speed[False-True] 0.5402ms 0.4048ms 2.4704 KOps/s 2.4942 KOps/s $\color{#d91a1a}-0.95\%$
test_vmap_mlp_speed[False-False] 0.6810ms 0.4069ms 2.4579 KOps/s 2.4849 KOps/s $\color{#d91a1a}-1.09\%$
test_vmap_mlp_speed_decorator[True-True] 1.1479ms 0.5685ms 1.7590 KOps/s 1.7588 KOps/s $\color{#35bf28}+0.01\%$
test_vmap_mlp_speed_decorator[True-False] 0.8355ms 0.5622ms 1.7787 KOps/s 1.7649 KOps/s $\color{#35bf28}+0.78\%$
test_vmap_mlp_speed_decorator[False-True] 0.9628ms 0.4737ms 2.1110 KOps/s 2.1688 KOps/s $\color{#d91a1a}-2.66\%$
test_vmap_mlp_speed_decorator[False-False] 0.7400ms 0.4672ms 2.1405 KOps/s 2.2045 KOps/s $\color{#d91a1a}-2.90\%$
test_to_module_speed[True] 2.1401ms 1.6831ms 594.1319 Ops/s 591.7776 Ops/s $\color{#35bf28}+0.40\%$
test_to_module_speed[False] 2.6608ms 1.6695ms 598.9841 Ops/s 529.5763 Ops/s $\textbf{\color{#35bf28}+13.11\%}$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants