Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Update macos image #645

Merged
merged 1 commit into from
Jan 30, 2024
Merged

[CI] Update macos image #645

merged 1 commit into from
Jan 30, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jan 30, 2024

No description provided.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 30, 2024
@vmoens vmoens added the CI label Jan 30, 2024
@vmoens vmoens merged commit c84b40e into main Jan 30, 2024
17 of 32 checks passed
@vmoens vmoens deleted the macos-update branch January 30, 2024 21:38
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 124. Improved: $\large\color{#35bf28}20$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_plain_set_nested 46.0760μs 16.2448μs 61.5583 KOps/s 58.5288 KOps/s $\textbf{\color{#35bf28}+5.18\%}$
test_plain_set_stack_nested 0.1944ms 0.1437ms 6.9605 KOps/s 6.7843 KOps/s $\color{#35bf28}+2.60\%$
test_plain_set_nested_inplace 0.2700ms 18.9244μs 52.8418 KOps/s 51.3557 KOps/s $\color{#35bf28}+2.89\%$
test_plain_set_stack_nested_inplace 0.3110ms 0.1729ms 5.7821 KOps/s 5.4819 KOps/s $\textbf{\color{#35bf28}+5.48\%}$
test_items 19.6260μs 2.4098μs 414.9665 KOps/s 407.5775 KOps/s $\color{#35bf28}+1.81\%$
test_items_nested 1.2828ms 0.2740ms 3.6499 KOps/s 3.6740 KOps/s $\color{#d91a1a}-0.66\%$
test_items_nested_locked 0.4270ms 0.2773ms 3.6057 KOps/s 3.6775 KOps/s $\color{#d91a1a}-1.95\%$
test_items_nested_leaf 0.7235ms 0.1699ms 5.8844 KOps/s 5.9368 KOps/s $\color{#d91a1a}-0.88\%$
test_items_stack_nested 2.4528ms 1.3184ms 758.4845 Ops/s 767.0515 Ops/s $\color{#d91a1a}-1.12\%$
test_items_stack_nested_leaf 1.5820ms 1.1845ms 844.2361 Ops/s 856.2081 Ops/s $\color{#d91a1a}-1.40\%$
test_items_stack_nested_locked 1.4907ms 0.8695ms 1.1500 KOps/s 1.1498 KOps/s $\color{#35bf28}+0.02\%$
test_keys 21.3300μs 3.8511μs 259.6631 KOps/s 258.0248 KOps/s $\color{#35bf28}+0.63\%$
test_keys_nested 1.6503ms 0.1472ms 6.7913 KOps/s 6.7650 KOps/s $\color{#35bf28}+0.39\%$
test_keys_nested_locked 0.2537ms 0.1491ms 6.7063 KOps/s 6.5641 KOps/s $\color{#35bf28}+2.17\%$
test_keys_nested_leaf 0.2568ms 0.1298ms 7.7038 KOps/s 7.6684 KOps/s $\color{#35bf28}+0.46\%$
test_keys_stack_nested 1.4225ms 1.2419ms 805.1991 Ops/s 800.7505 Ops/s $\color{#35bf28}+0.56\%$
test_keys_stack_nested_leaf 1.9745ms 1.2331ms 810.9576 Ops/s 787.4576 Ops/s $\color{#35bf28}+2.98\%$
test_keys_stack_nested_locked 1.1150ms 0.7853ms 1.2735 KOps/s 1.2583 KOps/s $\color{#35bf28}+1.21\%$
test_values 9.7557μs 1.1837μs 844.8339 KOps/s 862.8023 KOps/s $\color{#d91a1a}-2.08\%$
test_values_nested 0.1040ms 51.4468μs 19.4375 KOps/s 19.2410 KOps/s $\color{#35bf28}+1.02\%$
test_values_nested_locked 0.1083ms 51.5749μs 19.3893 KOps/s 18.9431 KOps/s $\color{#35bf28}+2.36\%$
test_values_nested_leaf 89.9690μs 45.9698μs 21.7534 KOps/s 21.0000 KOps/s $\color{#35bf28}+3.59\%$
test_values_stack_nested 1.9694ms 1.0078ms 992.3087 Ops/s 988.4100 Ops/s $\color{#35bf28}+0.39\%$
test_values_stack_nested_leaf 2.2470ms 1.0100ms 990.0566 Ops/s 991.4847 Ops/s $\color{#d91a1a}-0.14\%$
test_values_stack_nested_locked 0.9999ms 0.5879ms 1.7009 KOps/s 1.6938 KOps/s $\color{#35bf28}+0.42\%$
test_membership 20.2580μs 1.3242μs 755.1737 KOps/s 743.8710 KOps/s $\color{#35bf28}+1.52\%$
test_membership_nested 19.8480μs 3.4525μs 289.6477 KOps/s 290.6787 KOps/s $\color{#d91a1a}-0.35\%$
test_membership_nested_leaf 42.5400μs 3.4397μs 290.7231 KOps/s 287.7663 KOps/s $\color{#35bf28}+1.03\%$
test_membership_stacked_nested 31.9700μs 11.6545μs 85.8036 KOps/s 85.4193 KOps/s $\color{#35bf28}+0.45\%$
test_membership_stacked_nested_leaf 52.8290μs 11.5942μs 86.2498 KOps/s 82.1391 KOps/s $\textbf{\color{#35bf28}+5.00\%}$
test_membership_nested_last 27.1710μs 6.6264μs 150.9113 KOps/s 147.0964 KOps/s $\color{#35bf28}+2.59\%$
test_membership_nested_leaf_last 27.5510μs 6.4869μs 154.1562 KOps/s 149.8158 KOps/s $\color{#35bf28}+2.90\%$
test_membership_stacked_nested_last 0.3122ms 0.1734ms 5.7681 KOps/s 5.7073 KOps/s $\color{#35bf28}+1.07\%$
test_membership_stacked_nested_leaf_last 39.6850μs 13.8384μs 72.2625 KOps/s 72.4423 KOps/s $\color{#d91a1a}-0.25\%$
test_nested_getleaf 54.4820μs 10.5204μs 95.0538 KOps/s 93.1286 KOps/s $\color{#35bf28}+2.07\%$
test_nested_get 49.2720μs 10.0272μs 99.7292 KOps/s 96.8610 KOps/s $\color{#35bf28}+2.96\%$
test_stacked_getleaf 0.6937ms 0.3989ms 2.5070 KOps/s 2.5358 KOps/s $\color{#d91a1a}-1.14\%$
test_stacked_get 0.6448ms 0.3629ms 2.7558 KOps/s 2.7079 KOps/s $\color{#35bf28}+1.77\%$
test_nested_getitemleaf 61.0240μs 12.1825μs 82.0848 KOps/s 82.5229 KOps/s $\color{#d91a1a}-0.53\%$
test_nested_getitem 32.6210μs 11.4364μs 87.4397 KOps/s 86.1256 KOps/s $\color{#35bf28}+1.53\%$
test_stacked_getitemleaf 0.5894ms 0.3958ms 2.5266 KOps/s 2.4922 KOps/s $\color{#35bf28}+1.38\%$
test_stacked_getitem 0.6417ms 0.3628ms 2.7561 KOps/s 2.7205 KOps/s $\color{#35bf28}+1.31\%$
test_lock_nested 2.8461ms 0.3319ms 3.0133 KOps/s 3.0409 KOps/s $\color{#d91a1a}-0.91\%$
test_lock_stack_nested 75.1762ms 5.4518ms 183.4243 Ops/s 181.6902 Ops/s $\color{#35bf28}+0.95\%$
test_unlock_nested 69.9277ms 0.3991ms 2.5057 KOps/s 3.0182 KOps/s $\textbf{\color{#d91a1a}-16.98\%}$
test_unlock_stack_nested 82.5954ms 5.6986ms 175.4821 Ops/s 172.7505 Ops/s $\color{#35bf28}+1.58\%$
test_flatten_speed 0.6405ms 0.3617ms 2.7645 KOps/s 2.7195 KOps/s $\color{#35bf28}+1.66\%$
test_unflatten_speed 0.7768ms 0.4525ms 2.2098 KOps/s 2.1330 KOps/s $\color{#35bf28}+3.60\%$
test_common_ops 5.4291ms 0.6685ms 1.4959 KOps/s 1.4593 KOps/s $\color{#35bf28}+2.51\%$
test_creation 66.5540μs 1.8387μs 543.8567 KOps/s 523.3556 KOps/s $\color{#35bf28}+3.92\%$
test_creation_empty 31.0080μs 9.2836μs 107.7170 KOps/s 96.9330 KOps/s $\textbf{\color{#35bf28}+11.13\%}$
test_creation_nested_1 50.1140μs 11.7794μs 84.8943 KOps/s 78.0084 KOps/s $\textbf{\color{#35bf28}+8.83\%}$
test_creation_nested_2 40.6660μs 15.0792μs 66.3163 KOps/s 61.0371 KOps/s $\textbf{\color{#35bf28}+8.65\%}$
test_clone 60.0230μs 12.7081μs 78.6899 KOps/s 76.4043 KOps/s $\color{#35bf28}+2.99\%$
test_getitem[int] 26.5690μs 10.9567μs 91.2686 KOps/s 88.7347 KOps/s $\color{#35bf28}+2.86\%$
test_getitem[slice_int] 96.1810μs 22.4196μs 44.6039 KOps/s 44.6105 KOps/s $\color{#d91a1a}-0.01\%$
test_getitem[range] 0.1604ms 42.6721μs 23.4345 KOps/s 25.2140 KOps/s $\textbf{\color{#d91a1a}-7.06\%}$
test_getitem[tuple] 46.4570μs 18.3319μs 54.5497 KOps/s 54.7185 KOps/s $\color{#d91a1a}-0.31\%$
test_getitem[list] 0.1683ms 37.6102μs 26.5885 KOps/s 27.6374 KOps/s $\color{#d91a1a}-3.80\%$
test_setitem_dim[int] 59.4510μs 29.1606μs 34.2928 KOps/s 34.4958 KOps/s $\color{#d91a1a}-0.59\%$
test_setitem_dim[slice_int] 77.6560μs 54.6687μs 18.2920 KOps/s 18.2069 KOps/s $\color{#35bf28}+0.47\%$
test_setitem_dim[range] 0.1291ms 73.4921μs 13.6069 KOps/s 13.7287 KOps/s $\color{#d91a1a}-0.89\%$
test_setitem_dim[tuple] 85.3500μs 44.4536μs 22.4954 KOps/s 23.1725 KOps/s $\color{#d91a1a}-2.92\%$
test_setitem 72.0950μs 18.5002μs 54.0534 KOps/s 51.0619 KOps/s $\textbf{\color{#35bf28}+5.86\%}$
test_set 63.5790μs 17.7836μs 56.2316 KOps/s 52.8429 KOps/s $\textbf{\color{#35bf28}+6.41\%}$
test_set_shared 1.9074ms 0.1380ms 7.2458 KOps/s 7.1148 KOps/s $\color{#35bf28}+1.84\%$
test_update 89.7490μs 20.1231μs 49.6942 KOps/s 46.2287 KOps/s $\textbf{\color{#35bf28}+7.50\%}$
test_update_nested 91.7920μs 27.6544μs 36.1605 KOps/s 34.5148 KOps/s $\color{#35bf28}+4.77\%$
test_set_nested 63.6490μs 19.6255μs 50.9542 KOps/s 48.5038 KOps/s $\textbf{\color{#35bf28}+5.05\%}$
test_set_nested_new 0.1207ms 23.2383μs 43.0325 KOps/s 41.0163 KOps/s $\color{#35bf28}+4.92\%$
test_select 0.1290ms 36.3516μs 27.5091 KOps/s 27.3063 KOps/s $\color{#35bf28}+0.74\%$
test_select_nested 0.1281ms 56.7380μs 17.6249 KOps/s 17.0272 KOps/s $\color{#35bf28}+3.51\%$
test_exclude_nested 0.2980ms 0.1161ms 8.6168 KOps/s 8.6014 KOps/s $\color{#35bf28}+0.18\%$
test_empty[True] 0.7309ms 0.3993ms 2.5043 KOps/s 2.4814 KOps/s $\color{#35bf28}+0.92\%$
test_empty[False] 7.6584μs 1.0327μs 968.2941 KOps/s 917.0600 KOps/s $\textbf{\color{#35bf28}+5.59\%}$
test_unbind_speed 0.4257ms 0.2440ms 4.0987 KOps/s 3.8673 KOps/s $\textbf{\color{#35bf28}+5.98\%}$
test_unbind_speed_stack0 77.5293ms 3.2516ms 307.5435 Ops/s 284.6512 Ops/s $\textbf{\color{#35bf28}+8.04\%}$
test_unbind_speed_stack1 18.9350μs 2.0213μs 494.7375 KOps/s 503.2008 KOps/s $\color{#d91a1a}-1.68\%$
test_split 2.1909ms 1.4223ms 703.0772 Ops/s 604.7512 Ops/s $\textbf{\color{#35bf28}+16.26\%}$
test_chunk 71.8817ms 1.5233ms 656.4549 Ops/s 683.8067 Ops/s $\color{#d91a1a}-4.00\%$
test_creation[device0] 0.1665ms 0.1003ms 9.9738 KOps/s 9.9370 KOps/s $\color{#35bf28}+0.37\%$
test_creation_from_tensor 3.7357ms 82.3242μs 12.1471 KOps/s 12.1247 KOps/s $\color{#35bf28}+0.18\%$
test_add_one[memmap_tensor0] 0.2686ms 5.0919μs 196.3890 KOps/s 184.4397 KOps/s $\textbf{\color{#35bf28}+6.48\%}$
test_contiguous[memmap_tensor0] 24.7060μs 0.6473μs 1.5450 MOps/s 1.2541 MOps/s $\textbf{\color{#35bf28}+23.19\%}$
test_stack[memmap_tensor0] 53.0500μs 3.5510μs 281.6093 KOps/s 271.4833 KOps/s $\color{#35bf28}+3.73\%$
test_memmaptd_index 0.8758ms 0.2300ms 4.3470 KOps/s 4.3476 KOps/s $\color{#d91a1a}-0.01\%$
test_memmaptd_index_astensor 0.6911ms 0.2867ms 3.4881 KOps/s 3.4759 KOps/s $\color{#35bf28}+0.35\%$
test_memmaptd_index_op 1.1121ms 0.5425ms 1.8432 KOps/s 1.7443 KOps/s $\textbf{\color{#35bf28}+5.67\%}$
test_serialize_model 0.1775s 0.1093s 9.1452 Ops/s 9.8945 Ops/s $\textbf{\color{#d91a1a}-7.57\%}$
test_serialize_model_pickle 0.4510s 0.3789s 2.6390 Ops/s 2.5960 Ops/s $\color{#35bf28}+1.66\%$
test_serialize_weights 0.1731s 0.1066s 9.3786 Ops/s 9.8237 Ops/s $\color{#d91a1a}-4.53\%$
test_serialize_weights_returnearly 0.3471s 0.1467s 6.8177 Ops/s 7.8433 Ops/s $\textbf{\color{#d91a1a}-13.08\%}$
test_serialize_weights_pickle 1.0115s 0.5801s 1.7238 Ops/s 1.4657 Ops/s $\textbf{\color{#35bf28}+17.61\%}$
test_serialize_weights_filesystem 93.2570ms 89.7888ms 11.1372 Ops/s 9.9932 Ops/s $\textbf{\color{#35bf28}+11.45\%}$
test_serialize_model_filesystem 0.1609s 97.3417ms 10.2731 Ops/s 10.6591 Ops/s $\color{#d91a1a}-3.62\%$
test_reshape_pytree 55.8240μs 20.8728μs 47.9093 KOps/s 47.2179 KOps/s $\color{#35bf28}+1.46\%$
test_reshape_td 65.0120μs 29.6985μs 33.6717 KOps/s 33.1281 KOps/s $\color{#35bf28}+1.64\%$
test_view_pytree 63.6390μs 20.6910μs 48.3302 KOps/s 47.5913 KOps/s $\color{#35bf28}+1.55\%$
test_view_td 76.6975ms 11.0836μs 90.2232 KOps/s 70.9520 KOps/s $\textbf{\color{#35bf28}+27.16\%}$
test_unbind_pytree 59.5710μs 23.8840μs 41.8690 KOps/s 41.3779 KOps/s $\color{#35bf28}+1.19\%$
test_unbind_td 0.3685ms 35.0832μs 28.5037 KOps/s 28.0496 KOps/s $\color{#35bf28}+1.62\%$
test_split_pytree 61.8060μs 23.4862μs 42.5781 KOps/s 41.4954 KOps/s $\color{#35bf28}+2.61\%$
test_split_td 73.5880μs 38.9444μs 25.6776 KOps/s 25.6606 KOps/s $\color{#35bf28}+0.07\%$
test_add_pytree 64.6010μs 28.9930μs 34.4911 KOps/s 34.1481 KOps/s $\color{#35bf28}+1.00\%$
test_add_td 0.1377ms 48.4467μs 20.6412 KOps/s 20.6063 KOps/s $\color{#35bf28}+0.17\%$
test_distributed 0.7759ms 96.1551μs 10.3999 KOps/s 10.0545 KOps/s $\color{#35bf28}+3.44\%$
test_tdmodule 0.3453ms 22.0221μs 45.4090 KOps/s 43.5784 KOps/s $\color{#35bf28}+4.20\%$
test_tdmodule_dispatch 0.1897ms 42.4207μs 23.5734 KOps/s 23.0934 KOps/s $\color{#35bf28}+2.08\%$
test_tdseq 53.2890μs 24.8862μs 40.1829 KOps/s 39.0694 KOps/s $\color{#35bf28}+2.85\%$
test_tdseq_dispatch 0.1393ms 46.0949μs 21.6944 KOps/s 21.2667 KOps/s $\color{#35bf28}+2.01\%$
test_instantiation_functorch 2.0199ms 1.3039ms 766.9019 Ops/s 774.0225 Ops/s $\color{#d91a1a}-0.92\%$
test_instantiation_td 1.4938ms 1.0035ms 996.5215 Ops/s 1.0134 KOps/s $\color{#d91a1a}-1.67\%$
test_exec_functorch 0.2591ms 0.1542ms 6.4853 KOps/s 6.4446 KOps/s $\color{#35bf28}+0.63\%$
test_exec_functional_call 0.3829ms 0.1458ms 6.8564 KOps/s 6.8549 KOps/s $\color{#35bf28}+0.02\%$
test_exec_td 0.2071ms 0.1395ms 7.1668 KOps/s 7.0078 KOps/s $\color{#35bf28}+2.27\%$
test_exec_td_decorator 0.7418ms 0.1742ms 5.7401 KOps/s 5.6604 KOps/s $\color{#35bf28}+1.41\%$
test_vmap_mlp_speed[True-True] 1.3502ms 0.9142ms 1.0938 KOps/s 1.1338 KOps/s $\color{#d91a1a}-3.52\%$
test_vmap_mlp_speed[True-False] 0.7160ms 0.4566ms 2.1899 KOps/s 2.1502 KOps/s $\color{#35bf28}+1.85\%$
test_vmap_mlp_speed[False-True] 1.2131ms 0.7509ms 1.3318 KOps/s 1.2902 KOps/s $\color{#35bf28}+3.22\%$
test_vmap_mlp_speed[False-False] 0.5817ms 0.3773ms 2.6501 KOps/s 2.6388 KOps/s $\color{#35bf28}+0.43\%$
test_vmap_mlp_speed_decorator[True-True] 3.0731ms 2.2531ms 443.8244 Ops/s 435.4224 Ops/s $\color{#35bf28}+1.93\%$
test_vmap_mlp_speed_decorator[True-False] 0.8953ms 0.5079ms 1.9687 KOps/s 1.9231 KOps/s $\color{#35bf28}+2.37\%$
test_vmap_mlp_speed_decorator[False-True] 2.4731ms 1.8380ms 544.0602 Ops/s 533.4105 Ops/s $\color{#35bf28}+2.00\%$
test_vmap_mlp_speed_decorator[False-False] 0.7456ms 0.3951ms 2.5308 KOps/s 2.5201 KOps/s $\color{#35bf28}+0.43\%$

Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 132. Improved: $\large\color{#35bf28}15$. Worsened: $\large\color{#d91a1a}13$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_plain_set_nested 60.9915ms 18.0745μs 55.3266 KOps/s 69.4822 KOps/s $\textbf{\color{#d91a1a}-20.37\%}$
test_plain_set_stack_nested 0.1447ms 0.1198ms 8.3450 KOps/s 8.0938 KOps/s $\color{#35bf28}+3.10\%$
test_plain_set_nested_inplace 44.5810μs 14.4686μs 69.1150 KOps/s 63.7684 KOps/s $\textbf{\color{#35bf28}+8.38\%}$
test_plain_set_stack_nested_inplace 0.1990ms 0.1495ms 6.6905 KOps/s 6.6909 KOps/s $-0.01\%$
test_items 20.0010μs 4.8730μs 205.2122 KOps/s 209.4387 KOps/s $\color{#d91a1a}-2.02\%$
test_items_nested 0.3866ms 0.3422ms 2.9220 KOps/s 2.9238 KOps/s $\color{#d91a1a}-0.06\%$
test_items_nested_locked 0.4012ms 0.3464ms 2.8867 KOps/s 2.9072 KOps/s $\color{#d91a1a}-0.71\%$
test_items_nested_leaf 0.2402ms 0.2020ms 4.9501 KOps/s 4.9217 KOps/s $\color{#35bf28}+0.58\%$
test_items_stack_nested 1.5735ms 1.3415ms 745.4613 Ops/s 756.6302 Ops/s $\color{#d91a1a}-1.48\%$
test_items_stack_nested_leaf 1.3882ms 1.1787ms 848.3911 Ops/s 867.0355 Ops/s $\color{#d91a1a}-2.15\%$
test_items_stack_nested_locked 1.9733ms 0.9143ms 1.0937 KOps/s 1.1178 KOps/s $\color{#d91a1a}-2.16\%$
test_keys 25.4710μs 4.5808μs 218.3038 KOps/s 216.5556 KOps/s $\color{#35bf28}+0.81\%$
test_keys_nested 0.5831ms 95.2760μs 10.4958 KOps/s 10.4554 KOps/s $\color{#35bf28}+0.39\%$
test_keys_nested_locked 0.1270ms 98.7255μs 10.1291 KOps/s 10.1067 KOps/s $\color{#35bf28}+0.22\%$
test_keys_nested_leaf 0.1806ms 78.9106μs 12.6726 KOps/s 12.5596 KOps/s $\color{#35bf28}+0.90\%$
test_keys_stack_nested 1.2433ms 1.1749ms 851.1311 Ops/s 860.9476 Ops/s $\color{#d91a1a}-1.14\%$
test_keys_stack_nested_leaf 1.3768ms 1.1733ms 852.2992 Ops/s 882.0641 Ops/s $\color{#d91a1a}-3.37\%$
test_keys_stack_nested_locked 0.9132ms 0.7388ms 1.3536 KOps/s 1.3965 KOps/s $\color{#d91a1a}-3.08\%$
test_values 7.2733μs 1.9130μs 522.7284 KOps/s 525.0051 KOps/s $\color{#d91a1a}-0.43\%$
test_values_nested 67.4420μs 45.6252μs 21.9177 KOps/s 22.0497 KOps/s $\color{#d91a1a}-0.60\%$
test_values_nested_locked 68.7110μs 48.1712μs 20.7593 KOps/s 21.0840 KOps/s $\color{#d91a1a}-1.54\%$
test_values_nested_leaf 62.1020μs 40.1356μs 24.9155 KOps/s 25.2941 KOps/s $\color{#d91a1a}-1.50\%$
test_values_stack_nested 1.1789ms 0.9773ms 1.0232 KOps/s 1.0308 KOps/s $\color{#d91a1a}-0.74\%$
test_values_stack_nested_leaf 1.1034ms 0.9870ms 1.0132 KOps/s 1.0399 KOps/s $\color{#d91a1a}-2.56\%$
test_values_stack_nested_locked 0.7265ms 0.5895ms 1.6963 KOps/s 1.7427 KOps/s $\color{#d91a1a}-2.66\%$
test_membership 5.1202μs 0.9426μs 1.0609 MOps/s 1.0511 MOps/s $\color{#35bf28}+0.94\%$
test_membership_nested 29.9510μs 2.9065μs 344.0604 KOps/s 341.2159 KOps/s $\color{#35bf28}+0.83\%$
test_membership_nested_leaf 21.2710μs 2.9142μs 343.1503 KOps/s 340.9103 KOps/s $\color{#35bf28}+0.66\%$
test_membership_stacked_nested 32.2010μs 11.3604μs 88.0248 KOps/s 88.7866 KOps/s $\color{#d91a1a}-0.86\%$
test_membership_stacked_nested_leaf 42.3510μs 11.3832μs 87.8489 KOps/s 89.0989 KOps/s $\color{#d91a1a}-1.40\%$
test_membership_nested_last 31.1900μs 5.3358μs 187.4137 KOps/s 187.4485 KOps/s $\color{#d91a1a}-0.02\%$
test_membership_nested_leaf_last 32.0810μs 5.3537μs 186.7871 KOps/s 187.8226 KOps/s $\color{#d91a1a}-0.55\%$
test_membership_stacked_nested_last 0.7419ms 0.1562ms 6.4018 KOps/s 6.3631 KOps/s $\color{#35bf28}+0.61\%$
test_membership_stacked_nested_leaf_last 28.4910μs 13.1414μs 76.0953 KOps/s 75.3246 KOps/s $\color{#35bf28}+1.02\%$
test_nested_getleaf 37.6610μs 8.4960μs 117.7018 KOps/s 118.7430 KOps/s $\color{#d91a1a}-0.88\%$
test_nested_get 40.8810μs 8.0119μs 124.8137 KOps/s 125.7708 KOps/s $\color{#d91a1a}-0.76\%$
test_stacked_getleaf 0.3953ms 0.3329ms 3.0039 KOps/s 3.0144 KOps/s $\color{#d91a1a}-0.35\%$
test_stacked_get 0.3620ms 0.2968ms 3.3694 KOps/s 3.3271 KOps/s $\color{#35bf28}+1.27\%$
test_nested_getitemleaf 30.7210μs 9.9356μs 100.6478 KOps/s 101.0611 KOps/s $\color{#d91a1a}-0.41\%$
test_nested_getitem 27.5710μs 9.4911μs 105.3623 KOps/s 106.0970 KOps/s $\color{#d91a1a}-0.69\%$
test_stacked_getitemleaf 0.3983ms 0.3368ms 2.9693 KOps/s 2.9832 KOps/s $\color{#d91a1a}-0.47\%$
test_stacked_getitem 0.3614ms 0.3010ms 3.3225 KOps/s 3.3018 KOps/s $\color{#35bf28}+0.63\%$
test_lock_nested 0.8534ms 0.3593ms 2.7833 KOps/s 2.7917 KOps/s $\color{#d91a1a}-0.30\%$
test_lock_stack_nested 85.5803ms 6.3658ms 157.0898 Ops/s 154.9108 Ops/s $\color{#35bf28}+1.41\%$
test_unlock_nested 80.9765ms 0.4370ms 2.2886 KOps/s 2.8015 KOps/s $\textbf{\color{#d91a1a}-18.31\%}$
test_unlock_stack_nested 85.8071ms 6.4404ms 155.2691 Ops/s 153.0509 Ops/s $\color{#35bf28}+1.45\%$
test_flatten_speed 0.6598ms 0.2631ms 3.8005 KOps/s 3.7839 KOps/s $\color{#35bf28}+0.44\%$
test_unflatten_speed 0.3952ms 0.3622ms 2.7610 KOps/s 2.7622 KOps/s $\color{#d91a1a}-0.04\%$
test_common_ops 1.0314ms 0.6016ms 1.6622 KOps/s 1.5720 KOps/s $\textbf{\color{#35bf28}+5.74\%}$
test_creation 14.4800μs 1.5604μs 640.8625 KOps/s 634.6085 KOps/s $\color{#35bf28}+0.99\%$
test_creation_empty 21.1010μs 7.1934μs 139.0167 KOps/s 103.5751 KOps/s $\textbf{\color{#35bf28}+34.22\%}$
test_creation_nested_1 43.2510μs 9.0007μs 111.1026 KOps/s 87.8939 KOps/s $\textbf{\color{#35bf28}+26.41\%}$
test_creation_nested_2 34.7800μs 11.4285μs 87.5007 KOps/s 72.3076 KOps/s $\textbf{\color{#35bf28}+21.01\%}$
test_clone 67.1410μs 14.6976μs 68.0382 KOps/s 72.4452 KOps/s $\textbf{\color{#d91a1a}-6.08\%}$
test_getitem[int] 28.8910μs 11.4574μs 87.2796 KOps/s 89.9965 KOps/s $\color{#d91a1a}-3.02\%$
test_getitem[slice_int] 38.5210μs 22.1862μs 45.0731 KOps/s 46.1286 KOps/s $\color{#d91a1a}-2.29\%$
test_getitem[range] 0.1401ms 38.4893μs 25.9812 KOps/s 25.9021 KOps/s $\color{#35bf28}+0.31\%$
test_getitem[tuple] 53.0910μs 19.8098μs 50.4800 KOps/s 50.9597 KOps/s $\color{#d91a1a}-0.94\%$
test_getitem[list] 0.1835ms 34.7584μs 28.7700 KOps/s 28.9627 KOps/s $\color{#d91a1a}-0.67\%$
test_setitem_dim[int] 47.0310μs 27.3670μs 36.5403 KOps/s 34.8229 KOps/s $\color{#35bf28}+4.93\%$
test_setitem_dim[slice_int] 88.1320μs 50.3298μs 19.8689 KOps/s 20.0688 KOps/s $\color{#d91a1a}-1.00\%$
test_setitem_dim[range] 82.0620μs 63.8015μs 15.6736 KOps/s 15.3097 KOps/s $\color{#35bf28}+2.38\%$
test_setitem_dim[tuple] 58.8410μs 43.3250μs 23.0814 KOps/s 23.2602 KOps/s $\color{#d91a1a}-0.77\%$
test_setitem 91.5720μs 19.0619μs 52.4606 KOps/s 51.1682 KOps/s $\color{#35bf28}+2.53\%$
test_set 76.5520μs 18.4356μs 54.2429 KOps/s 52.4536 KOps/s $\color{#35bf28}+3.41\%$
test_set_shared 2.8273ms 0.1041ms 9.6039 KOps/s 9.8007 KOps/s $\color{#d91a1a}-2.01\%$
test_update 0.1087ms 20.1229μs 49.6945 KOps/s 45.3587 KOps/s $\textbf{\color{#35bf28}+9.56\%}$
test_update_nested 81.9620μs 26.8755μs 37.2086 KOps/s 35.0739 KOps/s $\textbf{\color{#35bf28}+6.09\%}$
test_set_nested 76.4120μs 19.8721μs 50.3219 KOps/s 48.9734 KOps/s $\color{#35bf28}+2.75\%$
test_set_nested_new 88.4520μs 22.6602μs 44.1303 KOps/s 42.7996 KOps/s $\color{#35bf28}+3.11\%$
test_select 96.5220μs 35.3060μs 28.3238 KOps/s 27.4461 KOps/s $\color{#35bf28}+3.20\%$
test_select_nested 76.7620μs 53.0542μs 18.8487 KOps/s 18.9163 KOps/s $\color{#d91a1a}-0.36\%$
test_exclude_nested 0.1394ms 0.1136ms 8.8014 KOps/s 8.6476 KOps/s $\color{#35bf28}+1.78\%$
test_empty[True] 0.4447ms 0.3934ms 2.5417 KOps/s 2.5691 KOps/s $\color{#d91a1a}-1.07\%$
test_empty[False] 3.3281μs 0.8505μs 1.1758 MOps/s 1.1671 MOps/s $\color{#35bf28}+0.74\%$
test_to 77.2220μs 56.3114μs 17.7584 KOps/s 18.4954 KOps/s $\color{#d91a1a}-3.98\%$
test_to_nonblocking 58.2620μs 36.0957μs 27.7041 KOps/s 29.5554 KOps/s $\textbf{\color{#d91a1a}-6.26\%}$
test_unbind_speed 0.4716ms 0.2716ms 3.6821 KOps/s 3.6434 KOps/s $\color{#35bf28}+1.06\%$
test_unbind_speed_stack0 88.2492ms 3.5658ms 280.4408 Ops/s 265.9756 Ops/s $\textbf{\color{#35bf28}+5.44\%}$
test_unbind_speed_stack1 68.4250μs 1.7176μs 582.2044 KOps/s 533.4298 KOps/s $\textbf{\color{#35bf28}+9.14\%}$
test_split 81.7723ms 1.7825ms 560.9964 Ops/s 650.2830 Ops/s $\textbf{\color{#d91a1a}-13.73\%}$
test_chunk 80.6993ms 1.7197ms 581.4923 Ops/s 601.0636 Ops/s $\color{#d91a1a}-3.26\%$
test_creation[device0] 0.1415ms 72.2497μs 13.8409 KOps/s 13.8146 KOps/s $\color{#35bf28}+0.19\%$
test_creation_from_tensor 0.2838ms 57.2016μs 17.4820 KOps/s 18.6238 KOps/s $\textbf{\color{#d91a1a}-6.13\%}$
test_add_one[memmap_tensor0] 0.2086ms 7.7853μs 128.4476 KOps/s 136.7983 KOps/s $\textbf{\color{#d91a1a}-6.10\%}$
test_contiguous[memmap_tensor0] 12.0500μs 0.6513μs 1.5355 MOps/s 1.5344 MOps/s $\color{#35bf28}+0.07\%$
test_stack[memmap_tensor0] 0.2102ms 4.6667μs 214.2850 KOps/s 216.4778 KOps/s $\color{#d91a1a}-1.01\%$
test_memmaptd_index 1.0362ms 0.2806ms 3.5638 KOps/s 3.7576 KOps/s $\textbf{\color{#d91a1a}-5.16\%}$
test_memmaptd_index_astensor 0.6696ms 0.3367ms 2.9699 KOps/s 3.0878 KOps/s $\color{#d91a1a}-3.82\%$
test_memmaptd_index_op 0.9917ms 0.6494ms 1.5400 KOps/s 1.5394 KOps/s $\color{#35bf28}+0.04\%$
test_serialize_model 0.1748s 97.8244ms 10.2224 Ops/s 9.6500 Ops/s $\textbf{\color{#35bf28}+5.93\%}$
test_serialize_model_pickle 1.3693s 1.2392s 0.8070 Ops/s 0.8057 Ops/s $\color{#35bf28}+0.16\%$
test_serialize_weights 0.1739s 96.5488ms 10.3575 Ops/s 10.8012 Ops/s $\color{#d91a1a}-4.11\%$
test_serialize_weights_returnearly 0.2589s 72.8675ms 13.7235 Ops/s 13.6245 Ops/s $\color{#35bf28}+0.73\%$
test_serialize_weights_pickle 1.3500s 1.2372s 0.8083 Ops/s 0.8080 Ops/s $\color{#35bf28}+0.03\%$
test_reshape_pytree 46.8910μs 25.3322μs 39.4754 KOps/s 39.8698 KOps/s $\color{#d91a1a}-0.99\%$
test_reshape_td 55.6210μs 30.4037μs 32.8907 KOps/s 33.2872 KOps/s $\color{#d91a1a}-1.19\%$
test_view_pytree 75.0620μs 25.0731μs 39.8834 KOps/s 39.5631 KOps/s $\color{#35bf28}+0.81\%$
test_view_td 83.5233ms 9.9817μs 100.1831 KOps/s 87.3967 KOps/s $\textbf{\color{#35bf28}+14.63\%}$
test_unbind_pytree 59.4820μs 31.6072μs 31.6383 KOps/s 32.5781 KOps/s $\color{#d91a1a}-2.88\%$
test_unbind_td 0.4316ms 41.3213μs 24.2006 KOps/s 24.3452 KOps/s $\color{#d91a1a}-0.59\%$
test_split_pytree 94.5730μs 29.5011μs 33.8970 KOps/s 34.8411 KOps/s $\color{#d91a1a}-2.71\%$
test_split_td 0.1661ms 40.7302μs 24.5518 KOps/s 25.9847 KOps/s $\textbf{\color{#d91a1a}-5.51\%}$
test_add_pytree 0.1807ms 39.9979μs 25.0013 KOps/s 27.1619 KOps/s $\textbf{\color{#d91a1a}-7.95\%}$
test_add_td 0.2428ms 55.7775μs 17.9284 KOps/s 20.0004 KOps/s $\textbf{\color{#d91a1a}-10.36\%}$
test_distributed 2.5371ms 75.0604μs 13.3226 KOps/s 10.4228 KOps/s $\textbf{\color{#35bf28}+27.82\%}$
test_tdmodule 0.2008ms 18.3620μs 54.4602 KOps/s 52.8284 KOps/s $\color{#35bf28}+3.09\%$
test_tdmodule_dispatch 0.2526ms 36.0968μs 27.7033 KOps/s 25.8142 KOps/s $\textbf{\color{#35bf28}+7.32\%}$
test_tdseq 35.5700μs 20.3725μs 49.0858 KOps/s 45.6415 KOps/s $\textbf{\color{#35bf28}+7.55\%}$
test_tdseq_dispatch 55.1910μs 38.3918μs 26.0472 KOps/s 24.5265 KOps/s $\textbf{\color{#35bf28}+6.20\%}$
test_instantiation_functorch 1.7771ms 1.6986ms 588.7088 Ops/s 592.7973 Ops/s $\color{#d91a1a}-0.69\%$
test_instantiation_td 0.1135s 1.3233ms 755.6674 Ops/s 854.7971 Ops/s $\textbf{\color{#d91a1a}-11.60\%}$
test_exec_functorch 0.2235ms 0.1651ms 6.0557 KOps/s 6.1923 KOps/s $\color{#d91a1a}-2.21\%$
test_exec_functional_call 0.2579ms 0.1651ms 6.0573 KOps/s 6.1754 KOps/s $\color{#d91a1a}-1.91\%$
test_exec_td 0.1947ms 0.1589ms 6.2914 KOps/s 6.5175 KOps/s $\color{#d91a1a}-3.47\%$
test_exec_td_decorator 0.7922ms 0.1952ms 5.1240 KOps/s 5.2085 KOps/s $\color{#d91a1a}-1.62\%$
test_vmap_mlp_speed[True-True] 1.2632ms 1.0704ms 934.2459 Ops/s 941.5221 Ops/s $\color{#d91a1a}-0.77\%$
test_vmap_mlp_speed[True-False] 0.7412ms 0.6390ms 1.5650 KOps/s 1.6233 KOps/s $\color{#d91a1a}-3.59\%$
test_vmap_mlp_speed[False-True] 1.1140ms 0.9861ms 1.0141 KOps/s 1.0286 KOps/s $\color{#d91a1a}-1.42\%$
test_vmap_mlp_speed[False-False] 0.6277ms 0.5605ms 1.7843 KOps/s 1.8392 KOps/s $\color{#d91a1a}-2.98\%$
test_vmap_mlp_speed_decorator[True-True] 3.0560ms 2.3851ms 419.2680 Ops/s 418.6830 Ops/s $\color{#35bf28}+0.14\%$
test_vmap_mlp_speed_decorator[True-False] 1.1784ms 0.6800ms 1.4706 KOps/s 1.5087 KOps/s $\color{#d91a1a}-2.53\%$
test_vmap_mlp_speed_decorator[False-True] 2.4837ms 2.0125ms 496.8969 Ops/s 490.1736 Ops/s $\color{#35bf28}+1.37\%$
test_vmap_mlp_speed_decorator[False-False] 1.0621ms 0.5729ms 1.7455 KOps/s 1.7645 KOps/s $\color{#d91a1a}-1.07\%$
test_vmap_transformer_speed[True-True] 13.3188ms 12.7949ms 78.1559 Ops/s 79.9104 Ops/s $\color{#d91a1a}-2.20\%$
test_vmap_transformer_speed[True-False] 8.7509ms 8.3194ms 120.2005 Ops/s 121.7056 Ops/s $\color{#d91a1a}-1.24\%$
test_vmap_transformer_speed[False-True] 13.1616ms 12.6412ms 79.1064 Ops/s 79.5779 Ops/s $\color{#d91a1a}-0.59\%$
test_vmap_transformer_speed[False-False] 8.3660ms 8.2533ms 121.1641 Ops/s 122.3988 Ops/s $\color{#d91a1a}-1.01\%$
test_vmap_transformer_speed_decorator[True-True] 78.2163ms 75.0493ms 13.3246 Ops/s 13.4201 Ops/s $\color{#d91a1a}-0.71\%$
test_vmap_transformer_speed_decorator[True-False] 21.6027ms 19.9635ms 50.0913 Ops/s 50.3448 Ops/s $\color{#d91a1a}-0.50\%$
test_vmap_transformer_speed_decorator[False-True] 0.2029s 76.2201ms 13.1199 Ops/s 14.8608 Ops/s $\textbf{\color{#d91a1a}-11.71\%}$
test_vmap_transformer_speed_decorator[False-False] 21.7950ms 19.6077ms 51.0004 Ops/s 51.4584 Ops/s $\color{#d91a1a}-0.89\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants