Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Upgrade macos runner image #785

Merged
merged 3 commits into from
Aug 12, 2024
Merged

[CI] Upgrade macos runner image #785

merged 3 commits into from
Aug 12, 2024

Conversation

mshr-h
Copy link
Collaborator

@mshr-h mshr-h commented Jul 25, 2024

  • macOS runner 12->14

@mshr-h mshr-h force-pushed the upgrade-runner-image branch 2 times, most recently from 0916841 to 9dd3265 Compare July 25, 2024 10:12
@mshr-h
Copy link
Collaborator Author

mshr-h commented Jul 25, 2024

Segmentation fault on macos-13 :(

https://github.com/microsoft/hummingbird/actions/runs/10092260187/job/27905556470

Fatal Python error: Segmentation fault

Thread 0x000070000b868000 (most recent call first):
tests/test_backends.py .........
File "/Users/runner/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/selectors.py", line 416 in select
File "/Users/runner/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/socketserver.py", line 232 in serve_forever
File "/Users/runner/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/threading.py", line 953 in run
File "/Users/runner/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
File "/Users/runner/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/threading.py", line 973 in _bootstrap

Thread 0x000070000a865000 (most recent call first):
File "/Users/runner/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/py4j/clientserver.py", line 58 in run
File "/Users/runner/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
File "/Users/runner/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/threading.py", line 973 in _bootstrap

Thread 0x00007ff850e457c0 (most recent call first):
File "/Users/runner/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/functools.py", line 63 in update_wrapper
File "/Users/runner/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/onnx/symbolic_helper.py", line 308 in decorator
File "/Users/runner/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/onnx/symbolic_opset13.py", line 339 in
File "/Users/runner/work/_temp/5491f745-7b5e-4065-a0ea-547c39b8115a.sh: line 1: 15671 Segmentation fault: 11 pytest

@mshr-h
Copy link
Collaborator Author

mshr-h commented Jul 25, 2024

With macos-14, we hit another error.
https://github.com/microsoft/hummingbird/actions/runs/10093420039/job/27909084340

Do we have to be more tolerant? :(

_______ TestSklearnMatrixDecomposition.test_kernel_pca_converter_linear ________

self = <test_sklearn_decomposition.TestSklearnMatrixDecomposition testMethod=test_kernel_pca_converter_linear>

def test_kernel_pca_converter_linear(self):
  self._fit_model_pca(KernelPCA(n_components=5, kernel="linear"))

tests/test_sklearn_decomposition.py:79:


tests/test_sklearn_decomposition.py:35: in _fit_model_pca
np.testing.assert_allclose(model.transform(X_test), torch_model.transform(X_test), rtol=1e-6, atol=2 * 1e-5)


args = (<function assert_allclose..compare at 0x13e9743a0>, array([[ 17.46886258, 8.0934041 , 12.31343715, -5.393... -11.795859 ],
[-14.634164 , 15.9322605 , 0.75495166, 2.1367702 ,
12.430414 ]], dtype=float32))
kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=1e-06, atol=2e-05', 'verbose': True}

@wraps(func)
def inner(*args, **kwds):
    with self._recreate_cm():
      return func(*args, **kwds)

E AssertionError:
E Not equal to tolerance rtol=1e-06, atol=2e-05
E
E Mismatched elements: 1 / 1800 (0.0556%)
E Max absolute difference: 4.51823837e-05
E Max relative difference: 1.33559948e-05
E x: array([[ 17.468863, 8.093404, 12.313437, -5.393746, -0.939549],
E [-13.784834, 7.6482 , -7.446306, 3.170859, 7.215178],
E [-20.562256, 2.393334, 9.65356 , -7.913929, 4.56979 ],...
E y: array([[ 17.468843, 8.093403, 12.313433, -5.393748, -0.939549],
E [-13.784828, 7.648199, -7.446309, 3.170859, 7.215181],
E [-20.562273, 2.393334, 9.653566, -7.913927, 4.569791],...

@mshr-h mshr-h changed the title [CI] Upgrade ubuntu and macos runner image [CI] Upgrade macos runner image Jul 26, 2024
@mshr-h mshr-h force-pushed the upgrade-runner-image branch 5 times, most recently from 35a003b to 5db4155 Compare August 10, 2024 07:24
@mshr-h mshr-h force-pushed the upgrade-runner-image branch 2 times, most recently from 4de436d to 2946d8b Compare August 10, 2024 09:06
@mshr-h
Copy link
Collaborator Author

mshr-h commented Aug 10, 2024

All tests have passed. But I changed atol and rtol in tests/test_sklearn_decomposition.py. I don't know if it's ok.
@ksaur @interesaaat

@mshr-h mshr-h marked this pull request as ready for review August 10, 2024 09:48
@ksaur
Copy link
Contributor

ksaur commented Aug 12, 2024

All tests have passed. But I changed atol and rtol in tests/test_sklearn_decomposition.py. I don't know if it's ok. @ksaur @interesaaat

I think this rtol change is ok for this one test. confirming with @interesaaat ?

@ksaur
Copy link
Contributor

ksaur commented Aug 12, 2024

In the meantime, i actually updated the runner checks this time!! 😆

Copy link
Contributor

@ksaur ksaur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for all your help!!

@ksaur ksaur merged commit 0b566c3 into main Aug 12, 2024
14 of 21 checks passed
@ksaur ksaur deleted the upgrade-runner-image branch August 12, 2024 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants