osx arm fix #62

ngam · 2022-03-02T18:42:58Z

migration: OSXArm
MNT: Re-rendered with conda-build 3.21.8, conda-smithy 3.16.2, and conda-forge-pinning 2022.02.10.16.19.14
MNT: Re-rendered with conda-build 3.21.8, conda-smithy 3.17.2, and conda-forge-pinning 2022.03.02.18.03.40

Checklist

Used a personal fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

…nda-forge-pinning 2022.02.10.16.19.14

…nda-forge-pinning 2022.03.02.18.03.40

conda-forge-linter · 2022-03-02T18:43:02Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

ngam · 2022-03-02T18:43:40Z

@conda-forge-admin, please rerender

…nda-forge-pinning 2022.03.02.18.03.40

ngam · 2022-03-02T18:45:42Z

# To have conda build upload to anaconda.org automatically, use
# conda config --set anaconda_upload yes
anaconda upload \
    /Users/ngam/Repos/climlab-feedstock/miniforge3/conda-bld/osx-arm64/climlab-0.7.13-py39h3739992_0.tar.bz2
anaconda_upload is not set.  Not uploading wheels: []

INFO :: The inputs making up the hashes for the built packages are as follows:
{
  "climlab-0.7.13-py39h3739992_0": {
    "recipe": {
      "CONDA_BUILD_SYSROOT": "/opt/MacOSX11.0.sdk",
      "build_platform": "osx-64",
      "c_compiler": "clang",
      "c_compiler_version": "12",
      "channel_targets": "conda-forge main",
      "fortran_compiler": "gfortran",
      "fortran_compiler_version": "11",
      "numpy": "1.19",
      "python": "3.9.* *_cpython",
      "target_platform": "osx-arm64"
    }
  }
}

locally

ngam · 2022-03-02T18:54:03Z

@conda-forge-admin, please rerender

github-actions · 2022-03-02T18:55:36Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/climlab-feedstock/actions/runs/1924081051.

brian-rose · 2022-03-03T03:07:41Z

Hi @ngam, fantastic to see the osx_arm builds working here!

The pypy failures I've seen before. I really don't know how to begin debugging them since I don't know anything about that platform. But it's not a huge priority for me.

The cpython failures here look new. Looks like something went wrong with the Fortran builds. I'll have to dig in to that.

ngam · 2022-03-03T03:18:43Z

Hi @ngam, fantastic to see the osx_arm builds working here!

The pypy failures I've seen before. I really don't know how to begin debugging them since I don't know anything about that platform. But it's not a huge priority for me.

The cpython failures here look new. Looks like something went wrong with the Fortran builds. I'll have to dig in to that.

Sounds good. I will try to look into this locally and see where the errors with pypy are coming from. I will also try to isolate what's going on with the osx cython locally. We will hopefully get this to a good working order soon :)

ngam · 2022-03-03T03:21:17Z

The cython errors are all NameError in tests. I believe recently the compilers got upgraded so that may be it... but I am not too sure

FAILED tests/test_cam3rad.py::test_rce - NameError: name '_cam3' is not defined
FAILED tests/test_cam3rad.py::test_cam3_multidim - NameError: name '_cam3' is...
FAILED tests/test_emanuel_convection.py::test_rcm_emanuel - NameError: name '...
FAILED tests/test_rcm.py::test_convective_adjustment - NameError: name '_rrtm...
FAILED tests/test_rcm.py::test_coupled_rcm - NameError: name '_rrtmg_sw' is n...
FAILED tests/test_rrtm.py::test_rrtm_creation - NameError: name '_rrtmg_sw' i...
FAILED tests/test_rrtm.py::test_swap_component - NameError: name '_rrtmg_sw' ...
FAILED tests/test_rrtm.py::test_multidim - NameError: name '_rrtmg_lw' is not...
FAILED tests/test_rrtm.py::test_cloud - NameError: name '_rrtmg_sw' is not de...
FAILED tests/test_rrtm.py::test_fixed_insolation - NameError: name '_rrtmg_sw...
FAILED tests/test_rrtm.py::test_large_grid - NameError: name '_cam3' is not d...

ngam · 2022-03-03T03:46:44Z

@brian-rose, this is the problematic test for pypy

https://github.com/brian-rose/climlab/blob/main/climlab/tests/test_insolation.py

If you look at the error closely, I am not really sure it has anything to do with climlab, it may be a pooch error...

(pypy) ngam@t450:~/Repos/climlab-feedstock$ pytest -v -m fast --pyargs climlab.tests.test_insolation
========================================= test session starts =========================================
platform linux -- Python 3.7.12[pypy-7.3.7-final], pytest-7.0.1, pluggy-1.0.0 -- /home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/bin/python
cachedir: .pytest_cache
rootdir: /home/ngam/Repos/climlab-feedstock
collecting 0 items                                                                                    Fatal Python error: Aborted

Stack (most recent call first, approximate line numbers):
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pooch/hashes.py", line 43 in file_hash
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pooch/hashes.py", line 137 in hash_matches
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pooch/core.py", line 711 in download_action
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pooch/core.py", line 34 in retrieve
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/climlab/solar/orbital/long.py", line 14 in _get_Laskar_data
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/climlab/solar/orbital/long.py", line 1 in <module>
  File "<frozen importlib._bootstrap>", line 220 in _call_with_frames_removed
  File "<builtin>/frozen importlib._bootstrap_external", line 735 in exec_module
  File "<frozen importlib._bootstrap>", line 672 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 957 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 987 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1003 in _gcd_import
  File "<frozen importlib._bootstrap>", line 1083 in __import__
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/climlab/tests/test_insolation.py", line 1 in <module>
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/assertion/rewrite.py", line 131 in exec_module
  File "<frozen importlib._bootstrap>", line 672 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 957 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 987 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1003 in _gcd_import
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/lib-python/3/importlib/__init__.py", line 109 in import_module
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/pathlib.py", line 454 in import_path
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/python.py", line 595 in _importtestmodule
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/python.py", line 509 in _getobj
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/python.py", line 291 in obj
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/python.py", line 518 in _inject_setup_module_fixture
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/python.py", line 512 in collect
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/runner.py", line 371 in <lambda>
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/runner.py", line 317 in from_call
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/runner.py", line 370 in pytest_make_collect_report
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pluggy/_manager.py", line 77 in _hookexec
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/runner.py", line 541 in collect_one_node
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/main.py", line 816 in genitems
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/main.py", line 603 in perform_collect
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/main.py", line 331 in pytest_collection
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pluggy/_manager.py", line 77 in _hookexec
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/main.py", line 318 in _main
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/main.py", line 255 in wrap_session
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/main.py", line 314 in pytest_cmdline_main
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pluggy/_manager.py", line 77 in _hookexec
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/config/__init__.py", line 133 in main
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/site-packages/_pytest/config/__init__.py", line 181 in console_main
  File "/home/ngam/.Mambaforge-Linux-x86_64/envs/pypy/bin/pytest", line 4 in <module>
  File "<builtin>/app_main.py", line 867 in execfile
  File "<builtin>/app_main.py", line 109 in run_toplevel
  File "<builtin>/app_main.py", line 607 in run_command_line
  File "<builtin>/app_main.py", line 944 in entry_point
Aborted (core dumped)
(pypy) ngam@t450:~/Repos/climlab-feedstock$

I will go ahead and run all tests except this one to see if things go smoothly here.

conda-forge-linter · 2022-03-03T04:01:51Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.

Here's what I've got...

For recipe:

Failed to even lint the recipe, probably because of a conda-smithy bug 😢. This likely indicates a problem in your meta.yaml, though. To get a traceback to help figure out what's going on, install conda-smithy and run conda smithy recipe-lint . from the recipe directory.

conda-forge-linter · 2022-03-03T04:02:41Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

ngam · 2022-03-03T04:18:40Z

@conda-forge-admin, please rerender

🤞 this turns them all into green. Skipping that insolation test.

…nda-forge-pinning 2022.03.02.18.03.40

ngam · 2022-03-03T04:22:43Z

The cpython failures here look new. Looks like something went wrong with the Fortran builds. I'll have to dig in to that

The cython errors are all NameError in tests. I believe recently the compilers got upgraded so that may be it... but I am not too sure

The cython errors were my fault. I believe I confused it by adding gfortran in host when it wasn't needed for osx-64. The issue with osx-arm was that the build fortran compiler was actually osx-64 since it is cross-compiling and so it didn't play nice. When providing gfortran in host (host is osx-arm in the case of osx-arm), then it plays along for osx-arm.

Anyway, hopefully this fixes all of them. I am not sure how much I would bother with debugging the pypy failure. I would just skip that test for now and see if issues arise later. However, that's up to you. I suspect the issue is elsewhere (a different feedstock)

ngam · 2022-03-03T04:25:27Z

For pypy, you can also skip that whole framework if you don't think you need it. But it wouldn't hurt just letting it be out there. It seems everything works fine. If you want to test it locally, you can create a pypy env simply by mamba create -n pypy pypy and then you can install pypy-specific packages

ngam · 2022-03-03T04:34:25Z

To summarize the changes to review before merging:

adding gfortran for osx-arm in host

climlab-feedstock/recipe/meta.yaml

Line 30 in 00470df

- gfortran # [build_platform != target_platform]

skipping the one problematic test for pypy

climlab-feedstock/recipe/meta.yaml

Lines 45 to 46 in 00470df

    
           - pytest -v -m fast --pyargs climlab  # [python_impl != 'pypy'] 
        
           - pytest -v -m fast --pyargs climlab.tests.test_bandrc climlab.tests.test_cam3rad climlab.tests.test_domain2D climlab.tests.test_ebm climlab.tests.test_emanuel_convection climlab.tests.test_grey_radiation climlab.tests.test_rcm climlab.tests.test_rrtm climlab.tests.test_thermo climlab.tests.xarray_test  # [python_impl == 'pypy']

Everything else should be exactly the same. Note some misc automatic changes happen (e.g. adding impl stuff to enable the python interpreter selector, cpython vs pypy)

brian-rose · 2022-03-03T13:20:30Z

If you look at the error closely, I am not really sure it has anything to do with climlab, it may be a pooch error...

Thanks for sleuthing that out! Odd, because pooch gets called for some other tests too that are passing. I may play around with pypy locally at some point and look at it more carefully, but for now I like your workaround!

brian-rose

Fantastic. Thanks so much for this contribution @ngam!

ngam · 2022-03-03T13:32:32Z

Great! Thank YOU for your work --- it is inspiring :)

brian-rose · 2022-03-03T13:35:00Z

If you care to share your twitter handle, I'll give you a shoutout when I announce that the OSX ARM builds are available!

ngam · 2022-03-03T13:48:47Z

I am sleuth there too 😆 (just absorbing info)

Btw, I started the issue #63 to track and i think it is clearer now what's going on. I hurriedly thought it would be somewhere else. I am not that familiar with pypy and why it is doing this here, but a solution is to figure out an alternative way to call that line

seems to have worked in conda-forge/climlab-feedstock#62

regro-cf-autotick-bot and others added 3 commits February 11, 2022 00:04

migration: OSXArm

ecfe6d9

MNT: Re-rendered with conda-build 3.21.8, conda-smithy 3.16.2, and co…

a52d84b

…nda-forge-pinning 2022.02.10.16.19.14

MNT: Re-rendered with conda-build 3.21.8, conda-smithy 3.17.2, and co…

63a89aa

…nda-forge-pinning 2022.03.02.18.03.40

ngam requested review from brian-rose and jhkennedy as code owners March 2, 2022 18:42

gfortran

11b3df9

ngam mentioned this pull request Mar 2, 2022

ARM OSX Migrator #61

Merged

MNT: Re-rendered with conda-build 3.21.8, conda-smithy 3.17.2, and co…

eb65fe7

…nda-forge-pinning 2022.03.02.18.03.40

ngam changed the title ~~bot pr arm osx h522623~~ osx arm fix Mar 2, 2022

make gfortran just unix

ebaeda6

don't test test_insolation.py for pypy

66268a6

Update meta.yaml

2ef6047

ngam added 2 commits March 2, 2022 23:03

limit gfortran in host to osx-arm

be44dbe

typo

dd3ea55

MNT: Re-rendered with conda-build 3.21.8, conda-smithy 3.17.2, and co…

00470df

…nda-forge-pinning 2022.03.02.18.03.40

brian-rose approved these changes Mar 3, 2022

View reviewed changes

brian-rose merged commit 610b7c9 into conda-forge:master Mar 3, 2022

ngam deleted the bot-pr_arm_osx_h522623 branch March 3, 2022 13:32

minrk added a commit to minrk/mumps-feedstock that referenced this pull request Dec 12, 2023

maybe need gfortran, not just libgfortran

bd5dab5

seems to have worked in conda-forge/climlab-feedstock#62

minrk mentioned this pull request Dec 12, 2023

5.6.2 conda-forge/mumps-feedstock#99

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

osx arm fix #62

osx arm fix #62

ngam commented Mar 2, 2022

conda-forge-linter commented Mar 2, 2022

ngam commented Mar 2, 2022

ngam commented Mar 2, 2022

ngam commented Mar 2, 2022

github-actions bot commented Mar 2, 2022

brian-rose commented Mar 3, 2022

ngam commented Mar 3, 2022

ngam commented Mar 3, 2022

ngam commented Mar 3, 2022

conda-forge-linter commented Mar 3, 2022

conda-forge-linter commented Mar 3, 2022

ngam commented Mar 3, 2022

ngam commented Mar 3, 2022 •

edited

Loading

ngam commented Mar 3, 2022

ngam commented Mar 3, 2022

brian-rose commented Mar 3, 2022

brian-rose left a comment

ngam commented Mar 3, 2022

brian-rose commented Mar 3, 2022

ngam commented Mar 3, 2022

osx arm fix #62

osx arm fix #62

Conversation

ngam commented Mar 2, 2022

conda-forge-linter commented Mar 2, 2022

ngam commented Mar 2, 2022

ngam commented Mar 2, 2022

ngam commented Mar 2, 2022

github-actions bot commented Mar 2, 2022

brian-rose commented Mar 3, 2022

ngam commented Mar 3, 2022

ngam commented Mar 3, 2022

ngam commented Mar 3, 2022

conda-forge-linter commented Mar 3, 2022

conda-forge-linter commented Mar 3, 2022

ngam commented Mar 3, 2022

ngam commented Mar 3, 2022 • edited Loading

ngam commented Mar 3, 2022

ngam commented Mar 3, 2022

brian-rose commented Mar 3, 2022

brian-rose left a comment

Choose a reason for hiding this comment

ngam commented Mar 3, 2022

brian-rose commented Mar 3, 2022

ngam commented Mar 3, 2022

ngam commented Mar 3, 2022 •

edited

Loading