Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add aarch64, s390x, ppc64le manylinux2014 support #281

Merged
merged 17 commits into from
Dec 13, 2019

Conversation

mattip
Copy link
Collaborator

@mattip mattip commented Nov 12, 2019

No description provided.

README.rst Outdated
* `1` (see [PEP 513](https://www.python.org/dev/peps/pep-0513);
* `2010` (see [PEP
* `1` corresponding to manylinux1 (see [PEP 513](https://www.python.org/dev/peps/pep-0513);
* `2010` corresponding to manylinux2010 (see [PEP
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these changes are to better support searching for "manylinux1" and "manylinux2010"

@mattip
Copy link
Collaborator Author

mattip commented Nov 12, 2019

I only see appveyor checks, shouldn't PRs trigger travis as well?

@mattip
Copy link
Collaborator Author

mattip commented Nov 13, 2019

manylinux2014 supports and tests aarch64, and will soon get ppc64le support too. Would be nice to support that as well here.

@mattip
Copy link
Collaborator Author

mattip commented Nov 14, 2019

The suppress test is failing since i is not a valid option in bash to set? How does this pass on other platform? The functioning of set_opts is a bit obscure, what is it meant to do?

@mattip
Copy link
Collaborator Author

mattip commented Nov 18, 2019

According to the documentation, bash's set does not accept i

@hrw
Copy link

hrw commented Nov 21, 2019

aarch64 build fails because component you are building has too old 'gnu-config' files.

@mattip
Copy link
Collaborator Author

mattip commented Nov 21, 2019

it seems that is libtiff. Again (like the question about set -i on bash) - how does this pass on other platforms?

@hrw
Copy link

hrw commented Nov 21, 2019

Simple: other platforms are older than aarch64 so libtiff's copy of 'gnu-config' knows them.

Update config.{guess,sub} before running "./configure" and it will work.

@mattip
Copy link
Collaborator Author

mattip commented Nov 22, 2019

This PR is now stuck on openblas builds. NumPy is using builds for aarch64 and s390x from packages built and uploaded by @tylerjereddy to a dropbox location. I think it would make sense to copy those to the GF_LIB_URL used here.

@hrw
Copy link

hrw commented Nov 25, 2019

What is wrong with openblas CentOS packages?

@mattip
Copy link
Collaborator Author

mattip commented Nov 26, 2019

Yum is broken in the image. See the details below. I could download the openblas as used in numpy, but it is a different version. Ideally I would like to get openblas built and uploaded via https://github.com/MacPython/openblas-libs, but in order to do that this PR needs to go in first.

++(/io/./library_builders.sh:98): build_openblas(): yum install epel-release -y

There was a problem importing one of the Python modules

required to run yum. The error leading to this problem was:

   /usr/local/lib/liblzma.so.5: version `XZ_5.1.2alpha' not found (required by /lib64/librpmio.so.3)

Please install a package which provides this module, or

verify that the module is installed correctly.

It's possible that the above module doesn't match the

current version of Python, which is:

2.7.5 (default, Aug  7 2019, 00:57:09) 

[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]

If you cannot solve this problem yourself, please go to 

the yum faq at:

  http://yum.baseurl.org/wiki/Faq

@hrw
Copy link

hrw commented Nov 26, 2019

Remove /usr/local/lib from LD_LIBRARY_PATH when you call yum?

I do not get why you insist on rebuilding most of libraries by hand instead of using distro ones - would make it easier.

@mattip
Copy link
Collaborator Author

mattip commented Nov 27, 2019

@hrw for NumPy we would like to use OpenBLAS 3.7, which is not available in the distros.

@hrw
Copy link

hrw commented Dec 3, 2019

https://github.com/hrw/multibuild/tree/hrw-devel has gnu-config change.

for me openblas build fails due to fetching not existing file

@hrw
Copy link

hrw commented Dec 3, 2019

https://travis-ci.org/hrw/multibuild/jobs/620353346 is travis job for aarch64

@mattip mattip changed the title try adding a manylinux2014 CI run on aarch64 try adding a manylinux2014 CI run on other architectures Dec 4, 2019
@mattip
Copy link
Collaborator Author

mattip commented Dec 4, 2019

I was so encouraged by 6989374 passing tests that I tried adding s390x and ppc64le as well, so this still should not be merged

@hrw
Copy link

hrw commented Dec 4, 2019

Looks like ppc64le needs PLAT.

@hrw
Copy link

hrw commented Dec 4, 2019

ppc64le wants config.{guess,sub} update for yaml ;(

@mattip
Copy link
Collaborator Author

mattip commented Dec 4, 2019

libyaml was 0.1.5, latest is 0.2.2. The Major release announcement did not mention any API changes, so I will upgrade

@mattip
Copy link
Collaborator Author

mattip commented Dec 4, 2019

Now I think this is ready: the s390x, ppc64le, aarch64 builds passed in isolation. I restored the other CI test runs (lots of macOS there) and updated the README

@mattip
Copy link
Collaborator Author

mattip commented Dec 5, 2019

The only failing build is a macOS one with the message

warning: The i386 architecture is deprecated for macOS (remove from the Xcode build setting: ARCHS)

Let's leave that for a different "macOS test cleanup" PR (which I have no idea how to do since I know nothing about macOS).

@mattip
Copy link
Collaborator Author

mattip commented Dec 7, 2019

@matthew-brett any thoughts?

@matthew-brett
Copy link
Collaborator

Seems reasonable to me. Any other takers for review? @radarhere ?

@matthew-brett
Copy link
Collaborator

I guess the test failure here is because lcms2-2.9 has dropped support for dual arch build or 32-bit build, on MacOS. I wonder if this will get fixed by the just-merged default-to-macos-10.9 PR - #279

Co-Authored-By: Andrew Murray <3112309+radarhere@users.noreply.github.com>
@mattip mattip changed the title try adding a manylinux2014 CI run on other architectures add aarch64, s390x, ppc64le manylinux2014 CI runs Dec 11, 2019
@mattip mattip changed the title add aarch64, s390x, ppc64le manylinux2014 CI runs add aarch64, s390x, ppc64le manylinux2014 support Dec 11, 2019
matthew-brett added a commit that referenced this pull request Dec 11, 2019
MRG: update README, mainly for macOS

Mainly consolidating changes from #280 and #281
@mattip
Copy link
Collaborator Author

mattip commented Dec 13, 2019

All builds are green.

@matthew-brett
Copy link
Collaborator

Let's give it a shot ...

@matthew-brett matthew-brett merged commit e004942 into multi-build:devel Dec 13, 2019
@robbuckley
Copy link
Contributor

robbuckley commented Jan 6, 2020

I guess the test failure here is because lcms2-2.9 has dropped support for dual arch build or 32-bit build, on MacOS. I wonder if this will get fixed by the just-merged default-to-macos-10.9 PR - #279

@matthew-brett yes, this fail seems to have been due to the policy for macOS builds prior to #279 when it would default to building for 10.6 / dual arch for everything except MB_PYTHON_VER=3.8

see https://travis-ci.org/matthew-brett/multibuild/jobs/620819251#L389

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants