Update PyPy to 7.3.5 and use PyPA's manylinux images with PyPy #671

YannickJadoul · 2021-05-12T14:41:52Z

This also includes the leftover Windows changes from #629 and #666.

YannickJadoul · 2021-05-16T22:28:43Z

@mattip, FYI, 7.3.5rc2 seems to pass all our tests, also on Windows :-)

mayeut · 2021-05-23T18:34:52Z

@YannickJadoul, with PyPy 7.3.5 released and integrated in manylinux images, I've pushed a few commits and will push a bit more on this branch.

YannickJadoul · 2021-05-23T19:29:54Z

Great news, @mayeut! Let me quickly rebase this onto master (with #685).

YannickJadoul · 2021-05-23T19:31:13Z

(I hope this isn't going to cause any issues for you. If so, feel free to force-push whatever it is you currently have!)

for more information, see https://pre-commit.ci

Update bin/update_docker.py Update docker images to get PyPy 3.7 v7.3.5

mayeut · 2021-05-23T21:49:28Z

rebased onto master (with #689)

mayeut · 2021-05-24T10:06:33Z

@mattip, FYI, 7.3.5 seems to pass all our tests (including the new manylinux images with x86_64/i686/aarch64)

mattip · 2021-05-24T11:07:51Z

Whew! Do the windows wheels look normal?

mayeut · 2021-05-24T11:15:05Z

Do the windows wheels look normal?

@mattip, What do you mean by normal ?

They can be built with the proper tag, they can be installed and they can be used.

mayeut · 2021-05-24T11:17:05Z

I think this is now ready for review.

mayeut · 2021-05-24T11:22:29Z

README.md

@@ -110,7 +110,7 @@ Options
 |   | [`CIBW_BEFORE_ALL`](https://cibuildwheel.readthedocs.io/en/stable/options/#before-all)  | Execute a shell command on the build system before any wheels are built. |
 |   | [`CIBW_BEFORE_BUILD`](https://cibuildwheel.readthedocs.io/en/stable/options/#before-build)  | Execute a shell command preparing each wheel's build |
 |   | [`CIBW_REPAIR_WHEEL_COMMAND`](https://cibuildwheel.readthedocs.io/en/stable/options/#repair-wheel-command)  | Execute a shell command to repair each (non-pure Python) built wheel |
-|   | [`CIBW_MANYLINUX_X86_64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_I686_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_PYPY_X86_64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_AARCH64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_PPC64LE_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_S390X_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  | Specify alternative manylinux docker images |
+|   | [`CIBW_MANYLINUX_X86_64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_I686_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_PYPY_X86_64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_AARCH64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_PPC64LE_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_S390X_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_PYPY_AARCH64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  <br> [`CIBW_MANYLINUX_PYPY_I686_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)  | Specify alternative manylinux docker images |


I will do a single comment here that might apply throughout the remaining of the changes.

Should we remove the CIBW_MANYLINUX_PYPY_* environment variables (adding a deprecation warning for CIBW_MANYLINUX_PYPY_X86_64_IMAGE ) ?

I went with "we should keep it" and thus "them" for consistency's sake.

The only thing remaining would thus be how to best order those options in docs (either group by CPython/PyPy or by arch ? The current layout is by age).

I was thinking something similar, but seems you beat me :-) If manylinux now "compresses" the two into one, then shouldn't we do so as well?

The more I think about it, the more I think we should do this, if we can deprecate CIBW_MANYLINUX_PYPY_X86_64_IMAGE. Extra argument: the PyPy manylinux images also include everything from manylinux itself, as a base. So if someone needs to use these old ones, it should still be possible?

About deprecating CIBW_MANYLINUX_PYPY_X86_64_IMAGE: this is slightly more tricky. Before, when we did so, we basically added it to the new setting. But now, if both CIBW_MANYLINUX_X86_64_IMAGE and CIBW_MANYLINUX_PYPY_X86_64_IMAGE are set, it's not clear what the new value of CIBW_MANYLINUX_X86_64_IMAGE should be?

Given that I'm not expecting a lot of users to actually set CIBW_MANYLINUX_PYPY_X86_64_IMAGE, would it be possible to just raise an error and break the config?

The only thing remaining would thus be how to best order those options in docs (either group by CPython/PyPy or by arch ? The current layout is by age).

That's the other thing I noticed, yes. If we would keep them, I think maybe just lsting all PyPy things at the end in the same order as the default ones would be best, as we also e.g. do this for the Python versions.

About deprecating CIBW_MANYLINUX_PYPY_X86_64_IMAGE: this is slightly more tricky. Before, when we did so, we basically added it to the new setting. But now, if both CIBW_MANYLINUX_X86_64_IMAGE and CIBW_MANYLINUX_PYPY_X86_64_IMAGE are set, it's not clear what the new value of CIBW_MANYLINUX_X86_64_IMAGE should be?

Given that I'm not expecting a lot of users to actually set CIBW_MANYLINUX_PYPY_X86_64_IMAGE, would it be possible to just raise an error and break the config?

You're right, I didn't fully thought this through.

One benefit of keeping a separate key for PyPy is that it allows the following config to be valid:

# Use manylinux1 for CPython because pip isn't always updated... CIBW_MANYLINUX_X86_64_IMAGE: manylinux1 CIBW_MANYLINUX_I686_IMAGE: manylinux1 # Use the default for PyPy, do not skip it

Seeing this https://foss.heptapod.net/pypy/pypy/-/issues/3425 might also prove having a separate key for PyPy to be worth it.
While I do agree it should probably be dropped sooner rather than later, the "pip situation" does not agree with that...

henryiii · 2021-05-24T21:28:36Z

This is fine to merge, right? Then we could eventually consolidate the manylinux images, but for now, it's rather handy to have both. One idea could be to leave the pypy one blank, and then use the regular one if it's blank. (But than can be a separate discussion and PR, probably after #684 - it's also slightly non-backwards compatible, so a good idea to decide before 2.0)

mayeut · 2021-05-25T16:03:59Z

Let's merge this as proposed by @henryiii, I will add an item to #657 for the remaining tasks that should be done before 2.0

…695) * fix: `bin/update_pythons.py` to work properly with PyPy `win_amd64` * refactor: use the same pattern for all python updates * follow up to #671, for the record.

henryiii marked this pull request as draft May 15, 2021 03:41

YannickJadoul force-pushed the pypy-7.3.5 branch from b46f309 to efbfded Compare May 16, 2021 21:40

YannickJadoul force-pushed the pypy-7.3.5 branch from 6a51ad9 to 7084aed Compare May 21, 2021 22:29

mayeut mentioned this pull request May 23, 2021

Add pypy3.7 7.3.5 pypa/manylinux#1103

Merged

YannickJadoul force-pushed the pypy-7.3.5 branch from bb5bf8d to dce2979 Compare May 23, 2021 19:30

YannickJadoul changed the title ~~Try out PyPy 7.3.5 release candidates~~ Update PyPy to 7.3.5 and use PyPA's manylinux images with PyPy May 23, 2021

YannickJadoul and others added 13 commits May 23, 2021 23:46

Try out PyPy 7.3.4rc1 (dropping PyPy 3.6)

511606c

PyPy now has 64-bit Windows binaries

1ebad94

PyPy 7.3.4rc2

921a626

Fix Windows tests

6789fb5

Change overlooked architecture entry of PyPy in build-platforms.toml

ac40e31

PyPy 7.3.4 released

1a6b9d2

[pre-commit.ci] auto fixes from pre-commit.com hooks

acbfce0

for more information, see https://pre-commit.ci

Try out PyPy 7.3.5rc2

3ec1bfa

Update to PyPy 7.3.5rc3

0b0a37a

Update PyPy to 7.3.5 on macOS/Windows

d608cc5

Use quay.io/pypa/manylinux2010_x86_64 for PyPy

976d8a0

Update bin/update_docker.py Update docker images to get PyPy 3.7 v7.3.5

Fix tests linked to manylinux2010_x86_64 usage for PyPy

9abf199

Add manylinux2014 & manylinux_2_24 support for PyPy

7126c10

mayeut force-pushed the pypy-7.3.5 branch from dce2979 to 7126c10 Compare May 23, 2021 21:48

mayeut added 2 commits May 24, 2021 10:32

Add aarch64 support for PyPy

c29a701

Add i686 support for PyPy

d846654

mayeut force-pushed the pypy-7.3.5 branch from a2e2dee to d846654 Compare May 24, 2021 08:33

mayeut marked this pull request as ready for review May 24, 2021 11:16

mayeut reviewed May 24, 2021

View reviewed changes

mayeut mentioned this pull request May 24, 2021

feat: support pypa build #521

Merged

henryiii approved these changes May 24, 2021

View reviewed changes

mayeut merged commit 58b2dc3 into pypa:master May 25, 2021

mayeut mentioned this pull request May 25, 2021

Plan for 2.0 #657

Closed

12 tasks

henryiii mentioned this pull request May 25, 2021

fix: bin/update_pythons.py to work properly with PyPy win_amd64 #695

Merged

YannickJadoul deleted the pypy-7.3.5 branch May 26, 2021 13:23

lkeegan mentioned this pull request Jun 1, 2021

update python wheel docker CI setup spatial-model-editor/spatial-model-editor#556

Closed

henryiii mentioned this pull request Jun 3, 2021

manylinux-pypy settings? #711

Closed

mattip mentioned this pull request Jul 1, 2021

NumPy 1.21 requires PyPy 7.3.4 or later numpy/numpy#19383

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update PyPy to 7.3.5 and use PyPA's manylinux images with PyPy #671

Update PyPy to 7.3.5 and use PyPA's manylinux images with PyPy #671

YannickJadoul commented May 12, 2021

YannickJadoul commented May 16, 2021

mayeut commented May 23, 2021

YannickJadoul commented May 23, 2021

YannickJadoul commented May 23, 2021

mayeut commented May 23, 2021

mayeut commented May 24, 2021

mattip commented May 24, 2021

mayeut commented May 24, 2021

mayeut commented May 24, 2021

mayeut May 24, 2021

YannickJadoul May 24, 2021 •

edited

Loading

YannickJadoul May 24, 2021

YannickJadoul May 24, 2021

mayeut May 24, 2021 •

edited

Loading

henryiii commented May 24, 2021

mayeut commented May 25, 2021

Update PyPy to 7.3.5 and use PyPA's manylinux images with PyPy #671

Update PyPy to 7.3.5 and use PyPA's manylinux images with PyPy #671

Conversation

YannickJadoul commented May 12, 2021

YannickJadoul commented May 16, 2021

mayeut commented May 23, 2021

YannickJadoul commented May 23, 2021

YannickJadoul commented May 23, 2021

mayeut commented May 23, 2021

mayeut commented May 24, 2021

mattip commented May 24, 2021

mayeut commented May 24, 2021

mayeut commented May 24, 2021

mayeut May 24, 2021

Choose a reason for hiding this comment

YannickJadoul May 24, 2021 • edited Loading

Choose a reason for hiding this comment

YannickJadoul May 24, 2021

Choose a reason for hiding this comment

YannickJadoul May 24, 2021

Choose a reason for hiding this comment

mayeut May 24, 2021 • edited Loading

Choose a reason for hiding this comment

henryiii commented May 24, 2021

mayeut commented May 25, 2021

YannickJadoul May 24, 2021 •

edited

Loading

mayeut May 24, 2021 •

edited

Loading