How to build cp36-abi3 - py-limited-api=cp36 not handled?

[th0ma7]

Versions of

• Python: 3.10.6
• cryptography: 38.0.1
• cffi: 1.15.1
• pip: 22.2.2
• setuptools: 63.4.3

Relates to PR SynoCommunity/spksrc#5435 which in turns depends on new Docker image SynoCommunity/spksrc#5441. As such logs from github-action are not useful at this time. Info provided above is using local build.

• How you installed cryptography: cross-compiling successfully using Synology Linux toolchain.

Issue:

I'm trying to build an cp36-abi3 version of cryptography. Normally I simply use --build-option=--py-limited-api=cp36 and it build successfully, such as:

===>  _PYTHON_HOST_PLATFORM=aarch64-unknown-linux-gnu /home/spksrc/rustc2/spksrc/spk/python310/work-aarch64-7.0/crossenv/cross/bin/pip wheel --disable-pip-version-check --no-binary :all: --cache-dir /home/spksrc/rustc2/spksrc/spk/python310/../../distrib/pip --no-deps --wheel-dir /home/spksrc/rustc2/spksrc/spk/python310/work-aarch64-7.0/wheelhouse --no-use-pep517 --no-build-isolation --build-option=--py-limited-api=cp36 pycryptodomex==3.12.0
WARNING: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
Collecting pycryptodomex==3.12.0
  Using cached pycryptodomex-3.12.0.zip (3.7 MB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: pycryptodomex
  Building wheel for pycryptodomex (setup.py): started
  Building wheel for pycryptodomex (setup.py): finished with status 'done'
  Created wheel for pycryptodomex: filename=pycryptodomex-3.12.0-cp36-abi3-linux_aarch64.whl size=1855547 sha256=3b41882f81acf802a15d27d9c30c370544ca6999f69f77d90340be6c9f13736f
  Stored in directory: /home/spksrc/rustc2/spksrc/distrib/pip/wheels/a9/9a/27/08c0316c48e7db0239abb5e789d36146f3647afe3da42dc693
Successfully built pycryptodomex

But with cryptography, while it succesfully build, the output file always is cp310-cp310:

===>  _PYTHON_HOST_PLATFORM=aarch64-unknown-linux-gnu /home/spksrc/rustc2/spksrc/spk/python310/work-aarch64-7.0/crossenv/cross/bin/pip wheel --disable-pip-version-check --no-binary :all: --cache-dir /home/spksrc/rustc2/spksrc/spk/python310/../../distrib/pip --no-deps --wheel-dir /home/spksrc/rustc2/spksrc/spk/python310/work-aarch64-7.0/wheelhouse --no-build-isolation --build-option=--py-limited-api=cp36 cryptography==38.0.1
WARNING: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
Collecting cryptography==38.0.1
  Using cached cryptography-38.0.1.tar.gz (599 kB)
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: cryptography
  WARNING: Ignoring --build-option when building cryptography using PEP 517
  Building wheel for cryptography (pyproject.toml): started
  Building wheel for cryptography (pyproject.toml): finished with status 'done'
  Created wheel for cryptography: filename=cryptography-38.0.1-cp310-cp310-linux_aarch64.whl size=1810581 sha256=daad855da335a0bb74bd91706a5e63099f200941c9276e5ba5d4d42854f728f4
  Stored in directory: /home/spksrc/rustc2/spksrc/distrib/pip/wheels/38/85/c2/e0e9a4846393723fc82538d119665e1c8cd78d919c9f9001e8
Successfully built cryptography

I tried disabling PEP 517 but it then fails:

===>  _PYTHON_HOST_PLATFORM=aarch64-unknown-linux-gnu /home/spksrc/rustc2/spksrc/spk/python310/work-aarch64-7.0/crossenv/cross/bin/pip wheel --disable-pip-version-check --no-binary :all: --cache-dir /home/spksrc/rustc2/spksrc/spk/python310/../../distrib/pip --no-deps --wheel-dir /home/spksrc/rustc2/spksrc/spk/python310/work-aarch64-7.0/wheelhouse --no-use-pep517 --no-build-isolation --build-option=--py-limited-api=cp36 cryptography==38.0.1
WARNING: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
Collecting cryptography==38.0.1
  Using cached cryptography-38.0.1.tar.gz (599 kB)
ERROR: Disabling PEP 517 processing is invalid: project specifies a build backend of setuptools.build_meta in pyproject.toml

[reaperhulk]

We build using python setup.py bdist_wheel --py-limited-api=cp36 in our wheel builders for this reason (pip disallowing build-options with PEP517)

[th0ma7]

So there is no way this can be achieved using pip anymore ... pypa/pip#6304
Is there an easy way to know what's the latest version of cryptography not mandatory needing PEP517?
Or the opposite, an older pip version I could use?

[reaperhulk]

When we added the pyproject.toml (to comply with Python packaging best practices) the PEP517 requirements became relevant, but weren't enforced until...not sure which pip version. Honestly, I'd strongly suggest not using pip wheel, as it isn't fit for purpose 😞

[th0ma7]

A proposed option pypa/pip#6304 (comment) mentions using --config-options. Could that be an "option" (punt intended).

[alex]

I don't think we know if --config-options will work, as that's a question of a pip/setuptools interaction.

At this point I think this is a general question about python pacakging, and not cryptography specific in any way, so I'm going to close this.