Skip to content

Commit

Permalink
Drop Python 2.7
Browse files Browse the repository at this point in the history
  • Loading branch information
mayeut committed Feb 14, 2021
1 parent d2772c2 commit 988f5c3
Show file tree
Hide file tree
Showing 28 changed files with 129 additions and 438 deletions.
6 changes: 1 addition & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,14 @@ What does it do?

| | macOS Intel | macOS Apple Silicon | Windows 64bit | Windows 32bit | manylinux x86_64 | manylinux i686 | manylinux aarch64 | manylinux ppc64le | manylinux s390x |
|---|---|---|---|---|---|---|---|---|---|
| CPython 2.7 || | ✅¹ | ✅¹ ||| | | |
| CPython 3.5 || ||||||||
| CPython 3.6 || ||||||||
| CPython 3.7 || ||||||||
| CPython 3.8 || ||||||||
| CPython 3.9 ||||||||||
| PyPy 2.7 v7.3.3 || | ||| | | | |
| PyPy 3.6 v7.3.3 || | ||| | | | |
| PyPy 3.7 (beta) v7.3.3 || | ||| | | | |

<sup>¹ Not supported on Travis</sup><br>

- Builds manylinux, macOS 10.9+, and Windows wheels for CPython and PyPy
- Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, and GitLab CI
- Bundles shared library dependencies on Linux and macOS through [auditwheel](https://github.com/pypa/auditwheel) and [delocate](https://github.com/matthew-brett/delocate)
Expand Down Expand Up @@ -99,7 +95,7 @@ jobs:
path: ./wheelhouse/*.whl
```
For more information, including building on Python 2, PyPI deployment, and the use of other CI services or the dedicated GitHub Action, check out the [documentation](https://cibuildwheel.readthedocs.org) and the [examples](https://github.com/joerick/cibuildwheel/tree/master/examples).
For more information, including PyPI deployment, and the use of other CI services or the dedicated GitHub Action, check out the [documentation](https://cibuildwheel.readthedocs.org) and the [examples](https://github.com/joerick/cibuildwheel/tree/master/examples).
Options
-------
Expand Down
4 changes: 0 additions & 4 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,6 @@ jobs:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.6'
- script: choco install vcpython27 -f -y
displayName: Install Visual C++ for Python 2.7
- bash: |
python -m pip install -e ".[dev]" pytest-azurepipelines
python ./bin/run_tests.py
Expand All @@ -39,8 +37,6 @@ jobs:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.8'
- script: choco install vcpython27 -f -y
displayName: Install Visual C++ for Python 2.7
- bash: |
python -m pip install -e ".[dev]" pytest-azurepipelines
python ./bin/run_tests.py
2 changes: 1 addition & 1 deletion bin/update_dependencies.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
# regenerate the constraints files
os.environ['CUSTOM_COMPILE_COMMAND'] = "bin/update_dependencies.py"

PYTHON_VERSIONS = ['27', '35', '36', '37', '38', '39']
PYTHON_VERSIONS = ['35', '36', '37', '38', '39']

if '--no-docker' in sys.argv:
for python_version in PYTHON_VERSIONS:
Expand Down
3 changes: 1 addition & 2 deletions cibuildwheel/linux.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,7 @@ def build(options: BuildOptions) -> None:
dependency_constraint_flags: List[PathOrStr] = []
if config.identifier.startswith("pp"):
# Patch PyPy to make sure headers get installed into a venv
patch_version = '_27' if config.version == '2.7' else ''
patch_path = resources_dir / f'pypy_venv{patch_version}.patch'
patch_path = resources_dir / 'pypy_venv.patch'
patch_docker_path = PurePath('/pypy_venv.patch')
docker.copy_into(patch_path, patch_docker_path)
try:
Expand Down
11 changes: 5 additions & 6 deletions cibuildwheel/macos.py
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ def install_cpython(version: str, url: str) -> Path:

# if this version of python isn't installed, get it from python.org and install
python_package_identifier = f'org.python.Python.PythonFramework-{version}'
python_executable = 'python3' if version[0] == '3' else 'python'
python_executable = 'python3'
installation_bin_path = Path(f'/Library/Frameworks/Python.framework/Versions/{version}/bin')

if python_package_identifier not in installed_system_packages:
Expand All @@ -147,7 +147,7 @@ def install_cpython(version: str, url: str) -> Path:

call(["sudo", str(installation_bin_path/python_executable), str(install_certifi_script)])

pip_executable = 'pip3' if version[0] == '3' else 'pip'
pip_executable = 'pip3'
make_symlinks(installation_bin_path, python_executable, pip_executable)

return installation_bin_path
Expand All @@ -164,13 +164,12 @@ def install_pypy(version: str, url: str) -> Path:
download(url, downloaded_tar_bz2)
call(['tar', '-C', '/tmp', '-xf', downloaded_tar_bz2])
# Patch PyPy to make sure headers get installed into a venv
patch_version = '_27' if version == '2.7' else ''
patch_path = resources_dir / f'pypy_venv{patch_version}.patch'
patch_path = resources_dir / 'pypy_venv.patch'
call(['patch', '--force', '-p1', '-d', installation_path, '-i', patch_path])

installation_bin_path = installation_path / 'bin'
python_executable = 'pypy3' if version[0] == '3' else 'pypy'
pip_executable = 'pip3' if version[0] == '3' else 'pip'
python_executable = 'pypy3'
pip_executable = 'pip3'
make_symlinks(installation_bin_path, python_executable, pip_executable)

return installation_bin_path
Expand Down
10 changes: 0 additions & 10 deletions cibuildwheel/resources/build-platforms.toml
Original file line number Diff line number Diff line change
@@ -1,20 +1,15 @@
[linux]
python_configurations = [
{ identifier = "cp27-manylinux_x86_64", version = "2.7", path_str = "/opt/python/cp27-cp27m" },
{ identifier = "cp27-manylinux_x86_64", version = "2.7", path_str = "/opt/python/cp27-cp27mu" },
{ identifier = "cp35-manylinux_x86_64", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
{ identifier = "cp36-manylinux_x86_64", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
{ identifier = "cp37-manylinux_x86_64", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
{ identifier = "cp38-manylinux_x86_64", version = "3.8", path_str = "/opt/python/cp38-cp38" },
{ identifier = "cp39-manylinux_x86_64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp27-manylinux_i686", version = "2.7", path_str = "/opt/python/cp27-cp27m" },
{ identifier = "cp27-manylinux_i686", version = "2.7", path_str = "/opt/python/cp27-cp27mu" },
{ identifier = "cp35-manylinux_i686", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
{ identifier = "cp36-manylinux_i686", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
{ identifier = "cp37-manylinux_i686", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
{ identifier = "cp38-manylinux_i686", version = "3.8", path_str = "/opt/python/cp38-cp38" },
{ identifier = "cp39-manylinux_i686", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "pp27-manylinux_x86_64", version = "2.7", path_str = "/opt/python/pp27-pypy_73" },
{ identifier = "pp36-manylinux_x86_64", version = "3.6", path_str = "/opt/python/pp36-pypy36_pp73" },
{ identifier = "pp37-manylinux_x86_64", version = "3.7", path_str = "/opt/python/pp37-pypy37_pp73" },
{ identifier = "cp35-manylinux_aarch64", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
Expand All @@ -36,23 +31,19 @@ python_configurations = [

[macos]
python_configurations = [
{ identifier = "cp27-macosx_x86_64", version = "2.7", url = "https://www.python.org/ftp/python/2.7.18/python-2.7.18-macosx10.9.pkg" },
{ identifier = "cp35-macosx_x86_64", version = "3.5", url = "https://www.python.org/ftp/python/3.5.4/python-3.5.4-macosx10.6.pkg" },
{ identifier = "cp36-macosx_x86_64", version = "3.6", url = "https://www.python.org/ftp/python/3.6.8/python-3.6.8-macosx10.9.pkg" },
{ identifier = "cp37-macosx_x86_64", version = "3.7", url = "https://www.python.org/ftp/python/3.7.9/python-3.7.9-macosx10.9.pkg" },
{ identifier = "cp38-macosx_x86_64", version = "3.8", url = "https://www.python.org/ftp/python/3.8.7/python-3.8.7-macosx10.9.pkg" },
{ identifier = "cp39-macosx_x86_64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.1/python-3.9.1-macos11.0.pkg" },
{ identifier = "cp39-macosx_arm64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.1/python-3.9.1-macos11.0.pkg" },
{ identifier = "cp39-macosx_universal2", version = "3.9", url = "https://www.python.org/ftp/python/3.9.1/python-3.9.1-macos11.0.pkg" },
{ identifier = "pp27-macosx_x86_64", version = "2.7", url = "https://downloads.python.org/pypy/pypy2.7-v7.3.3-osx64.tar.bz2" },
{ identifier = "pp36-macosx_x86_64", version = "3.6", url = "https://downloads.python.org/pypy/pypy3.6-v7.3.3-osx64.tar.bz2" },
{ identifier = "pp37-macosx_x86_64", version = "3.7", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.3-osx64.tar.bz2" },
]

[windows]
python_configurations = [
{ identifier = "cp27-win32", version = "2.7.18", arch = "32" },
{ identifier = "cp27-win_amd64", version = "2.7.18", arch = "64" },
{ identifier = "cp35-win32", version = "3.5.4", arch = "32" },
{ identifier = "cp35-win_amd64", version = "3.5.4", arch = "64" },
{ identifier = "cp36-win32", version = "3.6.8", arch = "32" },
Expand All @@ -63,7 +54,6 @@ python_configurations = [
{ identifier = "cp38-win_amd64", version = "3.8.7", arch = "64" },
{ identifier = "cp39-win32", version = "3.9.1", arch = "32" },
{ identifier = "cp39-win_amd64", version = "3.9.1", arch = "64" },
{ identifier = "pp27-win32", version = "2.7", arch = "32", url = "https://downloads.python.org/pypy/pypy2.7-v7.3.3-win32.zip" },
{ identifier = "pp36-win32", version = "3.6", arch = "32", url = "https://downloads.python.org/pypy/pypy3.6-v7.3.3-win32.zip" },
{ identifier = "pp37-win32", version = "3.7", arch = "32", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.3-win32.zip" },
]
55 changes: 0 additions & 55 deletions cibuildwheel/resources/constraints-python27.txt

This file was deleted.

27 changes: 0 additions & 27 deletions cibuildwheel/resources/pypy_venv_27.patch

This file was deleted.

2 changes: 1 addition & 1 deletion cibuildwheel/util.py
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ def get_for_python_version(self, version: str) -> Path:
version_parts = version.split('.')

# try to find a version-specific dependency file e.g. if
# ./constraints.txt is the base, look for ./constraints-python27.txt
# ./constraints.txt is the base, look for ./constraints-python36.txt
specific_stem = self.base_file_path.stem + f'-python{version_parts[0]}{version_parts[1]}'
specific_name = specific_stem + self.base_file_path.suffix
specific_file_path = self.base_file_path.with_name(specific_name)
Expand Down
14 changes: 2 additions & 12 deletions cibuildwheel/windows.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,6 @@
read_python_configs,
)

IS_RUNNING_ON_AZURE = Path('C:\\hostedtoolcache').exists()
IS_RUNNING_ON_TRAVIS = os.environ.get('TRAVIS_OS_NAME') == 'windows'


def call(args: Sequence[PathOrStr], env: Optional[Dict[str, str]] = None,
cwd: Optional[str] = None) -> None:
Expand All @@ -42,7 +39,7 @@ def shell(command: str, env: Optional[Dict[str, str]] = None, cwd: Optional[str]


def get_nuget_args(version: str, arch: str) -> List[str]:
python_name = 'python' if version[0] == '3' else 'python2'
python_name = 'python'
if arch == '32':
python_name = python_name + 'x86'
return [python_name, '-Version', version, '-OutputDirectory', 'C:\\cibw\\python']
Expand All @@ -69,12 +66,6 @@ def get_python_configurations(
'64': Architecture.AMD64,
}

custom_compiler = os.environ.get('DISTUTILS_USE_SDK') and os.environ.get('MSSdk')
if IS_RUNNING_ON_TRAVIS and not custom_compiler:
# cannot install VCForPython27.msi which is needed for compiling C software
# try with (and similar): msiexec /i VCForPython27.msi ALLUSERS=1 ACCEPT=YES /passive
python_configurations = [c for c in python_configurations if not c.version.startswith('2.7')]

# skip builds as required
python_configurations = [
c for c in python_configurations
Expand Down Expand Up @@ -108,8 +99,7 @@ def install_pypy(version: str, arch: str, url: str) -> Path:
download(url, pypy_zip)
# Extract to the parent directory because the zip file still contains a directory
extract_zip(pypy_zip, installation_path.parent)
pypy_exe = 'pypy3.exe' if version[0] == '3' else 'pypy.exe'
(installation_path / 'python.exe').symlink_to(installation_path / pypy_exe)
(installation_path / 'python.exe').symlink_to(installation_path / 'pypy3.exe')
return installation_path


Expand Down
3 changes: 0 additions & 3 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ This has been moved to using docker, so you only need the following instructions
The dependency update script in the next section requires multiple python versions installed. One way to do this is to use `pyenv`:

```bash
pyenv install 2.7.18
pyenv install 3.5.9
pyenv install 3.6.11
pyenv install 3.7.8
Expand All @@ -77,8 +76,6 @@ pyenv install 3.7.8
Then, you need to make the required virtual environments:

```bash
$(pyenv prefix 2.7.18)/bin/python -m pip install virtualenv
$(pyenv prefix 2.7.18)/bin/python -m virtualenv env27
$(pyenv prefix 3.5.9)/bin/python -m venv env35
$(pyenv prefix 3.6.11)/bin/python -m venv env36
$(pyenv prefix 3.7.8)/bin/python -m venv env37
Expand Down
44 changes: 0 additions & 44 deletions docs/cpp_standards.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,6 @@ title: Modern C++ standards
Building Python wheels with modern C++ standards (C++11 and later) requires a few tricks.


## Python 2.7 and C++17

The Python 2.7 header files use the `register` keyword, which is [reserved and unused from C+17 onwards](https://en.cppreference.com/w/cpp/keyword/register). Compiling a wheel for Python 2.7 with the C++17 standard is still possible to allow usage of `register` using proper flag `-Wno-register` for gcc/clang and `/wd5033` for MSVC.

## manylinux1 and C++14
The default `manylinux1` image (based on CentOS 5) contains a version of GCC and libstdc++ that only supports C++11 and earlier standards. There are however ways to compile wheels with the C++14 standard (and later): https://github.com/pypa/manylinux/issues/118

Expand All @@ -27,43 +23,3 @@ To get C++17 support, Xcode 9.3+ is needed, requiring at least macOS 10.13 on th
However, if only C++17 compiler and standard template library (STL) features are used (not needing a C++17 runtime) it might be possible to set `MACOSX_DEPLOYMENT_TARGET` to a lower value, such as `"10.9"`. To find out if this is the case, try compiling and running with a lower `MACOSX_DEPLOYMENT_TARGET`: if C++17 features are used that require a more recent deployment target, building the wheel should fail.

For more details see https://en.cppreference.com/w/cpp/compiler_support, https://en.wikipedia.org/wiki/Xcode, and https://xcodereleases.com/: Xcode 10 needs macOS 10.13 and Xcode 11 needs macOS 10.14.

## Windows and Python 2.7

Visual C++ for Python 2.7 does not support modern C++ standards (i.e., C++11 and later). When building on Appveyor, you will need to either use the "Visual Studio 2017" or "Visual Studio 2019" image, but Python 2.7 is not supported on these images - skip it by setting `CIBW_SKIP=cp27-win*`.

There is an optional workaround for this, though: the pybind11 project argues and shows that it is [possible to compile Python 2.7 extension with a newer compiler](https://pybind11.readthedocs.io/en/stable/faq.html#working-with-ancient-visual-studio-2008-builds-on-windows) and has an example project showing how to do this: https://github.com/pybind/python_example. The main catch is that a user might need to install [a newer "Microsoft Visual C++ Redistributable"](https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads), since the newer C++ standard library's binaries are not included by default with the Python 2.7 installation.

Forcing `distutils` or `setuptools` to use a more recent version of MSVC that supports modern C++ can be done in the following way:

1. Set the environment variables `DISTUTILS_USE_SDK=1` and `MSSdk=1`. These two environment variables will tell `distutils`/`setuptools` to not search and set up a build environment that uses Visual C++ for Python 2.7 (aka. MSVC 9).
2. Set up the build Visual Studio build environment you want to use, making sure that e.g. `PATH` contains `cl`, `link`, etc.
- Usually, this can be done through `vcvarsall.bat x86` or `vcvarsall.bat x64`. The exact location of this file depends on the installation, but the default path for VS 2019 Community (e.g. used in AppVeyor's `Visual Studio 2019` image) is `C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat`. **Note**: `vcvarsall.bat` changes the environment variables, so this cannot be run in a subprocess/subshell and consequently running `vsvarsall.bat` in `CIBW_BEFORE_BUILD` does not have any effect.
- In Azure Pipelines, [a `VSBuild` task is available](https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/build/visual-studio-build)
- In GitHub Actions, the default shell is powershell, so you'll need to use `shell: cmd` to source a `.bat` file, and the directory is `"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvarsall.bat"`. Or you can use an action, such as the [`ilammy/msvc-dev-cmd@v1`](https://github.com/ilammy/msvc-dev-cmd) action to setup the environment (see example below).
3. Next, call `cibuildwheel`. Unfortunately, MSVC has separate toolchains for compiling 32-bit and 64-bit, so you will need to run `cibuildwheel` twice: once with `CIBW_BUILD=*-win32` after setting up the `x86` build environment, and once with `CIBW_BUILD=*-win_amd64` in a `x64` environment (see previous step).


**GitHub Action example with ilammy/msvc-dev-cmd**:

```yaml
- uses: ilammy/msvc-dev-cmd@v1

- name: Build 64-bit wheel
run: python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_BUILD: cp27-win_amd64
DISTUTILS_USE_SDK: 1
MSSdk: 1

- uses: ilammy/msvc-dev-cmd@v1
with:
arch: x86

- name: Build 32-bit wheel
run: python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_BUILD: cp27-win32
DISTUTILS_USE_SDK: 1
MSSdk: 1
```
Loading

0 comments on commit 988f5c3

Please sign in to comment.