Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-41282: Fix broken make install #26329

Merged
merged 2 commits into from
May 24, 2021
Merged

Conversation

encukou
Copy link
Member

@encukou encukou commented May 24, 2021

A previous commit broke a check in sysconfig when building cpython itself.
This caused builds of the standard library modules to search a wrong
location (the installed location rather than the source directory) for
header files with the net effect that a make install
incorrectly caused all extension modules to be rebuilt again and
with incorrect include file paths.

There's a complication. When building Python, we need two distinct
"include" directories:

  • source .h files
  • install target for .h files

Note that this doesn't matter except when building Python from source.

Historically:

  • source .h files were in the distutils scheme under 'include'
  • the install directory was in the distutils.command.install scheme
    under 'headers'

GH-24549 merged these; sysconfig is now the single source of truth and
distutils is derived from it.

Introduce a "secret" scheme path, 'headers', which contains
the install target. It is only present when building Python.
The distutils code uses it if present, and falls back to 'include'.

https://bugs.python.org/issue41282

ned-deily and others added 2 commits May 24, 2021 03:53
A previous commit broke a check in sysconfig when building cpython itself.
This caused builds of the standard library modules to search a wrong
location (the installed location rather than the source directory) for
header files with the net effect that a ``make install``
incorrectly caused all extension modules to be rebuilt again and
with incorrect include file paths.
When building Python, we need two distinct "include" directories:
- source .h files
- install target for .h files

Note that this doesn't matter except when building Python from source.

Historically:
- source .h files were in the distutils scheme under 'include'
- the install directory was in the distutils.command.install scheme
    under 'headers'

pythonGH-24549 merged these; sysconfig is now the single source of truth and
distutils is derived from it.

This commit introduces a "secret" scheme path, 'headers', which contains
the install target. It is only present when building Python.
The distutils code uses it if present, and falls back to 'include'.
@encukou encukou added the needs backport to 3.10 only security fixes label May 24, 2021
@encukou encukou requested review from ned-deily and pablogsal May 24, 2021 12:21
@encukou encukou added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 24, 2021
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @encukou for commit b300aac 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 24, 2021
@frenzymadness
Copy link
Contributor

Your analysis makes sense, the code looks good and I'm able to identify why I've never met this problem before when I was preparing the merge of sysconfig modules.

I'm sorry for the mistake I made.

Copy link
Member

@ned-deily ned-deily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me and a more robust one than my initial stab at it. Thanks, @encukou! If you don't get to it soonish, I'll probably merge this to get the backport to 3.10 going.

@ned-deily ned-deily merged commit 563bd5a into python:main May 24, 2021
@miss-islington
Copy link
Contributor

Thanks @encukou for the PR, and @ned-deily for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 24, 2021
A previous commit broke a check in sysconfig when building cpython itself.
This caused builds of the standard library modules to search a wrong
location (the installed location rather than the source directory) for
header files with the net effect that a ``make install``
incorrectly caused all extension modules to be rebuilt again and
with incorrect include file paths.

When building Python, we need two distinct "include" directories:
- source .h files
- install target for .h files

Note that this doesn't matter except when building Python from source.

Historically:
- source .h files were in the distutils scheme under 'include'
- the install directory was in the distutils.command.install scheme
    under 'headers'

pythonGH-24549 merged these; sysconfig is now the single source of truth and
distutils is derived from it.

This commit introduces a "secret" scheme path, 'headers', which contains
the install target. It is only present when building Python.
The distutils code uses it if present, and falls back to 'include'.

Co-authored-by: Ned Deily <nad@python.org>
(cherry picked from commit 563bd5a)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
@bedevere-bot
Copy link

GH-26336 is a backport of this pull request to the 3.10 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.10 only security fixes label May 24, 2021
miss-islington added a commit that referenced this pull request May 24, 2021
A previous commit broke a check in sysconfig when building cpython itself.
This caused builds of the standard library modules to search a wrong
location (the installed location rather than the source directory) for
header files with the net effect that a ``make install``
incorrectly caused all extension modules to be rebuilt again and
with incorrect include file paths.

When building Python, we need two distinct "include" directories:
- source .h files
- install target for .h files

Note that this doesn't matter except when building Python from source.

Historically:
- source .h files were in the distutils scheme under 'include'
- the install directory was in the distutils.command.install scheme
    under 'headers'

GH-24549 merged these; sysconfig is now the single source of truth and
distutils is derived from it.

This commit introduces a "secret" scheme path, 'headers', which contains
the install target. It is only present when building Python.
The distutils code uses it if present, and falls back to 'include'.

Co-authored-by: Ned Deily <nad@python.org>
(cherry picked from commit 563bd5a)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
@encukou encukou deleted the sysconfig-pr26327 branch May 25, 2021 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants