Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-21536: On Android, C extensions are linked to libpython #12989

Merged
merged 2 commits into from
Apr 29, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Doc/distutils/apiref.rst
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,8 @@ the full reference.

.. versionchanged:: 3.8

On Unix, C extensions are no longer linked to libpython.
On Unix, C extensions are no longer linked to libpython except on
Android.


.. class:: Distribution
Expand Down
12 changes: 6 additions & 6 deletions Doc/whatsnew/3.8.rst
Original file line number Diff line number Diff line change
Expand Up @@ -882,12 +882,12 @@ Changes in the Python API
Changes in the C API
--------------------

* On Unix, C extensions are no longer linked to libpython. When Python is
embedded, ``libpython`` must not be loaded with ``RTLD_LOCAL``, but
``RTLD_GLOBAL`` instead. Previously, using ``RTLD_LOCAL``, it was already not
possible to load C extensions which were not linked to ``libpython``, like C
extensions of the standard library built by the ``*shared*`` section of
``Modules/Setup``.
* On Unix, C extensions are no longer linked to libpython except on
Android. When Python is embedded, ``libpython`` must not be loaded with
``RTLD_LOCAL``, but ``RTLD_GLOBAL`` instead. Previously, using
``RTLD_LOCAL``, it was already not possible to load C extensions which were
not linked to ``libpython``, like C extensions of the standard library built
by the ``*shared*`` section of ``Modules/Setup``.

* Use of ``#`` variants of formats in parsing or building value (e.g.
:c:func:`PyArg_ParseTuple`, :c:func:`Py_BuildValue`, :c:func:`PyObject_CallFunction`,
Expand Down
15 changes: 15 additions & 0 deletions Lib/distutils/command/build_ext.py
Original file line number Diff line number Diff line change
Expand Up @@ -714,5 +714,20 @@ def get_libraries(self, ext):
# don't extend ext.libraries, it may be shared with other
# extensions, it is a reference to the original list
return ext.libraries + [pythonlib]
# On Android only the main executable and LD_PRELOADs are considered
# to be RTLD_GLOBAL, all the dependencies of the main executable
# remain RTLD_LOCAL and so the shared libraries must be linked with
# libpython when python is built with a shared python library (issue
# bpo-21536).
else:
from distutils.sysconfig import get_config_var
if get_config_var('Py_ENABLE_SHARED'):
# Either a native build on an Android device or the
# cross-compilation of Python.
if (hasattr(sys, 'getandroidapilevel') or
('_PYTHON_HOST_PLATFORM' in os.environ and
get_config_var('ANDROID_API_LEVEL') != 0)):
ldversion = get_config_var('LDVERSION')
return ext.libraries + ['python' + ldversion]

return ext.libraries
1 change: 1 addition & 0 deletions Makefile.pre.in
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ AR= @AR@
READELF= @READELF@
SOABI= @SOABI@
LDVERSION= @LDVERSION@
LIBPYTHON= @LIBPYTHON@
GITVERSION= @GITVERSION@
GITTAG= @GITTAG@
GITBRANCH= @GITBRANCH@
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
On Unix, C extensions are no longer linked to libpython.
On Unix, C extensions are no longer linked to libpython except on Android.

It is now possible to load a C extension built using a shared library Python
with a statically linked Python.
Expand Down
5 changes: 4 additions & 1 deletion Misc/python-config.in
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,10 @@ for opt in opt_flags:
print(' '.join(flags))

elif opt in ('--libs', '--ldflags'):
libs = getvar('LIBS').split() + getvar('SYSLIBS').split()
libpython = getvar('LIBPYTHON')
libs = [libpython] if libpython else []
libs.extend(getvar('LIBS').split() + getvar('SYSLIBS').split())

# add the prefix/lib/pythonX.Y/config dir, but only if there is no
# shared library in prefix/lib/.
if opt == '--ldflags':
Expand Down
2 changes: 1 addition & 1 deletion Misc/python-config.sh.in
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ LIBM="@LIBM@"
LIBC="@LIBC@"
SYSLIBS="$LIBM $LIBC"
ABIFLAGS="@ABIFLAGS@"
LIBS="@LIBS@ $SYSLIBS"
LIBS="@LIBPYTHON@ @LIBS@ $SYSLIBS"
BASECFLAGS="@BASECFLAGS@"
LDLIBRARY="@LDLIBRARY@"
OPT="@OPT@"
Expand Down
9 changes: 9 additions & 0 deletions configure
Original file line number Diff line number Diff line change
Expand Up @@ -631,6 +631,7 @@ SRCDIRS
THREADHEADERS
LIBPL
PY_ENABLE_SHARED
LIBPYTHON
EXT_SUFFIX
ALT_SOABI
SOABI
Expand Down Expand Up @@ -15153,6 +15154,14 @@ LDVERSION='$(VERSION)$(ABIFLAGS)'
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $LDVERSION" >&5
$as_echo "$LDVERSION" >&6; }

# On Android the shared libraries must be linked with libpython.

if test -z "$ANDROID_API_LEVEL"; then
LIBPYTHON=''
else
LIBPYTHON="-lpython${VERSION}${ABIFLAGS}"
fi


if test x$PLATFORM_TRIPLET = x; then
LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}"
Expand Down
8 changes: 8 additions & 0 deletions configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -4647,6 +4647,14 @@ AC_MSG_CHECKING(LDVERSION)
LDVERSION='$(VERSION)$(ABIFLAGS)'
AC_MSG_RESULT($LDVERSION)

# On Android the shared libraries must be linked with libpython.
AC_SUBST(LIBPYTHON)
if test -z "$ANDROID_API_LEVEL"; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, being that I'm unfamiliar with build issues on Android why the specific test for $ANDROID_API_LEVEL and not something more generic like:

case $host in
    *-linux-android*) ....

?

Reason being, I need to add additional cases where libpython should be linked explicitly (e.g. Cygwin, MinGW, etc.) and it would be easier to build off the host_os for this than something very Android-specific.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a test based on $ANDROID_API_LEVEL or on $host as you suggest is equivalent.

LIBPYTHON=''
else
LIBPYTHON="-lpython${VERSION}${ABIFLAGS}"
fi

dnl define LIBPL after ABIFLAGS and LDVERSION is defined.
AC_SUBST(PY_ENABLE_SHARED)
if test x$PLATFORM_TRIPLET = x; then
Expand Down