Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fedora 35(x64), error while loading shared libraries: libcrypt.so.1 #113

Closed
bigcat88 opened this issue Feb 17, 2022 · 6 comments
Closed
Labels
bug Something isn't working

Comments

@bigcat88
Copy link

Good project, thanks you for it.

Have a problem(subj).
When do what here proposed it starts working: missing-libcrypt-so-1-library-on-fedora
But the main goal to install everything without root privileges.

A bit more info: fedoraproject: Replace glibc libcrypt with libxcrypt

Is here can be any workaround? Or the only way is to build separate version for last two Fedora's?

@indygreg
Copy link
Collaborator

Note that there is prior history with libcrypt.so.1 and this project: indygreg/PyOxidizer#89.

It is still an unsolved problem. I suspect that in the wild a lot of users have libxcrypt-compat installed for various reasons.

FWIW in Fedora 35, libcrypt.so.2 and libcrypt.so.2.0.0 are installed by default. The libxcrypt-compat package provides libcrypt.so.1 and libcrypt.so.1.1.0.

I'll repeat what I said in the old PyOxidizer issue that the Linux Standard Base Core Specification defines libcrypt.so.1 as containing the API for libcrypt (https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-AMD64/LSB-Core-AMD64/libcrypt.html). So on casual glance it looks like Fedora is in violation of the LSB by not providing a libcrypt.so.1 by default.

The reason this matters is because the LSB defines an interface that projects like this can target with reasonable guarantees of cross-distro compatibility. And by not conforming with the LSB, Fedora is essentially foregoing that compatibility.

From my perspective, my preference would be for Fedora to be compliant with the LSB Core Specification and provide a libcrypt.so.1 out-of-the-box. What actually provides that file, I don't care: it just needs to exist and provide the interfaces as defined by the LSB.

An alternative solution is to provide a Python build that doesn't depend on libcrypt.so.1. That would entail disabling the _crypt extension module, which would remove (all?) functionality from the crypt module. That might be doable. FWIW we already disabled the nis extension module in #51 for similar Fedora/RHEL support reasons. Although libnis isn't defined by the LSB, so this was an easier decision.

At the very least we should document this quirk and its current workaround.

@indygreg indygreg added the bug Something isn't working label Feb 18, 2022
@indygreg
Copy link
Collaborator

On audit of the _crypt Python extension module, it calls crypt_r or crypt if crypt_r is not available. The availability of cyrpt_r is a configure-time check. https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Core-AMD64/LSB-Core-AMD64/libcrypt.html says both of these functions are in libcrypt.

If CPython were relying on functions not defined by the LSB, that would be grounds to remove the functionality. But since the functions it uses are defined by the LSB Core Specification, I don't believe CPython or this project are in the wrong in their dependence on libcrypt.so.1.

@indygreg
Copy link
Collaborator

See also pantsbuild/pants#7369 (comment) and https://patchwork.ozlabs.org/project/glibc/patch/20170829184247.6875-1-zackw@panix.com/.

It looks like glibc deprecated libcrypt back in 2017. So I guess glibc is the cause of Linux distros being incompatible with the LSB Core Specification?!

Standards. 🤷‍♂️

@indygreg
Copy link
Collaborator

If glibc did in fact delete libcrypt and distros are providing libcrypt.so.1 through some other implementation [to remain compliant with the LSB], that pushes me towards disabling the _crypt Python extension module by default. But I still wish I could rely on the LSB... because standards.

@indygreg
Copy link
Collaborator

indygreg commented Feb 18, 2022 via email

@indygreg
Copy link
Collaborator

Looks like a fix was committed a few weeks ago. So closing this issue. Will also push a commit adding documentation about this quirk later today.

indygreg added a commit that referenced this issue Jul 31, 2022
indygreg added a commit that referenced this issue Jul 19, 2023
Modern Linux distributions are starting to remove `libcrypt.so.1` from the
base disto. See #173 and #113 before it for more context.

The `_crypt` extension module depends on `libcrypt.so.1` and our static
linking of this extension is causing the full Python distribution to
depend on `libcrypt.so.1`, causing our binaries to not load/run on
these distributions.

This commit adds support for building extension modules as shared
libraries (the way CPython does things by default). We update our
YAML config to build `_crypt` as a shared library.
indygreg added a commit that referenced this issue Jul 19, 2023
Modern Linux distributions are starting to remove `libcrypt.so.1` from the
base disto. See #173 and #113 before it for more context.

The `_crypt` extension module depends on `libcrypt.so.1` and our static
linking of this extension is causing the full Python distribution to
depend on `libcrypt.so.1`, causing our binaries to not load/run on
these distributions.

This commit adds support for building extension modules as shared
libraries (the way CPython does things by default). We update our
YAML config to build `_crypt` as a shared library.
indygreg added a commit that referenced this issue Jul 20, 2023
Modern Linux distributions are starting to remove `libcrypt.so.1` from the
base disto. See #173 and #113 before it for more context.

The `_crypt` extension module depends on `libcrypt.so.1` and our static
linking of this extension is causing the full Python distribution to
depend on `libcrypt.so.1`, causing our binaries to not load/run on
these distributions.

This commit adds support for building extension modules as shared
libraries (the way CPython does things by default). We update our
YAML config to build `_crypt` as a shared library.

As part of this we remove a patch to `makesetup` that was removing
rules for emitting extension info to the generated `Makefile`. Git
commit messages and my mental memory are a bit foggy on the history.
But I want to say this patch had its origins in the very early days
of this project when I was still trying to figure out how to get
extension modules to statically link. The removed code created some
contamination of the generated `Makefile` making it harder to grok
behavior. I think that the presence of these additional variables and
rules is safe and doesn't cause unwanted regressions. We have pretty
strong validation of the built distributions - including ELF symbol
visibility checks. So I'm optimistic this reintroduction won't cause
problems.

Closes #173.
indygreg added a commit that referenced this issue Jul 22, 2023
Modern Linux distributions are starting to remove `libcrypt.so.1` from the
base disto. See #173 and #113 before it for more context.

The `_crypt` extension module depends on `libcrypt.so.1` and our static
linking of this extension is causing the full Python distribution to
depend on `libcrypt.so.1`, causing our binaries to not load/run on
these distributions.

This commit adds support for building extension modules as shared
libraries (the way CPython does things by default). We update our
YAML config to build `_crypt` as a shared library.

As part of this we remove a patch to `makesetup` that was removing
rules for emitting extension info to the generated `Makefile`. Git
commit messages and my mental memory are a bit foggy on the history.
But I want to say this patch had its origins in the very early days
of this project when I was still trying to figure out how to get
extension modules to statically link. The removed code created some
contamination of the generated `Makefile` making it harder to grok
behavior. I think that the presence of these additional variables and
rules is safe and doesn't cause unwanted regressions. We have pretty
strong validation of the built distributions - including ELF symbol
visibility checks. So I'm optimistic this reintroduction won't cause
problems.

Closes #173.
indygreg added a commit that referenced this issue Jul 23, 2023
Modern Linux distributions are starting to remove `libcrypt.so.1` from the
base disto. See #173 and #113 before it for more context.

The `_crypt` extension module depends on `libcrypt.so.1` and our static
linking of this extension is causing the full Python distribution to
depend on `libcrypt.so.1`, causing our binaries to not load/run on
these distributions.

This commit adds support for building extension modules as shared
libraries (the way CPython does things by default). We update our
YAML config to build `_crypt` as a shared library.

As part of this we remove a patch to `makesetup` that was removing
rules for emitting extension info to the generated `Makefile`. Git
commit messages and my mental memory are a bit foggy on the history.
But I want to say this patch had its origins in the very early days
of this project when I was still trying to figure out how to get
extension modules to statically link. The removed code created some
contamination of the generated `Makefile` making it harder to grok
behavior. I think that the presence of these additional variables and
rules is safe and doesn't cause unwanted regressions. We have pretty
strong validation of the built distributions - including ELF symbol
visibility checks. So I'm optimistic this reintroduction won't cause
problems.

Closes #173.
indygreg added a commit that referenced this issue Jul 24, 2023
Modern Linux distributions are starting to remove `libcrypt.so.1` from the
base disto. See #173 and #113 before it for more context.

The `_crypt` extension module depends on `libcrypt.so.1` and our static
linking of this extension is causing the full Python distribution to
depend on `libcrypt.so.1`, causing our binaries to not load/run on
these distributions.

This commit adds support for building extension modules as shared
libraries (the way CPython does things by default). We update our
YAML config to build `_crypt` as a shared library.

As part of this we remove a patch to `makesetup` that was removing
rules for emitting extension info to the generated `Makefile`. Git
commit messages and my mental memory are a bit foggy on the history.
But I want to say this patch had its origins in the very early days
of this project when I was still trying to figure out how to get
extension modules to statically link. The removed code created some
contamination of the generated `Makefile` making it harder to grok
behavior. I think that the presence of these additional variables and
rules is safe and doesn't cause unwanted regressions. We have pretty
strong validation of the built distributions - including ELF symbol
visibility checks. So I'm optimistic this reintroduction won't cause
problems.

Closes #173.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants