[python-package] LightGBM wheel on PyPI seems to have the wrong platform tag

[tdoublep]

Description

I'm having problems trying to use LightGBM as part of our a build process that runs inside a manylinux2010 container. In particular I am seeing the following error:

OSError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /opt/python/cp37-cp37m/lib/python3.7/site-packages/lightgbm/lib_lightgbm.so)

Looking on PyPI, I see that LightGBM is publishing wheels with the manylinux1 platform tag, which is what pip is pulling down in the above environment. Since manylinux1 is based on older glibc than manylinux2010, and glibc is forward compatible, I was expecting this to work fine. Some deeper investigation (see below) seems to suggest that the wheel may have the wrong platform tag.

This problem is not blocking us, since I can install using the --no-binary option and it works fine, but I thought it could be helpful to report this.

Please let me know if any more information could help! Cheers!

Reproducible example

Some additional inspection of the LightGBM manylinux1 wheel using the auditwheel tool (pip install auditwheel) shows:

$ auditwheel show lightgbm-3.3.1-py3-none-manylinux1_x86_64.whl 

lightgbm-3.3.1-py3-none-manylinux1_x86_64.whl is consistent with the
following platform tag: "linux_x86_64".

The wheel references external versioned symbols in these system-
provided shared libraries: libgcc_s.so.1 with versions {'GCC_3.0'},
libpthread.so.0 with versions {'GLIBC_2.3.4', 'GLIBC_2.2.5'},
libgomp.so.1 with versions {'OMP_1.0', 'GOMP_1.0'}, libc.so.6 with
versions {'GLIBC_2.3', 'GLIBC_2.6', 'GLIBC_2.17', 'GLIBC_2.14',
'GLIBC_2.2.5', 'GLIBC_2.3.4', 'GLIBC_2.4'}, libm.so.6 with versions
{'GLIBC_2.2.5'}, libstdc++.so.6 with versions {'CXXABI_1.3',
'GLIBCXX_3.4', 'GLIBCXX_3.4.19', 'GLIBCXX_3.4.9', 'GLIBCXX_3.4.14',
'CXXABI_1.3.5', 'GLIBCXX_3.4.11', 'GLIBCXX_3.4.18', 'CXXABI_1.3.7',
'CXXABI_1.3.3'}, libdl.so.2 with versions {'GLIBC_2.2.5'}

This constrains the platform tag to "manylinux2014_x86_64". In order
to achieve a more compatible tag, you would need to recompile a new
wheel from source on a system with earlier versions of these
libraries, such as a recent manylinux image.

This tool indicates the wheels are constrained to manylinux2014 and thus will not necessarily work with manylinux1 (or indeed manylinux2010 environments).

Environment info

LightGBM version or commit hash: 3.3.1

Command(s) you used to install LightGBM

pip install lightgbm

Manylinux2010 docker container.

[StrikerRUS]

@tdoublep Hey! Thanks for raising this issue!

We are aware of that LightGBM isn't fully compatible with manylinux1 tag. Please check this comment: #3421 (comment).
In short, we are planning to adopt PEP600 soon.

[tdoublep]

Alright! Yes adopting PEP600 would solve the issue.

In the short term, renaming the wheels to use the manylinux2014 platform tag (which they are fully compliant with) would solve my issue since it would force pip to download the source and compile on older platforms. However, if the current wheels still work on a majority of envs that are "older" than manylinux2014, this may not be desirable. PEP600 enables more flexibility in this regard.

[jameslamb]

Just updating for those following this. As of #5598, LightGBM will start publishing PEP600 (specifically manylinux_2_28_x86_64) wheels in its next release (#5153).

[jameslamb]

Linking related discussion: #5061.

This work is effectively done, but not closing this issue until an actual release actually corrects the tags on PyPI.

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.