[bug] Hail seems to need setuptools at runtime, but does not have it listed as a dependency

[pettyalex]

What happened?

Hello,

I installed hail into an empty, new Python 3.12.2 virtual environment, and was not able to import it. I see a failure like this:

(venv) (py312) alex@rpi400:~/hail $ python
Python 3.12.2 | packaged by conda-forge | (main, Feb 16 2024, 20:38:53) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hail
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/alex/hail/venv/lib/python3.12/site-packages/hail/__init__.py", line 2, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

It looks like in Python 3.12, the bundled setuptools was removed and new virtual environments will not have setuptools in them, it needs to be specifically installed through pip: python/cpython#95299

This could be fixed either by adding setuptools to hail's requirements so that it will be installed when users install hail, or hail could remove usage of setuptools & its associated modules (pkg_resources) at runtime, as some other projects have done: TDAmeritrade/stumpy#950

At a glance, the cleanest thing to do here may be to move off of the deprecated pkg_resources and to the recommended importlib if it has what you need: https://setuptools.pypa.io/en/latest/pkg_resources.html

I also have to admit that I discovered this while playing around with hail on a Raspberry Pi 4, so it is possible that something else broken caused this failure, but I believe I understand what's happening. Here's my full pip freeze for reference:

(venv) (py312) alex@rpi400:~/hail $ pip freeze
aiodns==2.0.0
aiohttp==3.9.3
aiosignal==1.3.1
attrs==23.2.0
avro==1.11.3
azure-common==1.1.28
azure-core==1.30.1
azure-identity==1.15.0
azure-mgmt-core==1.4.0
azure-mgmt-storage==20.1.0
azure-storage-blob==12.19.1
bokeh==3.4.0
boto3==1.34.73
botocore==1.34.73
cachetools==5.3.3
certifi==2024.2.2
cffi==1.16.0
charset-normalizer==3.3.2
click==8.1.7
commonmark==0.9.1
contourpy==1.2.0
cryptography==42.0.5
decorator==4.4.2
Deprecated==1.2.14
dill==0.3.8
frozenlist==1.4.1
google-auth==2.29.0
google-auth-oauthlib==0.8.0
hail==0.2.128
humanize==1.1.0
idna==3.6
isodate==0.6.1
janus==1.0.0
Jinja2==3.1.3
jmespath==1.0.1
jproperties==2.1.1
MarkupSafe==2.1.5
msal==1.28.0
msal-extensions==1.1.0
msrest==0.7.1
multidict==6.0.5
nest-asyncio==1.6.0
numpy==1.26.4
oauthlib==3.2.2
orjson==3.9.11
packaging==24.0
pandas==2.2.1
parsimonious==0.10.0
pillow==10.2.0
plotly==5.20.0
portalocker==2.8.2
protobuf==3.20.2
py4j==0.10.9.5
pyasn1==0.6.0
pyasn1_modules==0.4.0
pycares==4.4.0
pycparser==2.21
Pygments==2.17.2
PyJWT==2.8.0
pyspark==3.3.4
python-dateutil==2.9.0.post0
python-json-logger==2.0.7
pytz==2024.1
PyYAML==6.0.1
regex==2023.12.25
requests==2.31.0
requests-oauthlib==2.0.0
rich==12.6.0
rsa==4.9
s3transfer==0.10.1
scipy==1.11.4
six==1.16.0
sortedcontainers==2.4.0
tabulate==0.9.0
tenacity==8.2.3
tornado==6.4
typer==0.11.1
typing_extensions==4.10.0
tzdata==2024.1
urllib3==2.2.1
uvloop==0.19.0
wrapt==1.16.0
xyzservices==2023.10.1
yarl==1.9.4

Version

0.2.128

Relevant log output

No response

[ehigham]

Thanks for reporting and the suggestion of importlib.resources @pettyalex!