[Python][CI] verify-release for Python is currently failing due to old version of setuptools_scm being installed

[raulcd]

Describe the bug, including details regarding any error messages, version, and platform.

Due to the recent change to move some configuration from setup.py to pyproject.toml (#37929) we require a higher version of setuptools and setuptools_scm being installed for the verification script.
The jobs:

• verify-rc-source-python-linux-almalinux-8-amd64
• verify-rc-source-python-linux-ubuntu-20.04-amd64
• verify-rc-source-python-linux-ubuntu-22.04-amd64
• verify-rc-source-python-macos-amd64
• verify-rc-source-python-macos-arm64

Are currently failing with:

 Installing collected packages: typing-extensions, packaging, cython, setuptools_scm
Successfully installed cython-3.0.10 packaging-24.0 setuptools_scm-7.1.0 typing-extensions-4.12.1
/arrow/python /arrow /
/arrow/python/setup.py:34: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  import pkg_resources
Traceback (most recent call last):
  File "/arrow/python/setup.py", line 372, in <module>
    setup(
  File "/tmp/arrow-HEAD.2nue1/venv-source/lib64/python3.11/site-packages/setuptools/__init__.py", line 103, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/arrow-HEAD.2nue1/venv-source/lib64/python3.11/site-packages/setuptools/_distutils/core.py", line 146, in setup
    _setup_distribution = dist = klass(attrs)
                                 ^^^^^^^^^^^^
  File "/tmp/arrow-HEAD.2nue1/venv-source/lib64/python3.11/site-packages/setuptools/dist.py", line 307, in __init__
    _Distribution.__init__(self, dist_attrs)
  File "/tmp/arrow-HEAD.2nue1/venv-source/lib64/python3.11/site-packages/setuptools/_distutils/dist.py", line 284, in __init__
    self.finalize_options()
  File "/tmp/arrow-HEAD.2nue1/venv-source/lib64/python3.11/site-packages/setuptools/dist.py", line 659, in finalize_options
    ep(self)
  File "/tmp/arrow-HEAD.2nue1/venv-source/lib64/python3.11/site-packages/setuptools_scm/integration.py", line 123, in infer_version
    config = Configuration.from_file(dist_name=dist_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/arrow-HEAD.2nue1/venv-source/lib64/python3.11/site-packages/setuptools_scm/config.py", line 216, in from_file
    return cls(relative_to=name, **args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Configuration.__init__() got an unexpected keyword argument 'version_file'

Component(s)

Continuous Integration, Python

[raulcd]

Issue resolved by pull request 42007
#42007

[janosik47]

FYI - I just got hit by this error when following the official docs: https://arrow.apache.org/docs/developers/python.html#build-and-test

pushd arrow/python
export PYARROW_WITH_PARQUET=1
export PYARROW_WITH_DATASET=1
export PYARROW_PARALLEL=4
python setup.py build_ext --inplace
popd

and then

pip install wheel  # if not installed
python setup.py build_ext --build-type=$ARROW_BUILD_TYPE \
         --bundle-arrow-cpp bdist_wheel

[raulcd]

@janosik47 is this building from main or any other specific Arrow release?

[lidavidm]

This happens to me, too on main. Need to pip install --upgrade setuptools_scm - possibly conda-forge or conda doesn't have the newest version or doesn't resolve it properly?

[lidavidm]

Ah conda_env_python.txt does have a bound so perhaps the advice should be to create a new env