CI: add PyPI Trusted-Publishing “publish” job to wheels workflow (#61669)

[EvMossan]

• closes ENH: Switch to trusted publishing for package upload to PyPI in CI #61669
• all code checks passed (pre-commit & CI)
• added an entry in doc/source/whatsnew/v3.0.0.rst

Summary

This PR enables Trusted Publishing (OIDC) uploads to PyPI when a GitHub release is published.

What’s changed

• .github/workflows/wheels.yml

  • adds a new publish job that
    1. downloads all wheel / sdist artifacts from upstream jobs;
    2. excludes Pyodide wheels (*pyodide*.whl);
    3. runs pypa/gh-action-pypi-publish@v1 in the pypi environment.

• doc/source/whatsnew/v3.0.0.rst

  • adds a single Build / CI line announcing the switch to Trusted Publishing

• doc/source/development/maintaining.rst

  • drop manual twine step and note Trusted Publishing

No other files or CI matrix settings were changed.

Release prerequisites

• GitHub repo must have an environment named pypi (OIDC token permission enabled).
• The pandas project on PyPI must list pandas-dev/pandas → “pypi” as a Trusted Publisher (see https://docs.pypi.org/trusted-publishers/).

[EvMossan]

CI failed in the docstring-validation step with

AttributeError: 'getset_descriptor' object has no attribute '__module__'. Did you mean: '__reduce__'?

This occurs before my code runs, so it isn’t caused by the changes in this PR.
It looks related to the latest numpydoc release.
I’ll re-run CI once the upstream fix lands.

[EpicWink]

This workflow runs every day, on all pushes, and on all pull requests, but you said "uploads to PyPI when a release tag is pushed" in the description. Perhaps you want to put this in a new publish.yml workflow file, with on: { release: { types: [published] } }.

Because this makes the download-artifacts step run in a different workflow, you'll need to figure out the run-id argument.

You could alternatively add on: { release: { types: [published] } } to the wheels.yml workflow.

---

You need to update the release process documentation to change the new method for publishing (remove step 5: "Upload wheels to PyPI").

---

You likely want to document (in this pull request's description, and in the release process documentation) the new pypi GitHub environment needs to exist, and the corresponding publisher needs to be added to the project in PyPI.

[EvMossan]

@EpicWink All requested changes are in. Let me know if anything else’s needed - thanks!

[EpicWink]

You likely want to document (in this pull request's description, and in the release process documentation) the new pypi GitHub environment needs to exist, and the corresponding publisher needs to be added to the project in PyPI.

Still want to do this, at least in this pull request's description

[EvMossan]

All comments addressed and conversations resolved - ready for another look. Thanks!

[jorisvandenbossche]

@EvMossan thanks a lot for working on this! (and @EpicWink for the review)

@mroeschke I updated the PyPI configuration to enable trusted publishing on that side.

I suppose we can only test this really once doing a next pre-release?
Could we already test the first part of the added job by temporarily commenting out the last pypa/gh-action-pypi-publish step, but so at least we can check the downloading and filtering of the artifacts is working?

[jorisvandenbossche]

@EpicWink you suggested to add this, but I am wondering if that is needed? As I understand, this wheels.yml workflow will run on the commit pushed to main with a tag, and so that way it will already run for a release? And we can then upload the wheels from that run?

[EpicWink]

A GitHub release publish happens after a push, allowing you to review the CI, and write release notes in the GitHub release, before triggering the publish to PyPI. Up to you how you want the release workflow to be, if you want to publish unconditionally.

An alternative is to split the workflow into two files (and therefore two workflows), where the publish job downloads the artefacts from the build job of the other workflow.

[jorisvandenbossche]

Suggested change

	via [**Trusted Publishing**](https://docs.pypi.org/trusted-publishers/)
	via `**Trusted Publishing** <https://docs.pypi.org/trusted-publishers/>`__)

(this file is still in rst, not markdown)

[jorisvandenbossche]

Is this line needed? (it's not included in the example in https://docs.pypi.org/trusted-publishers/using-a-publisher/)

[EpicWink]

It's not necessary, and as far as I know there's no recommendation to hard-code it. When the upload API 2.0 PEP lands, in site this URL and action will change anyway.

[EpicWink]

I suppose we can only test this really once doing a next pre-release?

Yes, fortunately PyPI sports uploading pre-releases.

Could we already test the first part of the added job by temporarily commenting out the last pypa/gh-action-pypi-publish step, but so at least we can check the downloading and filtering of the artifacts is working?

That could work, as long as it's in a protected tag, but you would be creating a tag just to test this, unless you change the workflow definition further to support a non-version-tag ref.