Workflow Run Failure when installing Python packages: "error: externally-managed-environment"

[sayhiben]

Select Topic Area

Bug

Body

Overview

If you've encountered workflow run failures on ubuntu-latest runners starting on Sept. 23, 2024 (2024-09-23), and you're seeing errors while installing Python packages, it is due to a recent version bump to ubuntu-latest. The error appears as follows:

Run pip3 install pytest
  pip3 install pytest
  shell: /usr/bin/bash -e {0}
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
Error: Process completed with exit code 1.

Notes

I have not yet been able to locate any form of deprecation notice from GitHub leading up to this change, although there is a related Changelog post on the GitHub Blog published two days after the change began to be rolled out on Sept. 23.

Here's GitHub's related actions/runner-images issue, where they made this unfortunately incorrect evaluation:

On GitHub Actions, actions/setup-python can install any version on-flight so this change doesn't impact users

Solution

As noted in both the error message and GitHub's issue, to resolve this error, you will need to setup Python within your workflow and avoid using the system Python if you require additional packages. Here are the steps to fix the failing jobs in your workflow file(s):

1. Before any pip install.. steps in your workflows, run actions/setup-python:

   - uses: actions/setup-python@v5
     with:
         python-version: '3.x' 

2. Update any subsequent pip3 ... or python3 ... commands to use pip ... or python ... (remove the 3)

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[mkesper]

Alternatively, build a venv with the installed Python and activate it before running any pip / Python commands.