All packages with CLI failing with '/home/runner/.local/bin' which is not on PATH #99

heitorlessa · 2020-06-02T08:33:30Z

Describe the bug

Packages entrypoints/CLI are not available in PATH e.g. flake8, poetry, pytest.

When investigating Github Action logs I can see two noticeable warnings that explains the issue:

Defaulting to user installation because normal site-packages is not writeable
WARNING: The script flake8 is installed in '/home/runner/.local/bin' which is not on PATH.

As I have another repo using this same action, and it is working just fine, I wonder if something happened recently to cause this - Virtualenv manipulation, PATH manipulation, etc.

Which version of the action are you using?

Tried both - same issue

v1
v2
Some other tag (such as v2.0.1 or master)

Environment

self-hosted
Linux (ubuntu latest)
Windows
Mac

If applicable, please specify if you're using a container

Python Versions
Please list all of the effected versions of Python (3.8.2, etc.)

3.6, 3.7, 3.8

To Reproduce

Create a new Github action using the following steps:

    steps:
      - uses: actions/checkout@v1
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v1
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          pip install --upgrade pip poetry
          poetry install

It will fail when it attempts to run poetry command. Same happens with any other package.

Run/Repo Url
If applicable, and if your repo/run is public, please include a URL so it is easier for us to investigate.

https://github.com/aws-samples/cookiecutter-aws-sam-pipeline/pull/10/checks?check_run_id=730151630

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context

PATH content - Captured via tmate session

echo $PATH
/opt/hostedtoolcache/Python/3.6.10/x64/bin:/opt/hostedtoolcache/Python/3.6.10/x64:/usr/share/rust/.cargo/bin:/home/runner/.config/composer/vendor/bin:/home/runner/.dotnet/tools:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/runner/.dotnet/tools

Debug logs

Defaulting to user installation because normal site-packages is not writeable
Requirement already up-to-date: pip in /opt/hostedtoolcache/Python/3.6.10/x64/lib/python3.6/site-packages (20.1.1)
runner@fv-az78:~/work/cookiecutter-aws-sam-pipeline/cookiecutter-aws-sam-pipeline$ pip install flake8
Defaulting to user installation because normal site-packages is not writeable
Collecting flake8
  Downloading flake8-3.8.2-py2.py3-none-any.whl (72 kB)
     |████████████████████████████████| 72 kB 1.7 MB/s
Collecting mccabe<0.7.0,>=0.6.0
  Downloading mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)
Requirement already satisfied: importlib-metadata; python_version < "3.8" in /home/runner/.local/lib/python3.6/site-packages (from flake8) (1.1.3)
Collecting pycodestyle<2.7.0,>=2.6.0a1
  Downloading pycodestyle-2.6.0-py2.py3-none-any.whl (41 kB)
     |████████████████████████████████| 41 kB 796 kB/s
Collecting pyflakes<2.3.0,>=2.2.0
  Downloading pyflakes-2.2.0-py2.py3-none-any.whl (66 kB)
     |████████████████████████████████| 66 kB 9.7 MB/s
Requirement already satisfied: zipp>=0.5 in /home/runner/.local/lib/python3.6/site-packages (from importlib-metadata; python_version < "3.8"->flake8) (3.1.0)
Installing collected packages: mccabe, pycodestyle, pyflakes, flake8
  WARNING: The script pycodestyle is installed in '/home/runner/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script pyflakes is installed in '/home/runner/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script flake8 is installed in '/home/runner/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

Add any other context about the problem here.

The text was updated successfully, but these errors were encountered:

reillysiemens · 2020-10-16T21:08:40Z

For what it's worth, I think I ran into a similar issue. Looks like for some reason Python 2 is being used in the container even when I specified another version. Maybe that's related? 🤔

reillysiemens · 2020-10-16T21:21:18Z

Ah. Please disregard my earlier comment. Turns out I'd made a typo in my action. I pluralized my matrix with

python-versions: [3.6, 3.7, 3.8, 3.9]

instead of

python-version: [3.6, 3.7, 3.8, 3.9]

I imagine that causes ${{ matrix.python-version }} to be empty then, so Python 2.7 was the default python. 🤷‍♂️

indirectlylit · 2020-11-04T06:02:35Z

I'm reproducing a very similar issue. I have the same Python action running in two repos, and it succeeds in one while failing in another.

In this case I'm trying to run flake8, but get the same "command not found" issue with black and pytest:

name: Python tests
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.6]
    steps:
    - uses: actions/checkout@v2
    - run: |
        # Hack to get setup-python to work on act
        # https://github.com/nektos/act/issues/251#issuecomment-706412948
        if [ ! -f "/etc/lsb-release" ] ; then
          echo "DISTRIB_RELEASE=18.04" > /etc/lsb-release
        fi
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install flake8
    - name: Lint with flake8
      run: |
        flake8 .

In the repo where this doesn't work, it gives the warning:

WARNING: The script flake8 is installed in '/opt/hostedtoolcache/Python/3.6.12/x64/lib/my_project' which is not on PATH.

In the repo where it does work, there is no warning.

Interestingly, I can reproduce this both in the github runner, and also locally using the act local Docker runner on mac. This implies that there's something about the structure of the two repos which is causing different behaviors. Will try to narrow it down to a reproducible scenario.

indirectlylit · 2020-11-04T07:42:19Z

Update:

I've narrowed down the issue to a setup.cfg file in the root directory of the affected repo. There was a config like:

[install]
install-scripts=...

which was affecting the location to which pip installed the commands

@heitorlessa looks like your repo might have a similar issue:

https://github.com/aws-samples/cookiecutter-aws-sam-pipeline/blob/f50dfee4f1dd2e618bd3c23098ce0e8b679d5c77/setup.cfg#L1-L2

[install]
prefix=

A workaround is to cd out of the repo directory before installing dependencies:

    - name: Install dependencies
      run: |
        cd /
        python -m pip install --upgrade pip
        pip install flake8

actions/setup-python#99

abitrolly · 2021-01-15T15:33:31Z

Same here. /home/runner/.local/bin is not added to $PATH. This dir is not present on Fedora 33 either, but this doesn't really make sense for a build system.

  Creating /home/runner/.local/bin
  changing mode of /home/runner/.local/bin/coverage to 755
  changing mode of /home/runner/.local/bin/coverage-3.8 to 755
  changing mode of /home/runner/.local/bin/coverage3 to 755

Also trying newer Ubuntu 20.04 as a workaround of this PATH issue actions/setup-python#99

actions/setup-python#99

januszm · 2021-03-04T15:15:39Z

It may happen in the case where python dependencies are mistakingly referenced BEFORE Set Up Python step, i.e. /home/runner/.local/bin not in PATH error will happen with the following:

    - name: Install dependencies
      run: |
        sudo apt-get -y install ...stuff...
        python -m pip install --upgrade pipenv
        pipenv sync --dev
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}

But when you install Python dependencies after setting-up Python it'll be OK:

    - name: Install dependencies
      run: sudo apt-get -y ...stuff...
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install Python dependencies
      run: |
        python -m pip install --upgrade pipenv
        pipenv sync --dev

I just tested with ubuntu-latest and matrix: python-version: ['3.9'] and it works fine.

nikita-bykov · 2021-12-10T09:21:07Z

Hello @heitorlessa! We apologize for such a late reply.
Is this issue still relevant to you? I tried to reproduce it and everything works for me as expected.

nikita-bykov · 2021-12-17T10:14:37Z

Hello @heitorlessa, I'm going to close the issue because we haven't received a response from you.
If the issue is still relevant, feel free to contact us.
Thank you!

heitorlessa · 2021-12-17T10:39:36Z

Yes please do. I moved away from trying again as I couldn't reproduce it either so killed the project

heitorlessa added the bug Something isn't working label Jun 2, 2020

heitorlessa mentioned this issue Jun 2, 2020

Feat: Github actions as CI aws-samples/cookiecutter-aws-sam-pipeline#10

Draft

abitrolly added a commit to yakshaveinc/python that referenced this issue Jan 15, 2021

Try Ubuntu 20.04 for pip install workaround

ed64fad

actions/setup-python#99

abitrolly mentioned this issue Jan 15, 2021

Add /home/runner/.local/bin to PATH on Linux actions/runner-images#2455

Closed

7 tasks

abitrolly added a commit to yakshaveinc/python that referenced this issue Jan 17, 2021

Install coverage as a separate step

0e0bcf3

Also trying newer Ubuntu 20.04 as a workaround of this PATH issue actions/setup-python#99

suzuki-shunsuke added a commit to suzuki-shunsuke/ansible-gvm that referenced this issue Jan 23, 2021

ci: add '/home/runner/.local/bin' to PATH

d84aa40

actions/setup-python#99

tgmachina mentioned this issue Mar 24, 2021

Add GitHub actions for testing LLNL/gitlab-runner-auth#10

Merged

pnkraemer mentioned this issue Mar 26, 2021

Fix JAX-related issues in CI probabilistic-numerics/probnum#367

Merged

forest mentioned this issue Mar 26, 2021

Unable to locate executable file: gigalixir gigalixir/gigalixir-action#32

Closed

nikita-bykov closed this as completed Dec 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

All packages with CLI failing with '/home/runner/.local/bin' which is not on PATH #99

All packages with CLI failing with '/home/runner/.local/bin' which is not on PATH #99

heitorlessa commented Jun 2, 2020

reillysiemens commented Oct 16, 2020

reillysiemens commented Oct 16, 2020

indirectlylit commented Nov 4, 2020

indirectlylit commented Nov 4, 2020 •

edited

Loading

abitrolly commented Jan 15, 2021

januszm commented Mar 4, 2021

nikita-bykov commented Dec 10, 2021

nikita-bykov commented Dec 17, 2021

heitorlessa commented Dec 17, 2021

All packages with CLI failing with '/home/runner/.local/bin' which is not on PATH #99

All packages with CLI failing with '/home/runner/.local/bin' which is not on PATH #99

Comments

heitorlessa commented Jun 2, 2020

reillysiemens commented Oct 16, 2020

reillysiemens commented Oct 16, 2020

indirectlylit commented Nov 4, 2020

indirectlylit commented Nov 4, 2020 • edited Loading

abitrolly commented Jan 15, 2021

januszm commented Mar 4, 2021

nikita-bykov commented Dec 10, 2021

nikita-bykov commented Dec 17, 2021

heitorlessa commented Dec 17, 2021

indirectlylit commented Nov 4, 2020 •

edited

Loading