pip 10.0.0 fails silently if it cannot install due to missing write permissions

[pv]

• Pip version: 10.0.0
• Python version: 3.6.5
• Operating system: Linux/Fedora

Description:

When installing to a site-packages directory where the current user is not able to write, pip 10.0.0 does not show an error message. It returns with nonzero exit code, but since there is no error message printed it's easy to miss the installation failed.

This is a common situation when using system-wide Python installations and forgetting to give the --user flag to pip.

This is a regression, as in this situation pip 9 prints an the error traceback that says "PermissionError: [Errno 13] Permission denied".

What I've run:

Reproducing shell script

set -e -x

chmod u+w -Rf mytest
rm -rf mytest
mkdir mytest
cd mytest

python3 -mvenv env
./env/bin/pip install --upgrade 'pip>=10'

chmod -R a-w env

if ./env/bin/pip install six; then
    echo "Pip claims installation was successful?"
else
    echo "Pip claims installation failed."
fi

produces

+ chmod u+w -Rf mytest
+ rm -rf mytest
+ mkdir mytest
+ cd mytest
+ python3 -mvenv env
+ ./env/bin/pip install --upgrade 'pip>=10'
Collecting pip>=10
  Using cached pip-10.0.0-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 9.0.3
    Uninstalling pip-9.0.3:
      Successfully uninstalled pip-9.0.3
Successfully installed pip-10.0.0
+ chmod -R u-w env
+ ./env/bin/pip install six
Collecting six
  Using cached six-1.11.0-py2.py3-none-any.whl
Installing collected packages: six

+ echo 'Pip claims installation failed.'
Pip claims installation failed.

Note that pip does not print any message saying the installation was not successful, and what's printed looks like the installation was successful, contrary to what actually happened.

[pradyunsg]

Hey @pv! Thanks for reporting this.

---

Relavant Code:

pip/src/pip/_internal/commands/install.py

Lines 361 to 380 in 28f3ce6

	except EnvironmentError as e:
	message_parts = []
	user_option_part = "Consider using the `--user` option"
	permissions_part = "Check the permissions"
	if e.errno == errno.EPERM:
	if not options.use_user_site:
	message_parts.extend([
	user_option_part, " or ",
	permissions_part.lower(),
	])
	else:
	message_parts.append(permissions_part)
	message_parts.append("\n")
	logger.error(
	"".join(message_parts), exc_info=(self.verbosity > 1)
	)
	return ERROR

FWIW:

• default message_parts should not be empty.
• if this is not causing a permission error, what is it raising? @pv Could you somehow check what's the actual error being raised in this section, including the errno?

[pv]

Adding import traceback; traceback.print_exc() shows

Traceback (most recent call last):
  File "/home/pauli/tmp/pip-bugs/mytest/env/lib64/python3.6/site-packages/pip/_internal/commands/install.py", line 335, in run
    use_user_site=options.use_user_site,
  File "/home/pauli/tmp/pip-bugs/mytest/env/lib64/python3.6/site-packages/pip/_internal/req/__init__.py", line 49, in install_given_reqs
    **kwargs
  File "/home/pauli/tmp/pip-bugs/mytest/env/lib64/python3.6/site-packages/pip/_internal/req/req_install.py", line 748, in install
    use_user_site=use_user_site, pycompile=pycompile,
  File "/home/pauli/tmp/pip-bugs/mytest/env/lib64/python3.6/site-packages/pip/_internal/req/req_install.py", line 961, in move_wheel_files
    warn_script_location=warn_script_location,
  File "/home/pauli/tmp/pip-bugs/mytest/env/lib64/python3.6/site-packages/pip/_internal/wheel.py", line 314, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/home/pauli/tmp/pip-bugs/mytest/env/lib64/python3.6/site-packages/pip/_internal/wheel.py", line 292, in clobber
    shutil.copyfile(srcfile, destfile)
  File "/usr/lib64/python3.6/shutil.py", line 121, in copyfile
    with open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: '/home/pauli/tmp/pip-bugs/mytest/env/lib/python3.6/site-packages/six.py'

so it's EACCES, not EPERM

[pradyunsg]

Thanks!

so it's EACCES, not EPERM

Indeed. Just looked it up at https://serialio.com/support/UnixErrorNumbers.htm and currently feeling stupid. :)

[pradyunsg]

@pv Could you take #5239 for a spin and confirm that it fixes this issue?

[pfmoore]

@pv will you be able to check the PR? ideally, we'd like to get this into 10.0.1.

[pv]

I'm ok with the proposed changes, lgtm from my side.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.