Reasoning behind unsafe packages

[elaichi]

Why exactly is it that some packages are considered unsafe in a requirements file?

I first noticed the comment after using pip-compile and then found this set in piptools/utils.py:

UNSAFE_PACKAGES = {'setuptools', 'distribute', 'pip'}

I tried googling for an answer but no luck.

Environment Versions

1. OS Type
2. Python version: $ python -V — 2.7.13
3. pip version: $ pip --version — 9.0.1
4. pip-tools version: $ pip-compile --version — 1.9.0

Steps to replicate

❯ pip-compile --dry-run <(echo "pip")
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --output-file /dev/fd/11.txt /dev/fd/11
#

# The following packages are considered to be unsafe in a requirements file:
pip==9.0.1
Dry-run, so nothing updated.

---

This is clearly not an actual issue but just a question :)

[vphilippon]

That was brought a while agot, by this commit:
50f06cb

And I'll admit I'd really like to have @nvie input on this.

The only reasons I could think of would be one of those:

1. Some packages, like pip, can be installed with a package manager on linux, and could cause some conflict, but I never tested (I'm mostly on Windows these days).
2. Those are setup/installation packages, and most app (not all though) don't need to pin them.
3. When doing pip freeze, some of the installed packages are not listed, unless --all is specified. Maybe the original intent was to follow that.

[simon-weber]

I'm interested in this too. I've got two guesses:

• upgrading pip/etc at the same time as upgrading other packages might cause issues (though I haven't actually tested this)
• this has something to do with pip usually being globally installed by a system package manager, and is a band-aid for the conflict described in pip-sync using global pip inside virtualenvs on Ubuntu #328

Am I on the right track?

[vphilippon]

Coming back on this:

pip, setuptools and distribute and responsible for building/installing packages. Generally speaking, one shouldn't have to pin these as they should have little impact on an application. (I've actually met quite a few exceptions to this assumption in my daily work, but I'll go ahead and hope that I just happen to be an unlucky fellow doing some funky things.)

For, PACKAGES_TO_IGNORE, it seems to be to cover some edged cases with global packages. for pip, setuptools and wheel, I would say it's to avoid issues while pip-sync is running. For pip-tools, that's just to be keep the tool installed. For pip-review, I think its a simple exclusion for this tool that is used to track packages upgrade and might be interesting to use alongside pip-tools`.

I'll close this issue, as it seems there's not much else we can add other than guessing. If this cause any other issue in the future, we can spend the time to figure out the exact needs and impact.