Add pip-sync option to restrict attention to user-local directory.

[jbergknoff-rival]

I'm setting up Python code in Docker images, setting a project-local PYTHONUSERBASE environment variable and installing packages with pip install --user .... This is instead of using a virtual environment, as described in this article.

Currently pip-tools almost works with --user-installed packages: if PYTHONUSERBASE is set then pip-sync will run, but it will look at all packages. This PR is essentially a one-line change to add support for managing packages installed in a user-local directory.

I'd appreciate advice on adding a test. It doesn't look like there's currently much coverage for how the pip-sync CLI tool operates.

Changelog-friendly one-liner: Adds pip-sync option to restrict attention to user-local directory.

Contributor checklist

• Provided the tests for the changes
• Requested (or received) a review from another contributor
• Gave a clear one-line description in the PR (that the maintainers can add to CHANGELOG.md afterwards).

[vphilippon]

Hi @jbergknoff-10e, thanks for the PR!

First-off, I'd like to point out that using pip install -r requirements.txt with a requirements.txt generated from a pip-compile would likely be sufficient in a container IMO, as I you're always starting from fresh, so you shouldn't have to uninstall any pre-existing python packages, right?

Nonetheless, this is still a good feature to have on pip-sync.
Few things though:

• We should name the flag --user to mirror pip interface (I know that this isn't consistent on all options, we'll get that done over time).
• We should pass the --user flag to the install command so that the packages are installed in the configured user local directory.

For the tests, given the nature of the change, you can skip that. Sometime, "perfect" is the enemy of "better".

[vphilippon]

LGTM 👍

[jbergknoff-rival]

Thanks! I've updated the CLI parameter to --user, though kept its name internally as user_only for clarity, and the --user flag is now passed through to pip install (pretty important!).

Tangent on usage in a container: what you say is largely true. However, during development, when it makes more sense to volume the vendor directory into a container than to build it into an image, it's convenient to have that vendor directory front and center (i.e. $(pwd)/vendor) rather than tucked away in the usual place, so I still prefer the pip install --user approach. Also, sometimes I work from a vanilla Python base image, but other times from an image with various CLI tools globally installed (e.g. piptools, bumpversion). In that case, having the --user separation is very useful.

Here's a quick rundown of how I've manually tested this change:

$ docker run -it --rm -v $(pwd):$(pwd) -w $(pwd) python:3.6.4-alpine3.7 sh
# apk add --no-cache git
# python setup.py install
# cd /tmp
# echo requests > requirements.in
# pip-compile requirements.in
...
certifi==2018.1.18        # via requests
chardet==3.0.4            # via requests
idna==2.6                 # via requests
requests==2.18.4
urllib3==1.22             # via requests

# pip install bumpversion

# pip freeze
bumpversion==0.5.3
click==6.7
first==2.0.1
pip-tools==1.11.1.dev20+g712823f
six==1.11.0

# PYTHONUSERBASE=/tmp/vendor pip freeze --user

# PYTHONUSERBASE=/tmp/vendor pip install --user flask
# PYTHONUSERBASE=/tmp/vendor pip freeze --user
Flask==0.12.2
itsdangerous==0.24
Jinja2==2.10
MarkupSafe==1.0
Werkzeug==0.14.1

# PYTHONUSERBASE=/tmp/vendor pip-sync --user
...
Successfully installed certifi-2018.1.18 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22

# pip freeze
bumpversion==0.5.3
click==6.7
first==2.0.1
pip-tools==1.11.1.dev20+g712823f
six==1.11.0

# PYTHONUSERBASE=/tmp/vendor pip freeze --user
certifi==2018.1.18
chardet==3.0.4
idna==2.6
requests==2.18.4
urllib3==1.22

So Flask and its dependencies get installed in /tmp/vendor, and bumpversion gets installed globally. The pip-compile output has just requests and its dependencies. The result of pip-sync is that global stuff (bumpversion) is left alone, Flask and its dependencies are uninstalled, and requests and its dependencies are installed in the user base directory. To me this all seems correct.

[vphilippon]

Excellent, thanks for the tests and detailed report.

[vphilippon]

Thanks for the PR!