Support requirements files from pip-compile

[Zac-HD]

pip-compile is a lovely tool for managing pinned dependencies, similar to a Pipfile (see #197), with the advantage that it works with standard tooling. We check that pip-compile is a no-op in CI, meaning that our pinned requirements are always consistent and up-to-date.

Unfortunately, this can break in several ways (HypothesisWorks/hypothesis#747):

• If the requirements files pins A and B, such that A depends on B up to a certain version, (eg B>=1,<2) Pyup will update B beyond the acceptable range anyway. This is related to Support version ranges natively #203 and possibly Pin packages with >= ranges? #50, albeit with very different motivation - we want to pin everything to a specific version, but understanding downstream dependency ranges is required to do so correctly.

• pip-compile output files have a specific format (and header), including fixed-column comments identifying the source(s) of transitive dependencies. This is very useful, but means that compiling is not a no-op when the length of the version string changes, because the amount of whitespace changes.

It would be great for Hypothesis if Pyup supported a "use pip-tools" option, or simply if these two pain points could be fixed.

[jayfk]

If the requirements files pins A and B, such that A depends on B up to a certain version, (eg B>=1,<2) Pyup will update B beyond the acceptable range anyway. This is related to #203 and possibly #50, albeit with very different motivation - we want to pin everything to a specific version, but understanding downstream dependency ranges is required to do so correctly.

The main problem with pip-compile (and with Python dependencies in general) is that you have to install the package in order to resolve the dependency tree. In a lot of cases this means running its setup.py, opening the service for remote code execution. This is something I really want to avoid as it adds a lot of complexity to do this in a secure way.

But, there's currently a lot going on in Python packaging in general that might be able to resolve this issue.

pip-compile output files have a specific format (and header), including fixed-column comments identifying the source(s) of transitive dependencies. This is very useful, but means that compiling is not a no-op when the length of the version string changes, because the amount of whitespace changes.

This looks like a bug :). The bot tries to keep the original whitespace between the version and the comment as it was prior to the update: https://github.com/pyupio/dparse/blob/master/dparse/updater.py#L31-L37

I've opened a new issue here: pyupio/dparse#15

[Zac-HD]

This looks like a bug :). The bot tries to keep the original whitespace between the version and the comment as it was prior to the update: https://github.com/pyupio/dparse/blob/master/dparse/updater.py#L31-L37

That's the problem - if the version goes from 1.2.9 to 1.2.10, the same number of spaces leaves the comment in a different column. So it's working as intended, but I'd like it to work differently 😄

(also, the whitespace string gets reversed, which matters for mixed tabs-and-spaces, and just above you could requirement, comment = dependency.line.split("#", maxsplit=1) in a single line)

easy dependency resolution can lead to remote code execution, which is bad

You'll probably have to do this eventually - Hypothesis has had at least one incompatible update of a transitive dependency - but I'm certainly sympathetic. Better slow and careful than pwned!

[Zac-HD]

Closing as stale; these days I use pip-compile in a scheduled github action instead.