Skip to content

Commit

Permalink
Merge pull request #10899 from pradyunsg/docs/more-reference
Browse files Browse the repository at this point in the history
  • Loading branch information
@pradyunsg
pradyunsg authored Feb 26, 2022
2 parents 6d1c027 + 8c7490c commit 82aebdb
Show file tree
Hide file tree
Showing 8 changed files with 299 additions and 325 deletions.
311 changes: 9 additions & 302 deletions docs/html/cli/pip_install.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -154,91 +154,15 @@ Requirements File Format

This section has been moved to :doc:`../reference/requirements-file-format`.

.. _`Requirement Specifiers`:

Requirement Specifiers
----------------------

pip supports installing from a package index using a :term:`requirement
specifier <pypug:Requirement Specifier>`. Generally speaking, a requirement
specifier is composed of a project name followed by optional :term:`version
specifiers <pypug:Version Specifier>`. :pep:`508` contains a full specification
of the format of a requirement. Since version 18.1 pip supports the
``url_req``-form specification.

Some examples:

::

SomeProject
SomeProject == 1.3
SomeProject >=1.2,<2.0
SomeProject[foo, bar]
SomeProject~=1.4.2

Since version 6.0, pip also supports specifiers containing `environment markers
<https://www.python.org/dev/peps/pep-0508/#environment-markers>`__ like so:

::

SomeProject ==5.4 ; python_version < '3.8'
SomeProject; sys_platform == 'win32'

Since version 19.3, pip also supports `direct references
<https://www.python.org/dev/peps/pep-0440/#direct-references>`__ like so:

::

SomeProject @ file:///somewhere/...

Environment markers are supported in the command line and in requirements files.

.. note::

Use quotes around specifiers in the shell when using ``>``, ``<``, or when
using environment markers. Don't use quotes in requirement files. [1]_


.. _`Per-requirement Overrides`:
This section has been moved to :doc:`../reference/requirement-specifiers`.

Per-requirement Overrides
-------------------------

Since version 7.0 pip supports controlling the command line options given to
``setup.py`` via requirements files.

.. warning::

This disables the use of wheels (cached or otherwise).

The ``--global-option`` and ``--install-option`` options are used to pass
options to ``setup.py``. For example:

::

FooProject >= 1.2 --global-option="--no-user-cfg" \
--install-option="--prefix='/usr/local'" \
--install-option="--no-compile"

The above translates roughly into running FooProject's ``setup.py``
script as:

::

python setup.py --no-user-cfg install --prefix='/usr/local' --no-compile

Note that the only way of giving more than one option to ``setup.py``
is through multiple ``--global-option`` and ``--install-option``
options, as shown in the example above. The value of each option is
passed as a single argument to the ``setup.py`` script. Therefore, a
line such as the following is invalid and would result in an
installation error.

::

# Invalid. Please use '--install-option' twice as shown above.
FooProject >= 1.2 --install-option="--prefix=/usr/local --no-compile"

This is now covered in :doc:`../reference/requirements-file-format`.

.. _`Pre Release Versions`:

Expand Down Expand Up @@ -315,232 +239,20 @@ Wheel Cache

This is now covered in :doc:`../topics/caching`.

.. _`hash-checking mode`:

Hash-Checking Mode
Hash checking mode
------------------

Since version 8.0, pip can check downloaded package archives against local
hashes to protect against remote tampering. To verify a package against one or
more hashes, add them to the end of the line::

FooProject == 1.2 --hash=sha256:2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 \
--hash=sha256:486ea46224d1bb4fb680f34f7c9ad96a8f24ec88be73ea8e5a6c65260e9cb8a7

(The ability to use multiple hashes is important when a package has both
binary and source distributions or when it offers binary distributions for a
variety of platforms.)

The recommended hash algorithm at the moment is sha256, but stronger ones are
allowed, including all those supported by ``hashlib``. However, weaker ones
such as md5, sha1, and sha224 are excluded to avoid giving a false sense of
security.

Hash verification is an all-or-nothing proposition. Specifying a ``--hash``
against any requirement not only checks that hash but also activates a global
*hash-checking mode*, which imposes several other security restrictions:

* Hashes are required for all requirements. This is because a partially-hashed
requirements file is of little use and thus likely an error: a malicious
actor could slip bad code into the installation via one of the unhashed
requirements. Note that hashes embedded in URL-style requirements via the
``#md5=...`` syntax suffice to satisfy this rule (regardless of hash
strength, for legacy reasons), though you should use a stronger
hash like sha256 whenever possible.
* Hashes are required for all dependencies. An error results if there is a
dependency that is not spelled out and hashed in the requirements file.
* Requirements that take the form of project names (rather than URLs or local
filesystem paths) must be pinned to a specific version using ``==``. This
prevents a surprising hash mismatch upon the release of a new version
that matches the requirement specifier.
* ``--egg`` is disallowed, because it delegates installation of dependencies
to setuptools, giving up pip's ability to enforce any of the above.

.. _`--require-hashes`:

Hash-checking mode can be forced on with the ``--require-hashes`` command-line
option:

.. tab:: Unix/macOS

.. code-block:: console
$ python -m pip install --require-hashes -r requirements.txt
...
Hashes are required in --require-hashes mode (implicitly on when a hash is
specified for any package). These requirements were missing hashes,
leaving them open to tampering. These are the hashes the downloaded
archives actually had. You can add lines like these to your requirements
files to prevent tampering.
pyelasticsearch==1.0 --hash=sha256:44ddfb1225054d7d6b1d02e9338e7d4809be94edbe9929a2ec0807d38df993fa
more-itertools==2.2 --hash=sha256:93e62e05c7ad3da1a233def6731e8285156701e3419a5fe279017c429ec67ce0
.. tab:: Windows

.. code-block:: console
C:\> py -m pip install --require-hashes -r requirements.txt
...
Hashes are required in --require-hashes mode (implicitly on when a hash is
specified for any package). These requirements were missing hashes,
leaving them open to tampering. These are the hashes the downloaded
archives actually had. You can add lines like these to your requirements
files to prevent tampering.
pyelasticsearch==1.0 --hash=sha256:44ddfb1225054d7d6b1d02e9338e7d4809be94edbe9929a2ec0807d38df993fa
more-itertools==2.2 --hash=sha256:93e62e05c7ad3da1a233def6731e8285156701e3419a5fe279017c429ec67ce0
This can be useful in deploy scripts, to ensure that the author of the
requirements file provided hashes. It is also a convenient way to bootstrap
your list of hashes, since it shows the hashes of the packages it fetched. It
fetches only the preferred archive for each package, so you may still need to
add hashes for alternatives archives using :ref:`pip hash`: for instance if
there is both a binary and a source distribution.

The :ref:`wheel cache <Wheel cache>` is disabled in hash-checking mode to
prevent spurious hash mismatch errors. These would otherwise occur while
installing sdists that had already been automatically built into cached wheels:
those wheels would be selected for installation, but their hashes would not
match the sdist ones from the requirements file. A further complication is that
locally built wheels are nondeterministic: contemporary modification times make
their way into the archive, making hashes unpredictable across machines and
cache flushes. Compilation of C code adds further nondeterminism, as many
compilers include random-seeded values in their output. However, wheels fetched
from index servers are the same every time. They land in pip's HTTP cache, not
its wheel cache, and are used normally in hash-checking mode. The only downside
of having the wheel cache disabled is thus extra build time for sdists, and
this can be solved by making sure pre-built wheels are available from the index
server.

Hash-checking mode also works with :ref:`pip download` and :ref:`pip wheel`.
See :doc:`../topics/repeatable-installs` for a comparison of hash-checking mode
with other repeatability strategies.

.. warning::

Beware of the ``setup_requires`` keyword arg in :file:`setup.py`. The
(rare) packages that use it will cause those dependencies to be downloaded
by setuptools directly, skipping pip's hash-checking. If you need to use
such a package, see :ref:`Controlling
setup_requires <controlling-setup_requires>`.

.. warning::

Be careful not to nullify all your security work when you install your
actual project by using setuptools directly: for example, by calling
``python setup.py install``, ``python setup.py develop``, or
``easy_install``. Setuptools will happily go out and download, unchecked,
anything you missed in your requirements file—and it’s easy to miss things
as your project evolves. To be safe, install your project using pip and
:ref:`--no-deps <install_--no-deps>`.

Instead of ``python setup.py develop``, use...

.. tab:: Unix/macOS

.. code-block:: shell
python -m pip install --no-deps -e .
This is now covered in :doc:`../topics/secure-installs`.

.. tab:: Windows

.. code-block:: shell
py -m pip install --no-deps -e .
Instead of ``python setup.py install``, use...

.. tab:: Unix/macOS

.. code-block:: shell
python -m pip install --no-deps .
.. tab:: Windows

.. code-block:: shell
py -m pip install --no-deps .
Hashes from PyPI
^^^^^^^^^^^^^^^^

PyPI provides an MD5 hash in the fragment portion of each package download URL,
like ``#md5=123...``, which pip checks as a protection against download
corruption. Other hash algorithms that have guaranteed support from ``hashlib``
are also supported here: sha1, sha224, sha384, sha256, and sha512. Since this
hash originates remotely, it is not a useful guard against tampering and thus
does not satisfy the ``--require-hashes`` demand that every package have a
local hash.


Local project installs
Local Project Installs
----------------------

pip supports installing local project in both regular mode and editable mode.
You can install local projects by specifying the project path to pip:

.. tab:: Unix/macOS

.. code-block:: shell
python -m pip install path/to/SomeProject
.. tab:: Windows

.. code-block:: shell
This is now covered in :doc:`../topics/local-project-installs`.

py -m pip install path/to/SomeProject
.. note::

Depending on the build backend used by the project, this may generate
secondary build artifacts in the project directory, such as the
``.egg-info`` and ``build`` directories in the case of the setuptools
backend.

Pip has a legacy behaviour that copies the entire project directory to a
temporary location and installs from there. This approach was the cause of
several performance and correctness issues, so it is now disabled by
default, and it is planned that pip 22.1 will remove it.

To opt in to the legacy behavior, specify the
``--use-deprecated=out-of-tree-build`` option in pip's command line.


.. _`editable-installs`:

"Editable" Installs
^^^^^^^^^^^^^^^^^^^

"Editable" installs are fundamentally `"setuptools develop mode"
<https://setuptools.readthedocs.io/en/latest/userguide/development_mode.html>`_
installs.

You can install local projects or VCS projects in "editable" mode:

.. tab:: Unix/macOS

.. code-block:: shell
python -m pip install -e path/to/SomeProject
python -m pip install -e git+http://repo/my_project.git#egg=SomeProject
.. tab:: Windows

.. code-block:: shell
py -m pip install -e path/to/SomeProject
py -m pip install -e git+http://repo/my_project.git#egg=SomeProject
(See the :doc:`../topics/vcs-support` section above for more information on VCS-related syntax.)

For local projects, the "SomeProject.egg-info" directory is created relative to
the project path. This is one advantage over just using ``setup.py develop``,
which creates the "egg-info" directly relative the current working directory.
Editable installs
-----------------

This is now covered in :doc:`../topics/local-project-installs`.

Build System Interface
----------------------
Expand Down Expand Up @@ -834,10 +546,5 @@ Examples
py -m pip install SomePackage1 SomePackage2 --no-binary SomePackage1
----

.. [1] This is true with the exception that pip v7.0 and v7.0.1 required quotes
around specifiers containing environment markers in requirement files.
.. _extras: https://www.python.org/dev/peps/pep-0508/#extras
.. _PyPI: https://pypi.org/
1 change: 1 addition & 0 deletions docs/html/reference/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,6 @@ interoperability standards that pip utilises/implements.
:titlesonly:
build-system/index
requirement-specifiers
requirements-file-format
```
Loading

0 comments on commit 82aebdb

Please sign in to comment.