tests take too long to run

[grzn]

mainly because of the various repository cloning being done in the tests from the internet, mainly from GitHub.

Here are some very slow tests, for example:

• test_install_using_install_option_and_editable
• 145.65s - tests/functional/test_install_upgrade.py - test_upgrade_vcs_req_with_dist_found
• 69.90s - tests/functional/test_uninstall.py - test_uninstall_editable_with_source_outside_venv
• 21.67s - tests/functional/test_bundle.py - test_cleanup_after_create_bundle
• 17.91s - tests/functional/test_freeze.py - test_freeze_bazaar_clone
• 14.80s - tests/functional/test_help.py - test_help_commands_equally_functional
• 14.56s - tests/functional/test_install.py - test_vcs_url_urlquote_normalization
• 14.55s - tests/functional/test_install.py - test_install_editable_from_bazaar
• 12.64s - tests/functional/test_install_download.py - test_download_wheel
• 10.84s - tests/functional/test_install_config.py - test_command_line_options_override_env_vars
• 10.72s - tests/functional/test_install_reqs.py - test_multiple_requirements_files

[grzn]

Some suggestions:

• clone virtualenv once to a tempdir and have all the tests use that repository instead of cloning from GitHub over and over again
• clone a very small repository instead of virtualenv (which got pretty heavy already) with the minimal set of files (setup.py, empty python module, what else?)
• although GitHub is much slower than PyPI these days, we can set up a temp devpi-server for caching the packages

[Ivoz]

Thanks, done the *first in #1726, updated the issue to be a checklist

[grzn]

cool.
another improvement worth doing is this:

• a lot of tests call local_checkout and pass tmpdir.join("cache") as the directory. thing is that tmpdir is not shared between the tests, so there isn't really a cached directory for repository clones.

[grzn]

about test_upgrade_vcs_req_with_dist_found: why not upload the pip-test-package to pypi and then use that?

[grzn]

slowest tests on my Mac when using local devpi server:

482.88s call     tests/functional/test_install_upgrade.py::test_upgrade_vcs_req_with_dist_found
276.17s call     tests/functional/test_install.py::test_install_using_install_option_and_editable
220.59s call     tests/functional/test_uninstall.py::test_uninstall_editable_with_source_outside_venv
17.07s call     tests/functional/test_install_upgrade.py::TestUpgradeSetuptools::test_from_setuptools_7_to_setuptools_7_with_distribute_7_installed
13.11s call     tests/functional/test_install_upgrade.py::TestUpgradeSetuptools::test_py2_py3_from_distribute_6_to_setuptools_7
12.49s call     tests/functional/test_help.py::test_help_commands_equally_functional
8.56s call     tests/functional/test_freeze.py::test_freeze_bazaar_clone
7.66s call     tests/functional/test_install.py::test_vcs_url_urlquote_normalization
7.32s call     tests/functional/test_install.py::test_install_editable_from_bazaar
7.29s call     tests/functional/test_install_config.py::test_config_file_override_stack

[Ivoz]

@grzn that particular test is about testing vcs paths, not downloading from pypi

[grzn]

isn't the test installs virtualenv from a specific commit, then attempts to upgrade from the index (pypi)? we can't just replace it with pip-test-package because its not on pypi?
or I did misunderstood the test?

[grzn]

the 3 at the top take 60% of total time

[Ivoz]

Definitely, it would be great to chop pip's testing time in half or more, if these can be optimized.

[Ivoz]

@grzn oh about test_upgrade_vcs_req_with_dist_found, I didn't read it through enough. Conservatively, I think it would be preferable to find a simple package already on pypi, and keep that one on github only.

[grzn]

I prefer to build one for this test and not rely on other packages that are not controlled by pypa and that can disappear from pypi or github and then break the test

[Ivoz]

https://github.com/kennethreitz/envoy could be a good candidate

[alex]

You can check test_upgrade_vcs_req_with_dist_found off your list.

[pradyunsg]

Closing in favour of #4497.