uniquify paths in PYTHONPATH

[robnagler]

When running in a complex environment with lots of installed
packages, PYTHONPATH gets way too long. Instead, just make sure
that paths_on_pythonpath doesn't contain duplicates

I don't expect this to be accepted. Just letting you know there's a problem here and how to fix it.

[jaraco]

After some investigation, I can see that the bug lies here, where setuptools is attempting to make sure that every package that was just installed has its path on PYTHONPATH, so that any subprocess invocation also has those packages available for import.

The fact that all of these paths are the same, however, is a little surprising. These paths would be installed by easy_install, which should provide a unique location for each package installed. If these paths are in fact always just the site-packages directory, perhaps this code does not have its intended effect.

Since this code is part of the pending-deprecated test command, I'm going to save trouble investigating the root cause and simply adopt a technique similar to the one you've proposed here.

[robnagler]

Thanks @jaraco for taking a look at this.

These paths would be installed by easy_install, which should provide a unique location for each package installed.

This seems like this approach would be problematic in complex installations, like ours. Why is this necessary?

[jaraco]

That’s just how easy_install works. Each package is meant to be an encapsulated egg, such that uninstallation is simply an rm operation (similar to how macOS installs Applications). To make those packages live on the python path, easy_install would have a .pth file to add these paths... but for these on-demand test dependencies, another technique was needed.

I think one motivation for moving away from easy_install was to avoid the essential performance hit of each package having its own path.