Pex 1.4.6 builds a failing pex #550

DrumMachineLearning · 2018-09-20T22:37:44Z

pex 1.4.6 will allow you to build a pex against the responses library, but blows up when you try to actually import it.

This works fine outside of a pex environment and with pex 1.4.5.

 ~/sandbox $ . venv/bin/activate
(venv)  ~/sandbox $ python -V
Python 2.7.15
(venv)  ~/sandbox $ pex --version
pex 1.4.6
(venv)  ~/sandbox $ pex responses
Python 2.7.15 (default, May  1 2018, 16:44:08)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import responses
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpReh1br/.deps/responses-0.9.0-py2.py3-none-any.whl/responses.py", line 41, in <module>
    import mock as std_mock
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpReh1br/.deps/mock-2.0.0-py2.py3-none-any.whl/mock/__init__.py", line 2, in <module>
    import mock.mock as _mock
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpReh1br/.deps/mock-2.0.0-py2.py3-none-any.whl/mock/mock.py", line 71, in <module>
    _v = VersionInfo('mock').semantic_version()
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpReh1br/.deps/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 462, in semantic_version
    self._semantic = self._get_version_from_pkg_resources()
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpReh1br/.deps/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 439, in _get_version_from_pkg_resources
    import pkg_resources
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpReh1br/.bootstrap/pkg_resources/__init__.py", line 77, in <module>
    from . import py31compat
ImportError: cannot import name py31compat

The text was updated successfully, but these errors were encountered:

jsirois · 2018-09-21T03:26:43Z

I don't repro. I'm pretty sure this is going to hinge on a setuptools version. If you could provide more details about your environment I'd be grateful.

My test:

$ venv -2 -c pex1.4.6
New python executable in /home/jsirois/.venv/pex1.4.6/bin/python2
Also creating executable in /home/jsirois/.venv/pex1.4.6/bin/python
Installing setuptools, pip, wheel...done.
/home/jsirois/.venv/pex1.4.6
$ venv_activate pex1.4.6
(pex1.4.6) $ python --version
Python 2.7.15
(pex1.4.6) $ pip install pex
Collecting pex
  Using cached https://files.pythonhosted.org/packages/57/1f/892b5e810df3435fd3ef18f78c9cdffb5df80cb4c5bf2b2bd8a1d3c8b590/pex-1.4.6-py2.py3-none-any.whl
Requirement already satisfied: wheel<0.32,>=0.26.0 in /home/jsirois/.venv/pex1.4.6/lib/python2.7/site-packages (from pex) (0.31.1)
Requirement already satisfied: setuptools!=34.*,!=35.*,<41,>=20.3 in /home/jsirois/.venv/pex1.4.6/lib/python2.7/site-packages (from pex) (40.4.1)
Installing collected packages: pex
Successfully installed pex-1.4.6
(pex1.4.6) $ pip list
Package    Version
---------- -------
pex        1.4.6  
pip        18.0   
setuptools 40.4.1 
wheel      0.31.1 
(pex1.4.6) $ which pex
/home/jsirois/.venv/pex1.4.6/bin/pex
(pex1.4.6) $ pex --version
pex 1.4.6
(pex1.4.6) $ pex responses
Python 3.7.0 (default, Sep 15 2018, 19:13:07) 
[GCC 8.2.1 20180831] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import responses
>>> 
now exiting InteractiveConsole...
$ venv_activate pex1.4.6
(pex1.4.6) $ pex --python=python2.7 responses
Python 2.7.15 (default, Jun 27 2018, 13:05:28) 
[GCC 8.1.1 20180531] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import responses
>>> 
$ venv_activate pex1.4.6

zsimic · 2018-09-21T05:13:31Z

This started happening for me too just now, with pex 1.4.6 (was not an issue before).

I'm on OSX 10.13.6. This happens with system python (not with more modern installations). I use pyenv, but package with system python to be able to deliver my apps with just whatever system python is.

I've installed pip (a long time ago, and not much else), the list below shows what's on my "system python" currently (notable setuptools==18.5 and pip==9.0.1)

~: /usr/local/bin/pip freeze --all
altgraph==0.10.2
bdist-mpkg==0.5.0
bonjour-py==0.3
macholib==1.5.1
matplotlib==1.3.1
modulegraph==0.10.4
numpy==1.8.0rc1
pip==9.0.1
pluggy==0.6.0
py==1.5.2
py2app==0.7.3
pyobjc-core==2.5.1
pyobjc-framework-Accounts==2.5.1
pyobjc-framework-AddressBook==2.5.1
pyobjc-framework-AppleScriptKit==2.5.1
pyobjc-framework-AppleScriptObjC==2.5.1
pyobjc-framework-Automator==2.5.1
pyobjc-framework-CFNetwork==2.5.1
pyobjc-framework-Cocoa==2.5.1
pyobjc-framework-Collaboration==2.5.1
pyobjc-framework-CoreData==2.5.1
pyobjc-framework-CoreLocation==2.5.1
pyobjc-framework-CoreText==2.5.1
pyobjc-framework-DictionaryServices==2.5.1
pyobjc-framework-EventKit==2.5.1
pyobjc-framework-ExceptionHandling==2.5.1
pyobjc-framework-FSEvents==2.5.1
pyobjc-framework-InputMethodKit==2.5.1
pyobjc-framework-InstallerPlugins==2.5.1
pyobjc-framework-InstantMessage==2.5.1
pyobjc-framework-LatentSemanticMapping==2.5.1
pyobjc-framework-LaunchServices==2.5.1
pyobjc-framework-Message==2.5.1
pyobjc-framework-OpenDirectory==2.5.1
pyobjc-framework-PreferencePanes==2.5.1
pyobjc-framework-PubSub==2.5.1
pyobjc-framework-QTKit==2.5.1
pyobjc-framework-Quartz==2.5.1
pyobjc-framework-ScreenSaver==2.5.1
pyobjc-framework-ScriptingBridge==2.5.1
pyobjc-framework-SearchKit==2.5.1
pyobjc-framework-ServiceManagement==2.5.1
pyobjc-framework-Social==2.5.1
pyobjc-framework-SyncServices==2.5.1
pyobjc-framework-SystemConfiguration==2.5.1
pyobjc-framework-WebKit==2.5.1
pyOpenSSL==0.13.1
pyparsing==2.0.1
python-dateutil==1.5
pytz==2013.7
scipy==0.13.0b1
setuptools==18.5
six==1.4.1
vboxapi==1.0
virtualenv==15.1.0
xattr==0.6.4
zope.interface==4.1.1
You are using pip version 9.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

jsirois · 2018-09-21T05:22:38Z

Pex explicitly requires setuptools>=20.3 and your output shows 18.5:
https://github.com/pantsbuild/pex/blob/64d2cd30b7e8e72a2bb60918237ad3f7d65395ff/pex/version.py#L8

Are you sure pip installation of pex doesn't issue a warning about this? Can you try installing in a fresh virtualenv or else upgrading your setuptools to 20.3 or greater?

zsimic · 2018-09-21T07:00:21Z

I package with a recent version of python, in this case 2.7.15 which has setuptools 39.0.1
The issue manifests at runtime: ie, when the pex gets executed with the OSX system python 2.7.10

My cli (https://pypi.org/project/pickley/) is universal, shebang is #!/usr/bin/env python.
At runtime, pickley --version fails, pex unpacks itself etc, then crashes trying to determine version.

To clarify: pex itself isn't crashing, but the package it created doesn't work anymore with the OSX system python, I get a similar error ImportError: cannot import name py31compat (in my case, when click or my own code tries to determine version).

I'm looking into an alternative way of determining version. It's just new version of pex triggered this situation.

DrumMachineLearning · 2018-09-21T13:03:04Z

Setup details. Please let me know if you need anything else

~/sandbox $ virtualenv --version
16.0.0
~/sandbox $ virtualenv venv
New python executable in /Users/Nakis/sandbox/venv/bin/python2.7
Also creating executable in /Users/Nakis/sandbox/venv/bin/python
Installing setuptools, pip, wheel...done.
 ~/sandbox $ . venv/bin/activate
(venv)  ~/sandbox $ pip install responses pex
...snip...
(venv)  ~/sandbox $ pip freeze --all
certifi==2018.8.24
chardet==3.0.4
cookies==2.2.1
funcsigs==1.0.2
idna==2.7
mock==2.0.0
pbr==4.2.0
pex==1.4.6
pip==18.0
requests==2.19.1
responses==0.9.0
setuptools==40.4.1
six==1.11.0
urllib3==1.23
wheel==0.31.1
(venv)  ~/sandbox $ pex responses
Python 2.7.15 (default, May  1 2018, 16:44:08)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import responses
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpun0c31/.deps/responses-0.9.0-py2.py3-none-any.whl/responses.py", line 41, in <module>
    import mock as std_mock
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpun0c31/.deps/mock-2.0.0-py2.py3-none-any.whl/mock/__init__.py", line 2, in <module>
    import mock.mock as _mock
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpun0c31/.deps/mock-2.0.0-py2.py3-none-any.whl/mock/mock.py", line 71, in <module>
    _v = VersionInfo('mock').semantic_version()
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpun0c31/.deps/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 462, in semantic_version
    self._semantic = self._get_version_from_pkg_resources()
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpun0c31/.deps/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 439, in _get_version_from_pkg_resources
    import pkg_resources
  File "/private/var/folders/mk/bz4w0f3505190h3n4x21_jlc0000gn/T/tmpun0c31/.bootstrap/pkg_resources/__init__.py", line 77, in <module>
    from . import py31compat
ImportError: cannot import name py31compat
>>> quit()
(venv)  ~/sandbox $ pip uninstall pex
Uninstalling pex-1.4.6:
  Would remove:
    /Users/Nakis/sandbox/venv/bin/pex
    /Users/Nakis/sandbox/venv/lib/python2.7/site-packages/pex-1.4.6.dist-info/*
    /Users/Nakis/sandbox/venv/lib/python2.7/site-packages/pex/*
Proceed (y/n)? y
  Successfully uninstalled pex-1.4.6
(venv)  ~/sandbox $ pip install pex==1.4.5
...snip...
Installing collected packages: setuptools, pex
  Found existing installation: setuptools 40.4.1
    Uninstalling setuptools-40.4.1:
      Successfully uninstalled setuptools-40.4.1
Successfully installed pex-1.4.5 setuptools-33.1.1
(venv)  ~/sandbox $
(venv)  ~/sandbox $ pex responses
Python 2.7.15 (default, May  1 2018, 16:44:08)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import responses
>>>

zvezdan · 2018-09-24T09:50:31Z

The 1.4.6 release is definitely broken when used with Python 2.7.
This is the commit that causes the issue: #542

I created virtualenv with Python 2.7.13 and used pex-1.4.6.

(py27) $ pip list
Package    Version
---------- -------
pex        1.4.6  
pip        18.0   
setuptools 40.4.1 
wheel      0.31.1 

(py27) $ pex responses
Python 2.7.13 ...
(InteractiveConsole)
>>> import responses
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/private/var/folders/wz/zvcwh0gs3xqdvjp1_p6xs1p8000_lt/T/tmpC7plPj/.deps/responses-0.9.0-py2.py3-none-any.whl/responses.py", line 41, in <module>
    import mock as std_mock
  File "/private/var/folders/wz/zvcwh0gs3xqdvjp1_p6xs1p8000_lt/T/tmpC7plPj/.deps/mock-2.0.0-py2.py3-none-any.whl/mock/__init__.py", line 2, in <module>
    import mock.mock as _mock
  File "/private/var/folders/wz/zvcwh0gs3xqdvjp1_p6xs1p8000_lt/T/tmpC7plPj/.deps/mock-2.0.0-py2.py3-none-any.whl/mock/mock.py", line 71, in <module>
    _v = VersionInfo('mock').semantic_version()
  File "/private/var/folders/wz/zvcwh0gs3xqdvjp1_p6xs1p8000_lt/T/tmpC7plPj/.deps/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 462, in semantic_version
    self._semantic = self._get_version_from_pkg_resources()
  File "/private/var/folders/wz/zvcwh0gs3xqdvjp1_p6xs1p8000_lt/T/tmpC7plPj/.deps/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 439, in _get_version_from_pkg_resources
    import pkg_resources
  File "/private/var/folders/wz/zvcwh0gs3xqdvjp1_p6xs1p8000_lt/T/tmpC7plPj/.bootstrap/pkg_resources/__init__.py", line 77, in <module>
    from . import py31compat
ImportError: cannot import name py31compat
>>> ^D

Then I created a branch on the clone of my pex fork, bumped the version to 1.4.7, and did

$ git revert 8f9c8dd
[test-breakage fb7f49f] Revert "Cleanup bootstrap dependencies before handoff. (#542)"
 4 files changed, 37 insertions(+), 144 deletions(-)

$ python setup.py sdist
running sdist
running egg_info
...
creating dist
Creating tar archive
removing 'pex-1.4.7' (and everything under it)

Then the import works:

(py27) $ pip install -U .../pex/dist/pex-1.4.7.tar.gz 
Processing .../pex/dist/pex-1.4.7.tar.gz
...
Building wheels for collected packages: pex
  Running setup.py bdist_wheel for pex ... done
...
Successfully built pex
Installing collected packages: pex
  Found existing installation: pex 1.4.6
    Uninstalling pex-1.4.6:
      Successfully uninstalled pex-1.4.6
Successfully installed pex-1.4.7

(py27) $ pip list
Package    Version
---------- -------
pex        1.4.7  
pip        18.0   
setuptools 40.4.1 
wheel      0.31.1 

(py27) $ pex responses
Python 2.7.13 ...
(InteractiveConsole)
>>> import responses
>>> ^D

With Python 3.7 everything works with or without revert of #542.

(py37) $ pip list
Package    Version
---------- -------
pex        1.4.6  
pip        18.0   
setuptools 40.4.1 
wheel      0.31.1 

(py37) $ pex responses
Python 3.7.0 ...
(InteractiveConsole)
>>> import responses
>>> ^D
now exiting InteractiveConsole...

(py37) $ pip install -U .../pex/dist/pex-1.4.7.tar.gz 
Processing .../pex/dist/pex-1.4.7.tar.gz
...
Building wheels for collected packages: pex
  Running setup.py bdist_wheel for pex ... done
...
Successfully built pex
Installing collected packages: pex
  Found existing installation: pex 1.4.6
    Uninstalling pex-1.4.6:
      Successfully uninstalled pex-1.4.6
Successfully installed pex-1.4.7

(py37) $ pip list
Package    Version
---------- -------
pex        1.4.7  
pip        18.0   
setuptools 40.4.1 
wheel      0.31.1 

(py37) $ pex responses
Python 3.7.0 ...
(InteractiveConsole)
>>> import responses
>>> ^D
now exiting InteractiveConsole...

Notice that #542 was created in December 2017 and that setuptools had a lot of bug fixes and releases itself. Perhaps those changes were not needed any more? You will know better as maintainers the exact use case where this change might be necessary.

Would you consider reverting this change for users of Python 2.7 or at least looking why/how it causes issues and forward-fixing it so that we can have a new working release of pex that supports modern setuptools for the reasons I described previously in #544?

jsirois · 2018-09-24T15:58:39Z

In particular, the issues appear to be with py27 on osx. I'll take some time today to 1st tweak CI to add OSX shard(s) and get a repro with more details. Then fix forward or revert and finally release asap.
Thanks for your help and patience.

zvezdan · 2018-09-24T20:04:24Z

@jsirois I am able to reproduce this on Linux consistently too.

The only reason I worked on macOS last night is because I had the checkout of pex repo there and could quickly make an sdist of "1.4.7" to confirm that revert fixes the issue. In my initial testing last week I always used Linux.

Here are more details in case they can help with your investigation into the root cause.
The Python interpreter I used was not a system vended Python, but a clean compiled Python with no external packages added to its site-packages, so we're not affected by stale vended versions needed by yum, etc.

I used virtualenv-16.0.0 to build a virtual environment not the venv module from Python 3 standard library. This is, again, an intentional choice because venv still doesn't have all the features of virtualenv.

$ curl -LO https://pypi.org/packages/source/v/virtualenv/virtualenv-16.0.0.tar.gz
$ tar xzvf virtualenv-16.0.0.tar.gz
$ /path/to/bin/python virtualenv-16.0.0/virtualenv.py --no-download ~/.venv/py27
$ . ~/.venv/py27/bin/activate
(py27) $ pip install setuptools==40.4.1
(py27) $ pip install -U pip
(py27) $ pip install pex==1.4.6
(py27) $ pip list
Package    Version
---------- -------
pex        1.4.6  
pip        18.0   
setuptools 40.4.1 
wheel      0.31.1

With this virtualenv ready, the issue is reproducible on Linux too (I left the lines in the output that identify the interpreter on Linux and the /tmp directory is seen in traceback instead of /private/tmp from macOS):

(py27) $ pex responses
Python 2.7.13 (default, Apr 19 2017, 02:44:33) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import responses
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/tmp/tmpm8vjVH/.deps/responses-0.9.0-py2.py3-none-any.whl/responses.py", line 41, in <module>
    import mock as std_mock
  File "/tmp/tmpm8vjVH/.deps/mock-2.0.0-py2.py3-none-any.whl/mock/__init__.py", line 2, in <module>
    import mock.mock as _mock
  File "/tmp/tmpm8vjVH/.deps/mock-2.0.0-py2.py3-none-any.whl/mock/mock.py", line 71, in <module>
    _v = VersionInfo('mock').semantic_version()
  File "/tmp/tmpm8vjVH/.deps/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 462, in semantic_version
    self._semantic = self._get_version_from_pkg_resources()
  File "/tmp/tmpm8vjVH/.deps/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 439, in _get_version_from_pkg_resources
    import pkg_resources
  File "/tmp/tmpm8vjVH/.bootstrap/pkg_resources/__init__.py", line 77, in <module>
    from . import py31compat
ImportError: cannot import name py31compat
>>> Ctrl-D

(py27) $ pip install -U pex-1.4.7.tar.gz   # the same sdist I made after revert
Processing ./pex-1.4.7.tar.gz
...
Building wheels for collected packages: pex
  Running setup.py bdist_wheel for pex ... done
...
Successfully built pex
Installing collected packages: pex
  Found existing installation: pex 1.4.6
    Uninstalling pex-1.4.6:
      Successfully uninstalled pex-1.4.6
Successfully installed pex-1.4.7

(py27) $ pex responses
Python 2.7.13 (default, Apr 19 2017, 02:44:33) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import responses
>>> Ctrl-D

The revert fixes the issue on Linux too.

jsirois · 2018-09-24T20:14:48Z

Hrm. @zvezdan can you, instead of pex responses do one of these and provide full output?:

PEX_VERBOSE=9 pex responses
PEX_VERBOSE=9 pex responses -o responses.pex && PEX_VERBOSE=9 responses.pex

If so, #2 would be most useful, giving clear distinction between build-time info and run-time info.

zvezdan · 2018-09-25T01:47:24Z

@jsirois Here's the output for 1:

(py27) $ PEX_VERBOSE=9 pex responses
pex: Please build pex with the subprocess32 module for more reliable requirement installation and interpreter execution.
pex: Warning, using a UrllibContext which is known to be flaky.
pex: Please build pex with the requests module for more reliable downloads.
pex: Constructed UrllibContext context <pex.http.UrllibContext object at 0x7f83cdcf3290>
Python 2.7.13 (default, Apr 19 2017, 02:44:33)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import responses
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/tmp/tmp3rr5AM/.deps/responses-0.9.0-py2.py3-none-any.whl/responses.py", line 41, in <module>
    import mock as std_mock
  File "/tmp/tmp3rr5AM/.deps/mock-2.0.0-py2.py3-none-any.whl/mock/__init__.py", line 2, in <module>
    import mock.mock as _mock
  File "/tmp/tmp3rr5AM/.deps/mock-2.0.0-py2.py3-none-any.whl/mock/mock.py", line 71, in <module>
    _v = VersionInfo('mock').semantic_version()
  File "/tmp/tmp3rr5AM/.deps/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 462, in semantic_version
    self._semantic = self._get_version_from_pkg_resources()
  File "/tmp/tmp3rr5AM/.deps/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 439, in _get_version_from_pkg_resources
    import pkg_resources
  File "/tmp/tmp3rr5AM/.bootstrap/pkg_resources/__init__.py", line 77, in <module>
    from . import py31compat
ImportError: cannot import name py31compat
>>> Ctrl-D

and the output for 2 with scrubbed paths:

(py27) $ PEX_VERBOSE=9 pex responses -o responses.pex && PEX_VERBOSE=9 ./responses.pex
pex: Please build pex with the subprocess32 module for more reliable requirement installation and interpreter execution.
pex: Warning, using a UrllibContext which is known to be flaky.
pex: Please build pex with the requests module for more reliable downloads.
pex: Constructed UrllibContext context <pex.http.UrllibContext object at 0x7f7ac86ce290>
pex: Please build pex with the subprocess32 module for more reliable requirement installation and interpreter execution.
pex: Selecting runtime interpreter based on pexrc: 0.1ms
pex: Found site-library: /home/.../.venv/py27/lib/python2.7/site-packages
pex: Not a tainted path element: /home/.../src/tmp/pex-test/responses.pex/.bootstrap
pex: Not a tainted path element: /home/.../src/tmp/pex-test/responses.pex
pex: Not a tainted path element: /home/.../.venv/py27/lib/python27.zip
pex: Not a tainted path element: /home/.../.venv/py27/lib/python2.7
pex: Not a tainted path element: /home/.../.venv/py27/lib/python2.7/plat-linux2
pex: Not a tainted path element: /home/.../.venv/py27/lib/python2.7/lib-tk
pex: Not a tainted path element: /home/.../.venv/py27/lib/python2.7/lib-old
pex: Not a tainted path element: /home/.../.venv/py27/lib/python2.7/lib-dynload
pex: Not a tainted path element: /path/to/.../lib/python2.7
pex: Not a tainted path element: /path/to/.../lib/python2.7/plat-linux2
pex: Not a tainted path element: /path/to/.../lib/python2.7/lib-tk
pex: Tainted path element: /home/.../.venv/py27/lib/python2.7/site-packages
pex: Scrubbing from user site: /home/.../.local/lib/python2.7/site-packages
pex: Scrubbing from site-packages: /home/.../.venv/py27/lib/python2.7/site-packages
pex: Scrubbing from path_importer_cache: /home/.../.local/lib/python2.7/site-packages
pex: Scrubbing from path_importer_cache: /home/.../.venv/py27/lib/python2.7/site-packages
pex: E: tags for 'linux-x86_64' x PythonInterpreter('/home/.../.venv/py27/bin/python', PythonIdentity('CPython', 2, 7, 13), {('python', '2.7.13'): '/path/to/....13/lib/python2.7/lib-dynload', ('pex', '1.4.6'): '/home/.../.venv/py27/lib/python2.7/site-packages', ('wheel', '0.31.1'): '/home/.../.venv/py27/lib/python2.7/site-packages', ('pip', '18.0'): '/home/.../.venv/py27/lib/python2.7/site-packages', ('wsgiref', '0.1.2'): '/path/to/....13/lib/python2.7', ('setuptools', '40.4.1'): '/home/.../.venv/py27/lib/python2.7/site-packages'}) -> [('cp27', 'cp27u', 'linux_x86_64'), ('cp27', 'cp27m', 'manylinux1_x86_64'), ('py2', 'none', 'any'), ('cp27', 'cp27mu', 'manylinux1_x86_64'), ('cp27', 'none', 'any'), ('cp27', 'cp27dm', 'manylinux1_x86_64'), ('cp27', 'cp27u', 'manylinux1_x86_64'), ('cp27', 'cp27d', 'manylinux1_x86_64'), ('cp27', 'cp27mu', 'linux_x86_64'), ('cp27', 'none', 'linux_x86_64'), ('py2', 'none', 'linux_x86_64'), ('cp27', 'cp27dm', 'linux_x86_64'), ('cp27', 'cp27d', 'linux_x86_64'), ('cp27', 'cp27m', 'linux_x86_64'), ('py27', 'none', 'any'), ('cp27', 'cp27dmu', 'linux_x86_64'), ('cp27', 'none', 'manylinux1_x86_64'), ('cp2', 'none', 'any'), ('py2', 'none', 'manylinux1_x86_64'), ('cp27', 'cp27dmu', 'manylinux1_x86_64')]
pex: Activating PEX virtual environment from ./responses.pex :: Searching dependency cache: ./responses.pex/.deps :: pex: Activating PEX virtual environment from ./responses.pex :: Searching dependency cache: ./responses.pex/.deps :: pex: Activating PEX virtual environment from ./responses.pex :: Searching dependency cache: ./responses.pex/.deps :: pex: Activating PEX virtual environment from ./responses.pex :: Searching dependency cache: ./responses.pex/.deps :: pex: Activating PEX virtual environment from ./responses.pex :: Searching dependency cache: ./responses.pex/.deps :: pex: Activating PEX virtual environment from ./responses.pex :: Searching dependency cache: ./responses.pex/.deps :: pex: Activating PEX virtual environment from ./responses.pex :: Searching dependency cache: ./responses.pex/.deps :: pex: Activating PEX virtual environment from ./responses.pex :: Searching dependency cache: ./responses.pex/.deps :: pex: Activating PEX virtual environment from ./responses.pex :: Searching dependency cache: ./responses.pex/.deps :: pex: Activating PEX virtual environment from ./responses.pex :: Searching dependency cache: ./responses.pex/.deps :: pex: Activating PEX virtual environment from ./responses.pex :: Searching dependency cache: ./responses.pex/.deps :: pex: Activating PEX virtual environment from ./responses.pex :: Resolving cookies==2.2.1                             pex: Activating PEX virtual environment from ./responses.pex: 93.0ms
pex:   Searching dependency cache: ./responses.pex/.deps: 13.1ms
pex:     Adding pbr 4.2.0: 0.1ms
pex:     Adding idna 2.7: 0.1ms
pex:     Adding cookies 2.2.1: 0.1ms
pex:     Adding six 1.11.0: 0.1ms
pex:     Adding funcsigs 1.0.2: 0.1ms
pex:     Adding certifi 2018.8.24: 0.1ms
pex:     Adding responses 0.9.0: 0.1ms
pex:     Adding chardet 3.0.4: 0.1ms
pex:     Adding urllib3 1.23: 0.1ms
pex:     Adding mock 2.0.0: 0.1ms
pex:     Adding requests 2.19.1: 0.1ms
pex:   Resolving cookies==2.2.1: 0.7ms
pex:   Resolving funcsigs==1.0.2: 8.0ms
pex:   Resolving responses==0.9.0: 57.0ms
pex:   Resolving pbr==4.2.0: 0.3ms
pex:   Resolving six==1.11.0: 0.3ms
pex:   Resolving urllib3==1.23: 0.3ms
pex:   Resolving chardet==3.0.4: 0.3ms
pex:   Resolving requests==2.19.1: 1.0ms
pex:   Resolving certifi==2018.8.24: 0.3ms
pex:   Resolving idna==2.7: 0.3ms
pex:   Resolving mock==2.0.0: 0.7ms
pex:   Activating pbr 4.2.0: 0.4ms
pex:     Adding sitedir: 0.2ms
pex:   Activating urllib3 1.23: 0.4ms
pex:     Adding sitedir: 0.2ms
pex:   Activating mock 2.0.0: 0.4ms
pex:     Adding sitedir: 0.2ms
pex:   Activating six 1.11.0: 0.3ms
pex:     Adding sitedir: 0.2ms
pex:   Activating idna 2.7: 0.4ms
pex:     Adding sitedir: 0.2ms
pex:   Activating cookies 2.2.1: 0.4ms
pex:     Adding sitedir: 0.3ms
pex:   Activating funcsigs 1.0.2: 0.4ms
pex:     Adding sitedir: 0.3ms
pex:   Activating chardet 3.0.4: 0.4ms
pex:     Adding sitedir: 0.3ms
pex:   Activating certifi 2018.8.24: 0.4ms
pex:     Adding sitedir: 0.3ms
pex:   Activating responses 0.9.0: 0.5ms
pex:     Adding sitedir: 0.3ms
pex:   Activating requests 2.19.1: 0.5ms
pex:     Adding sitedir: 0.3ms
pex: No entry point specified, dropping into interpreter
pex: Bootstrap complete, performing final sys.path modifications...
pex: Un-importing third party bootstrap dependency pkg_resources from /home/.../src/tmp/pex-test/responses.pex/.bootstrap
pex: PYTHONPATH contains:
pex:     /home/.../src/tmp/pex-test/responses.pex
pex:   * /home/.../.venv/py27/lib/python27.zip
pex:     /home/.../.venv/py27/lib/python2.7
pex:   * /home/.../.venv/py27/lib/python2.7/plat-linux2
pex:   * /home/.../.venv/py27/lib/python2.7/lib-tk
pex:   * /home/.../.venv/py27/lib/python2.7/lib-old
pex:     /home/.../.venv/py27/lib/python2.7/lib-dynload
pex:     /path/to/.../lib/python2.7
pex:     /path/to/.../lib/python2.7/plat-linux2
pex:     /path/to/.../lib/python2.7/lib-tk
pex:     /home/.../.pex/install/pbr-4.2.0-py2.py3-none-any.whl.6be96516238e4e5b1baee293889642c4e4c3eac2/pbr-4.2.0-py2.py3-none-any.whl
pex:     /home/.../.pex/install/urllib3-1.23-py2.py3-none-any.whl.3630f56fd6bcc8cfd3c568a65834f5b56ebc405e/urllib3-1.23-py2.py3-none-any.whl
pex:     /home/.../.pex/install/mock-2.0.0-py2.py3-none-any.whl.9c729aadd91e3f45efeaa1ada526a02a56176e27/mock-2.0.0-py2.py3-none-any.whl
pex:     /home/.../.pex/install/six-1.11.0-py2.py3-none-any.whl.5fad94645bceb01109c56b5d9bf64de201140300/six-1.11.0-py2.py3-none-any.whl
pex:     /home/.../.pex/install/idna-2.7-py2.py3-none-any.whl.96a4a6d75966bb0aae4ce9bdd177fc4886b37a07/idna-2.7-py2.py3-none-any.whl
pex:     /home/.../.pex/install/cookies-2.2.1-py2.py3-none-any.whl.6b0c79d8a7243c5e05ef8286485f4ec02ce81152/cookies-2.2.1-py2.py3-none-any.whl
pex:     /home/.../.pex/install/funcsigs-1.0.2-py2.py3-none-any.whl.1abdb57bb4c560603d658d202032bc95bdfa8f8a/funcsigs-1.0.2-py2.py3-none-any.whl
pex:     /home/.../.pex/install/chardet-3.0.4-py2.py3-none-any.whl.c4332e34e38b781695ce775973fe40663558a897/chardet-3.0.4-py2.py3-none-any.whl
pex:     /home/.../.pex/install/certifi-2018.8.24-py2.py3-none-any.whl.d24ebf6f0b213fc082815d0f3a816ab6c64b2d1f/certifi-2018.8.24-py2.py3-none-any.whl
pex:     /home/.../.pex/install/responses-0.9.0-py2.py3-none-any.whl.a5224fde5182386549956aca3dd40fed5c9a198b/responses-0.9.0-py2.py3-none-any.whl
pex:     /home/.../.pex/install/requests-2.19.1-py2.py3-none-any.whl.9c19d1f49965483c8e82326b964ec1050dff6851/requests-2.19.1-py2.py3-none-any.whl
pex:   * /home/.../src/tmp/pex-test/responses.pex/.bootstrap
pex:   * - paths that do not exist or will be imported via zipimport
Python 2.7.13 (default, Apr 19 2017, 02:44:33)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import responses
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/.../.pex/install/responses-0.9.0-py2.py3-none-any.whl.a5224fde5182386549956aca3dd40fed5c9a198b/responses-0.9.0-py2.py3-none-any.whl/responses.py", line 41, in <module>
    import mock as std_mock
  File "/home/.../.pex/install/mock-2.0.0-py2.py3-none-any.whl.9c729aadd91e3f45efeaa1ada526a02a56176e27/mock-2.0.0-py2.py3-none-any.whl/mock/__init__.py", line 2, in <module>
    import mock.mock as _mock
  File "/home/.../.pex/install/mock-2.0.0-py2.py3-none-any.whl.9c729aadd91e3f45efeaa1ada526a02a56176e27/mock-2.0.0-py2.py3-none-any.whl/mock/mock.py", line 71, in <module>
    _v = VersionInfo('mock').semantic_version()
  File "/home/.../.pex/install/pbr-4.2.0-py2.py3-none-any.whl.6be96516238e4e5b1baee293889642c4e4c3eac2/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 462, in semantic_version
    self._semantic = self._get_version_from_pkg_resources()
  File "/home/.../.pex/install/pbr-4.2.0-py2.py3-none-any.whl.6be96516238e4e5b1baee293889642c4e4c3eac2/pbr-4.2.0-py2.py3-none-any.whl/pbr/version.py", line 439, in _get_version_from_pkg_resources
    import pkg_resources
  File ".bootstrap/pkg_resources/__init__.py", line 77, in <module>
ImportError: cannot import name py31compat
>>> Ctrl-D

jsirois · 2018-09-25T03:34:02Z

Alright - thanks for the help. I have things figured out, but with a little smash all things, here: #553

I'll pick this back up in the morning to whittle down the minimal crisp changes needed and provide a full, satisfying explanation.

Previously only root 3rdparty packages copied into the bootstrap were unimported leaving subpackages in-place. This could lead to a mismatch in 3rdparty package API expectations when a re-imported 3rdparty root package from the user PEX sys.path was from a different version of the 3rdparty package. Fixes pex-tool#550

Previously only root 3rdparty packages copied into the bootstrap were unimported leaving subpackages in-place. This could lead to a mismatch in 3rdparty package API expectations when a re-imported 3rdparty root package from the user PEX sys.path was from a different version of the 3rdparty package. Fixes #550

lorencarvalho mentioned this issue Sep 21, 2018

Upgrade to pex-1.4.6. linkedin/pygradle#247

Merged

jsirois self-assigned this Sep 24, 2018

jsirois added the bug label Sep 24, 2018

jsirois mentioned this issue Sep 24, 2018

add an OSX shard #479

Closed

jsirois added the in progress label Sep 24, 2018

This was referenced Sep 25, 2018

Fixup PEX.demote_bootstrap: fully unimport. #554

Merged

Release 1.4.7 #555

Closed

jsirois closed this as completed in #554 Sep 25, 2018

jsirois removed the in progress label Sep 26, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pex 1.4.6 builds a failing pex #550

Pex 1.4.6 builds a failing pex #550

DrumMachineLearning commented Sep 20, 2018 •

edited

Loading

jsirois commented Sep 21, 2018 •

edited

Loading

zsimic commented Sep 21, 2018

jsirois commented Sep 21, 2018

zsimic commented Sep 21, 2018 •

edited

Loading

DrumMachineLearning commented Sep 21, 2018 •

edited

Loading

zvezdan commented Sep 24, 2018

jsirois commented Sep 24, 2018

zvezdan commented Sep 24, 2018

jsirois commented Sep 24, 2018

zvezdan commented Sep 25, 2018

jsirois commented Sep 25, 2018

Pex 1.4.6 builds a failing pex #550

Pex 1.4.6 builds a failing pex #550

Comments

DrumMachineLearning commented Sep 20, 2018 • edited Loading

jsirois commented Sep 21, 2018 • edited Loading

zsimic commented Sep 21, 2018

jsirois commented Sep 21, 2018

zsimic commented Sep 21, 2018 • edited Loading

DrumMachineLearning commented Sep 21, 2018 • edited Loading

zvezdan commented Sep 24, 2018

jsirois commented Sep 24, 2018

zvezdan commented Sep 24, 2018

jsirois commented Sep 24, 2018

zvezdan commented Sep 25, 2018

jsirois commented Sep 25, 2018

DrumMachineLearning commented Sep 20, 2018 •

edited

Loading

jsirois commented Sep 21, 2018 •

edited

Loading

zsimic commented Sep 21, 2018 •

edited

Loading

DrumMachineLearning commented Sep 21, 2018 •

edited

Loading