aspy.refactor_imports.classify - _get_module_info fails for zipped third party libraries

[shea-parkes]

There is a special case logic gate in classify.py:

# special case pypy3 bug(?)
elif not os.path.exists(spec.origin):
    return True, '(builtin)', True

If you are using a zipped third-party package, then this will falsely identify it as part of the standard library.

Here's a code snippet showing the value of spec.origin in such a case:

Python 3.7.5 (default, Oct 31 2019, 15:18:51) [MSC v.1916 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.9.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import importlib.util

In [2]: spec = importlib.util.find_spec('py4j')

In [3]: spec
Out[3]: ModuleSpec(name='py4j', loader=<zipimporter object "C:\DevApps\spark\spark-2.4.4-bin\python\lib\py4j-0.10.7-src.zip">, origin='C:\\DevApps\\spark\\spark-2.4.4-bin\\python\\lib\\py4j-0.10.7-src.zip\\py4j\\__init__.py', submodule_search_locations=['C:\\DevApps\\spark\\spark-2.4.4-bin\\python\\lib\\py4j-0.10.7-src.zip\\py4j'])

In [4]: spec.origin
Out[4]: 'C:\\DevApps\\spark\\spark-2.4.4-bin\\python\\lib\\py4j-0.10.7-src.zip\\py4j\\__init__.py'

That spec.origin doesn't exist on the filesystem, so os.path.exists(spec.origin) is False.

I realize we're using %PythonPath% which isn't really supported, but I do believe this would also be an issue in other scenarios. (Sourcing py4j from a zip included with a Spark distribution is actually a fairly common example I believe.)

Would it be reasonable to get another logic gate in there to identify the zipped library scenario? I'd be happy to try and make the addition if you like.

Thanks for the great tool. We're fine with mixing our third-party and first-party imports due to using %PythonPath%. We really appreciate the black compatibility (and the focus on clean diffs).

[asottile]

PYTHONPATH and eggs are intentionally not supported but if you can submit a small tested patch I'll consider it