Contents of native modules not found when running in a virtualenv

[mthuurne]

Steps to reproduce

Lint this code:

from struct import Struct

Current behavior

E0611: No name 'Struct' in module 'struct' (no-name-in-module)

Expected behavior

No error is reported, since the class struct.Struct exists. This class is implemented in native code, which is probably why PyLint cannot discover it. However, since it is part of the standard library, I think it would be good to have dedicated support for it.

pylint --version output

pylint 2.4.4
astroid 2.3.3
Python 3.8.2 (default, Mar 05 2020, 18:58:42) [GCC]

[mthuurne]

A similar situation occurs for the zlib.adler32 function. Should I open a separate issue for that or amend this one?

[PCManticore]

Hmm, this is odd. For modules from standard library that are in native code, we should be able to properly analyze them already, since they are automatically imported and we build an AST from the live representation.

I'm not able to reproduce the issue with struct for instance with:

pylint 2.4.4
astroid 2.3.3
Python 3.8.1 (default, Feb 13 2020, 10:17:07)
[Clang 8.1.0 (clang-802.0.42)]

$ pylint a.py
a.py:1:0: C0114: Missing module docstring (missing-module-docstring)
a.py:1:0: W0611: Unused Struct imported from struct (unused-import)
a.py:2:0: W0611: Unused adler32 imported from zlib (unused-import)

[mthuurne]

Weird...

To make sure it is not something in my project, I created a fresh virtualenv and installed pylint into that using pip, then ran it with no pylintrc. The problem persists:

$ pylint testcase.py 
************* Module testcase
testcase.py:1:0: C0114: Missing module docstring (missing-module-docstring)
testcase.py:1:0: E0611: No name 'Struct' in module 'struct' (no-name-in-module)
testcase.py:2:0: E0611: No name 'adler32' in module 'zlib' (no-name-in-module)
testcase.py:1:0: W0611: Unused Struct imported from struct (unused-import)
testcase.py:2:0: W0611: Unused adler32 imported from zlib (unused-import)

----------------------------------------------------------------------
Your code has been rated at -55.00/10 (previous run: -55.00/10, +0.00)

When I run it under strace, I see the native library (_struct.cpython-38-x86_64-linux-gnu.so) being loaded without errors, so it is indeed attempting to load the native code.

This is on x86-64 Linux, openSUSE Tumbleweed.

There is no error if I try to run the source file containing the two imports.

Any hints on what I should investigate next?

[mthuurne]

I think I found something:

$ pylint --extension-pkg-whitelist=_struct,zlib testcase.py 
************* Module testcase
testcase.py:1:0: C0114: Missing module docstring (missing-module-docstring)
testcase.py:1:0: W0611: Unused Struct imported from struct (unused-import)
testcase.py:2:0: W0611: Unused adler32 imported from zlib (unused-import)

So it seems the _struct module (native part of the struct module) and the zlib module (fully native) are not whitelisted by default. Is it possible that using a virtualenv breaks the detection of what is in the standard library and what is not?

[mthuurne]

When I run Astroid's unit tests from within a virtualenv, four of them fail, including this one:

self = <tests.unittest_modutils.StandardLibModuleTest testMethod=test_4>

    def test_4(self):
>       self.assertTrue(modutils.is_standard_module("hashlib"))
E       AssertionError: False is not true

unittest_modutils.py:263: AssertionError

However, if I run the tests outside of a virtualenv, all of them pass.