ignore-missing-import option does not work on false positive "Library stubs not installed for X"

[kbrose]

Bug Report

Sometimes, multiple packages on pypi will share a namespace once installed. One example pair is google-cloud-storage and protobuf. If you pip install both of these packages, you can see that they both get installed under a top level google/ folder:

>>> from google.cloud import storage
>>> from google.protobuf import message
>>> storage.__file__
'.../python3.9/site-packages/google/cloud/storage/__init__.py'
>>> message.__file__
'.../python3.9/site-packages/google/protobuf/message.py'

This is a problem when one of the packages has a hit in typeshed, but the other does not. In this case, there is a types-protobuf package, but not a types-google-cloud-storage package. Running mypy will always error, even when --ignore-missing-imports is set. I believe this happens because it thinks that a known library stub is missing (probably because of the shared google namespace), but this is not the case.

Further, the error message Hints at an incorrect solution:

note: Hint: "python3 -m pip install types-protobuf"

This message shows up even when types-protobuf is already installed.

To Reproduce

Create two files, requirements.txt and repro.py:

# requirements.txt
protobuf
google-cloud-storage
types-protobuf
mypy

# repro.py
from google.cloud import storage
from google.protobuf import message

Install the packages: pip install -r requirements.txt

Run mypy:

$ mypy .
repro.py:1: error: Library stubs not installed for "google.cloud" (or incompatible with Python 3.9)
repro.py:1: note: Hint: "python3 -m pip install types-protobuf"
repro.py:1: note: (or run "mypy --install-types" to install all missing stub packages)
repro.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

Unfortunately, --ignore-missing-imports does not get rid of the error.

$ mypy . --ignore-missing-imports
repro.py:1: error: Library stubs not installed for "google.cloud" (or incompatible with Python 3.9)
repro.py:1: note: Hint: "python3 -m pip install types-protobuf"
repro.py:1: note: (or run "mypy --install-types" to install all missing stub packages)
repro.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

I may be missing something, but as far as I can tell there is no way to force this error to go away. Since it always errors out, this makes it very hard to typecheck my code, especially in an automated CICD pipeline.

Also note that the hint is not helpful, as it suggests installing a package that is already installed!

$ python3 -m pip install types-protobuf
Requirement already satisfied: types-protobuf in .../python3.9/site-packages (0.1.10)
Requirement already satisfied: types-futures in .../python3.9/site-packages (from types-protobuf) (0.1.3)

Expected Behavior

• I would expect --ignore-missing-imports to get rid of this error.
• The suggested hint is wrong, and I would expect it to not be shown.

Actual Behavior

• No way to get rid of this "error".
• Incorrect Hint is shown.

Your Environment

• Mypy version used: 0.900.
• Mypy command-line flags: see above
• Mypy configuration options from mypy.ini (and other config files): n/a
• Python version used: 3.9.1
• Operating system and version: macos

[kbrose]

It seems related to FindModuleCache.get_toplevel_possibilities(), but there's a comment there which seems to say the observed behavior shouldn't be happening.

            # False positives are fine for correctness here, since we will check
            # precisely later, so we only look at the root of every filename without
            # any concern for the exact details.

[JukkaL]

I'm working on a fix. I'm hoping to release mypy 0.902 with the fix this week.

[JukkaL]

Fix is in #10609 (waiting for review). This seems specific to google.* packages. If you've encountered this issues with other packages, please leave a comment.

[kbrose]

Thanks for the prompt fix @JukkaL, I appreciate it!

[JukkaL]

he fix to this issue was released in mypy 0.902.

[JukkaL]

@alexanderdamiani Your issue seems different. This issue about multiple libraries under the same top-level package, and I'm not aware other libraries that live under yaml. Please file a separate issue with more context.

[abhinavcoder]

@JukkaL
I am facing a similar issue when using the following code:

from google.cloud import storage

Error:

error: Module "google.cloud" has no attribute "storage"

• Settings:
  • mypy: 0.942
  • OS: macOS
  • python: 3.7
    I am running the mypy using the following command:

pipenv run mypy --install-types --non-interactive .

The above install the package types-protobuf which might be causing the issue.