NameError on pkg_resources

[jolaf]

Things to check first

• I have searched the existing issues and didn't find my bug already reported there

• I have checked that my bug is still present in the latest release

Typeguard version

4.1.4

Python version

3.10.12

What happened?

Traceback (most recent call last):
  File "test.py", line 3, in <module>
    import pkg_resources
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/home/devel/.local/lib/python3.10/site-packages/typeguard/_importhook.py", line 98, in exec_module
    super().exec_module(module)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 77, in <module>
    __import__('pkg_resources.extern.packaging.specifiers')
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/home/devel/.local/lib/python3.10/site-packages/typeguard/_importhook.py", line 98, in exec_module
    super().exec_module(module)
  File "/usr/lib/python3/dist-packages/pkg_resources/_vendor/packaging/specifiers.py", line 317, in <module>
    class Specifier(_IndividualSpecifier):
  File "/usr/lib/python3/dist-packages/pkg_resources/_vendor/packaging/specifiers.py", line 426, in Specifier
    def _compare_compatible(self, prospective: ParsedVersion, spec: str) -> bool:
  File "/usr/lib/python3/dist-packages/pkg_resources/_vendor/packaging/specifiers.py", line 306, in _require_version_compare
    fn: Callable[["Specifier", ParsedVersion, str], bool]
NameError: name 'Specifier' is not defined. Did you mean: 'BaseSpecifier'?

How can we reproduce the bug?

test.py:

from typeguard import install_import_hook
with install_import_hook(('pkg_resources',)):
    import pkg_resources

$ python3 test.py
<crash>

[jolaf]

See also #90.

[agronholm]

And what version of setuptools is this? It's frustrating trying to get the same results when I have no clue which version you're running against. I don't get this result with the latest setuptools.

[jolaf]

I've got setuptools 59.6.0.

Are you sure it's setuptools? I thought pkg_resources is a built-in package, it's located at /usr/lib/python3/dist-packages.

[agronholm]

pkg_resources is a well-known part of setuptools. It's deprecated though.

[agronholm]

Now that I installed setuptools 59.6.0, I can reproduce this error.

[jolaf]

On latest setuptools, 68.2.0, I get another crash:

Traceback (most recent call last):
  File "/media/sf_d/WWPass/git/test.py", line 3, in <module>
    import pkg_resources
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/home/jolaf/.local/lib/python3.10/site-packages/typeguard/_importhook.py", line 98, in exec_module
    super().exec_module(module)
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3328, in <module>
    def _initialize_master_working_set():
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3302, in _call_aside
    f(*args, **kwargs)
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3340, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 622, in _build_master
    ws = cls()
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 615, in __init__
    self.add_entry(entry)
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 671, in add_entry
    for dist in find_distributions(entry, True):
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2134, in find_on_path
    for dist in factory(fullpath):
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2199, in distributions_from_metadata
    yield Distribution.from_location(
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2665, in from_location
    return cls(
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2646, in __init__
    self._version = safe_version(version)
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1398, in safe_version
    return str(packaging.version.Version(version))
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/version.py", line 204, in __init__
    pre=_parse_letter_version(match.group("pre_l"), match.group("pre_n")),
  File "/home/jolaf/.local/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/version.py", line 453, in _parse_letter_version
    def _parse_letter_version(
  File "/home/jolaf/.local/lib/python3.10/site-packages/typeguard/_functions.py", line 138, in check_argument_types
    check_type_internal(value, annotation, memo)
  File "/home/jolaf/.local/lib/python3.10/site-packages/typeguard/_checkers.py", line 766, in check_type_internal
    raise TypeCheckError(f"is not an instance of {qualified_name(origin_type)}")
typeguard.TypeCheckError: argument "letter" (None) is not an instance of str

But this looks like error on their side.

[agronholm]

That's not a crash though, it's a legitimate error reported by Typeguard.

[agronholm]

It seems clear why this happens: The _require_version_compare() function is used as a decorator within the Specifier class. As the class hasn't been defined before the decorator is used, NameError is raised.

[jolaf]

For the legitimate error in latest setuptools, FYI: pypa/setuptools#4044

[agronholm]

I think it may be best to propose that patch against packaging instead.

[jolaf]

Ok, pypa/packaging#723

[jolaf]

It seems clear why this happens: The _require_version_compare() function is used as a decorator within the Specifier class. As the class hasn't been defined before the decorator is used, NameError is raised.

Maybe NameErrors occuring in typeguard-generated code should be caught right there and replaced with warnings?
That would notify user of the problem but allow the app to keep running.

[jolaf]

Maybe even ALL exceptions, except TypeCheckError, occurring in typeguard-generated code should be caught and replaced with warnings?

[jolaf]

I'm also thinking that it would be perfect to have an option to turn all TypeCheckErrors into warnings without stopping the app.

Compare it to, say, C or Java compiler. When there are errors in you code, it doesn't show you the first error it encounters and quit; instead, it checks all your code and shows you all the errors it finds. That makes fixing those errors simpler and faster. It would be nice if typeguard could behave this way too.

[agronholm]

I'm also thinking that it would be perfect to have an option to turn all TypeCheckErrors into warnings without stopping the app.

You mean something like typeguard.config.typecheck_fail_callback = typeguard.warn_on_error?

[jolaf]

You mean something like typeguard.config.typecheck_fail_callback = typeguard.warn_on_error?

Yep, for example.
I would prefer a keyword argument to install_import_hook().
But anything that can be set from Python will do.
Environment variable will do also.

[agronholm]

Yep, for example.

I mean, this already exist in Typeguard (link), so why not use it?

[jolaf]

WOW! Thanks!

[jolaf]

Still, I think that catching exceptions (except TypeCheckError) in instrumentation and turning them into warnings is still a good idea anyway.

[agronholm]

That warn_on_error was added to fulfill your own request here: #85

As for catching all exceptions, I'm not entirely comfortable with that as it might mask some actual bugs in Typeguard itself.

[jolaf]

Great, thank you very much! I somehow missed it.

Well, the reality now is there are bugs in typeguard itself. And some of them are still uncaught. For now, I run my Big App under typeguard, it crashes with a problem like this issue, I report it. Then I wait for you to fix it, and then I run my Big App again and catch the next bug, and the cycle repeats. This is extremely slow process. If typeguard bugs were warnings, not crashes, I would be able to catch and report all of them at once.

[jolaf]

Moreover, it seems that ALL exceptions in typeguard-generated code ARE indeed bugs in typeguard.
Because in normal operation generated code either returns silently or throws TypeCheckError.

[agronholm]

My fear is that other people may not report these warnings at all.

[jolaf]

Yeah, I understand it.

Maybe it could be another option?

[jolaf]

Something like this could be appropriate:
typeguard.config.internal_error_callback = typeguard.warn_on_error

Or something like typeguard.InternalErrorPolicy.

[jolaf]

Maybe an undocumented option would be safe enough.

[jolaf]

Thinking more about it, maybe NameErrors during type checking deserve special treatment. NameError means some typing annotation can't be properly processed, but that doesn't mean that annotation as incorrect. Probably those annotations should be treated like Any?