ImportError: bad magic number in ... does not indicate where is that file located

[hroncok]

BPO	46835
Nosy	@brettcannon, @encukou, @hroncok
PRs	gh-90991: Include path to pyc file in pyc-related ImportErrors #31534

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-02-23.12:43:36.296>
labels = ['type-feature', 'library', '3.11']
title = 'ImportError: bad magic number in ... does not indicate where is that file located'
updated_at = <Date 2022-02-24.20:09:00.809>
user = 'https://github.com/hroncok'

bugs.python.org fields:

activity = <Date 2022-02-24.20:09:00.809>
actor = 'brett.cannon'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2022-02-23.12:43:36.296>
creator = 'hroncok'
dependencies = []
files = []
hgrepos = []
issue_num = 46835
keywords = ['patch']
message_count = 3.0
messages = ['413788', '413789', '413815']
nosy_count = 3.0
nosy_names = ['brett.cannon', 'petr.viktorin', 'hroncok']
pr_nums = ['31534']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue46835'
versions = ['Python 3.11']

[hroncok]

Recently I've been debugging a very nasty bug report that looked like this:

    Traceback (most recent call last):
      File "/usr/bin/jupyter-notebook", line 5, in <module>
        from notebook.notebookapp import main
      File "/usr/lib/python3.10/site-packages/notebook/notebookapp.py", line 78, in <module>
        from .services.kernels.kernelmanager import MappingKernelManager, AsyncMappingKernelManager
      File "/usr/lib/python3.10/site-packages/notebook/services/kernels/kernelmanager.py", line 18, in <module>
        from jupyter_client.session import Session
      File "/usr/lib/python3.10/site-packages/jupyter_client/session.py", line 41, in <module>
        from jupyter_client.jsonutil import extract_dates, squash_dates, date_default
      File "/usr/lib/python3.10/site-packages/jupyter_client/jsonutil.py", line 10, in <module>
        from dateutil.parser import parse as _dateutil_parse
      File "/usr/lib/python3.10/site-packages/dateutil/parser/__init__.py", line 2, in <module>
        from ._parser import parse, parser, parserinfo, ParserError
      File "/usr/lib/python3.10/site-packages/dateutil/parser/_parser.py", line 42, in <module>
        import six
    ImportError: bad magic number in 'six': b'\x03\xf3\r\n'

For details, see https://bugzilla.redhat.com/2057340 and benjaminp/six#359

What would really make things much easier to understand would be if the exception mentioned what is the path of 'six'.

Consider this example:

A rogue .py file in /usr/bin:

    $ sudo touch /usr/bin/copy.py

Programs fail with:

    Traceback (most recent call last):
      File "/usr/bin/...", line ..., in <module>
        ...
    ImportError: cannot import name 'deepcopy' from 'copy' (/usr/bin/copy.py)

Immediately I can see there is /usr/bin/copy.py which is probably not supposed to be there.

However, when it is a pyc instead:

    $ sudo touch /usr/bin/copy.pyc

Programs fail with:

    Traceback (most recent call last):
      File "/usr/bin/...", line ..., in <module>
        ...
    ImportError: bad magic number in 'copy': b''

Now I have no idea where "copy" is.

The is a request for that exception to give that infomartion.

[hroncok]

Apparently, the exception already contains a path attribute with exactly the kind of information I'd like to see.

What if the message was:

ImportError: bad magic number in '/usr/bin/six.pyc': b'\x03\xf3\r\n'

Would that be accepted as a PR? Should I discuss this on the mailing list?

[encukou]

I assume a PR review should be enough.