0.902 crashes on tuple assignment

[xqzhou]

Crash Report

Mypy crashed after upgraded from 0.812 to 0.902

Traceback

/Users/admin/Library/Caches/pypoetry/virtualenvs/test/lib/python3.8/site-packages/mode/services.py:684: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.902
Traceback (most recent call last):
  File "mypy/checker.py", line 401, in accept
  File "mypy/nodes.py", line 1073, in accept
  File "mypy/checker.py", line 2013, in visit_assignment_stmt
  File "mypy/checker.py", line 2051, in check_assignment
  File "mypy/checker.py", line 2551, in check_assignment_to_multiple_lvalues
  File "mypy/checker.py", line 2592, in check_multi_assignment
  File "mypy/checker.py", line 2701, in check_multi_assignment_from_tuple
AssertionError:
/Users/admin/Library/Caches/pypoetry/virtualenvs/test/lib/python3.8/site-packages/mode/services.py:684: : note: use --pdb to drop into pdb

To Reproduce

Your Environment

• Mypy version used: 0.902
• Mypy command-line flags: --html-report .mypy_report --txt-report .mypy_report src/
• Mypy configuration options from mypy.ini (and other config files):
• Python version used: 3.8.8
• Operating system and version: macOS Big Sur 11.4

[JelleZijlstra]

Could you share the crashing code?

[hellocoldworld]

Same issue here.

/home/nico/.local/share/virtualenvs/blacklists_service--dBzAapY/lib/python3.9/site-packages/mode/services.py:684: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.902
Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "mypy/checker.py", line 401, in accept
  File "mypy/nodes.py", line 1073, in accept
  File "mypy/checker.py", line 2013, in visit_assignment_stmt
  File "mypy/checker.py", line 2051, in check_assignment
  File "mypy/checker.py", line 2551, in check_assignment_to_multiple_lvalues
  File "mypy/checker.py", line 2592, in check_multi_assignment
  File "mypy/checker.py", line 2701, in check_multi_assignment_from_tuple
AssertionError: 
/home/nico/.local/share/virtualenvs/blacklists_service--dBzAapY/lib/python3.9/site-packages/mode/services.py:684: : note: use --pdb to drop into pdb

In this case as you can see, the error is when type-checking the package mode.
In particular it fails at the line:

            done, pending = await asyncio.wait(
                futures.values(),
                return_when=asyncio.FIRST_COMPLETED,
                timeout=timeout,
                loop=self.loop,
            )

[TH3CHARLie]

73f3d73 is the first bad commit
commit 73f3d73
Author: Shantanu 12621235+hauntsaninja@users.noreply.github.com
Date: Tue Apr 20 23:06:35 2021 -0700

Sync typeshed (#10346)

This commit updates the typeshed for asyncio which triggers the internal crash

[JelleZijlstra]

Should be python/typeshed#5193 then.

[JukkaL]

@TH3CHARLie Have you made any further progress with this crash? I can help if you can share a repro. I couldn't reproduce this (but I only spent a little time on this).

[TH3CHARLie]

I don't have a minimal repro right now, I can reproduce this on mode while this makes debugging complicated.

[JelleZijlstra]

Here is a minimal(ish) reporo whittled down from the code in mode:

import asyncio

from typing import Awaitable, Set


async def wait_first(*coros: Awaitable) -> None:
    futures = {coro: asyncio.ensure_future(coro) for coro in coros}
    done: Set[asyncio.Future]
    pending: Set[asyncio.Future]
    done, pending = await asyncio.wait(futures.values())

stack trace

% mypy --show-traceback services.py
services.py:10: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.910+dev.2ebdbca3b5afbfa1113f01b583522f4afcc4b3e3
Traceback (most recent call last):
  File "/Users/jelle/py/venvs/mypy/bin/mypy", line 33, in <module>
    sys.exit(load_entry_point('mypy', 'console_scripts', 'mypy')())
  File "/Users/jelle/py/mypy/mypy/__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/Users/jelle/py/mypy/mypy/main.py", line 87, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/Users/jelle/py/mypy/mypy/main.py", line 165, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/jelle/py/mypy/mypy/build.py", line 179, in build
    result = _build(
  File "/Users/jelle/py/mypy/mypy/build.py", line 254, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/jelle/py/mypy/mypy/build.py", line 2696, in dispatch
    process_graph(graph, manager)
  File "/Users/jelle/py/mypy/mypy/build.py", line 3020, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/jelle/py/mypy/mypy/build.py", line 3118, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/Users/jelle/py/mypy/mypy/build.py", line 2165, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/Users/jelle/py/mypy/mypy/checker.py", line 294, in check_first_pass
    self.accept(d)
  File "/Users/jelle/py/mypy/mypy/checker.py", line 401, in accept
    stmt.accept(self)
  File "/Users/jelle/py/mypy/mypy/nodes.py", line 689, in accept
    return visitor.visit_func_def(self)
  File "/Users/jelle/py/mypy/mypy/checker.py", line 726, in visit_func_def
    self._visit_func_def(defn)
  File "/Users/jelle/py/mypy/mypy/checker.py", line 730, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/Users/jelle/py/mypy/mypy/checker.py", line 792, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/Users/jelle/py/mypy/mypy/checker.py", line 975, in check_func_def
    self.accept(item.body)
  File "/Users/jelle/py/mypy/mypy/checker.py", line 401, in accept
    stmt.accept(self)
  File "/Users/jelle/py/mypy/mypy/nodes.py", line 1017, in accept
    return visitor.visit_block(self)
  File "/Users/jelle/py/mypy/mypy/checker.py", line 1984, in visit_block
    self.accept(s)
  File "/Users/jelle/py/mypy/mypy/checker.py", line 401, in accept
    stmt.accept(self)
  File "/Users/jelle/py/mypy/mypy/nodes.py", line 1075, in accept
    return visitor.visit_assignment_stmt(self)
  File "/Users/jelle/py/mypy/mypy/checker.py", line 2022, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/Users/jelle/py/mypy/mypy/checker.py", line 2060, in check_assignment
    self.check_assignment_to_multiple_lvalues(lvalue.items, rvalue, rvalue,
  File "/Users/jelle/py/mypy/mypy/checker.py", line 2563, in check_assignment_to_multiple_lvalues
    self.check_multi_assignment(lvalues, rvalue, context, infer_lvalue_type)
  File "/Users/jelle/py/mypy/mypy/checker.py", line 2604, in check_multi_assignment
    self.check_multi_assignment_from_tuple(lvalues, rvalue, rvalue_type,
  File "/Users/jelle/py/mypy/mypy/checker.py", line 2713, in check_multi_assignment_from_tuple
    assert isinstance(reinferred_rvalue_type, TupleType)
AssertionError: 
services.py:10: : note: use --pdb to drop into pdb

[TH3CHARLie]

i remember last time i debug mode , checkexpr infer the rhs type as Any

[JelleZijlstra]

And here's a version with wait inlined from typeshed. Interestingly, it stops reproing when I try to inline ensure_future or Task.

import asyncio
from asyncio.futures import Future
from asyncio.tasks import Task
from typing import Any, Awaitable, Iterable, overload, Tuple, Set, TypeVar

_T = TypeVar("_T")
_FT = TypeVar("_FT", bound=Awaitable[Any])

@overload
def wait(fs: Iterable[_FT]) -> Future[Tuple[Set[_FT], Set[_FT]]]: ...
@overload
def wait(fs: Iterable[Awaitable[_T]]) -> Future[Tuple[Set[Task[_T]], Set[Task[_T]]]]: ...
def wait(fs: Any) -> Any:
    raise Exception

async def wait_first(*coros: Awaitable) -> None:
    futures = {coro: asyncio.ensure_future(coro) for coro in coros}
    done: Set[asyncio.Future]
    pending: Set[asyncio.Future]
    done, pending = await wait(futures.values())

[JukkaL]

Here's a version with still fewer dependencies that doesn't depend on asyncio:

from typing import Any, Awaitable, Iterable, overload, Tuple, List, TypeVar, Generic

T = TypeVar("T")
FT = TypeVar("FT", bound=Awaitable[Any])

class Future(Awaitable[T], Iterable[T]):
    pass

class Task(Future[T]):
    pass

@overload
def wait(fs: Iterable[FT]) -> Future[Tuple[List[FT], List[FT]]]: ...
@overload
def wait(fs: Iterable[Awaitable[T]]) -> Future[Tuple[List[Task[T]], List[Task[T]]]]: ...
def wait(fs: Any) -> Any:
    pass

async def f() -> None:
    done: Any
    pending: Any
    futures: Iterable[Task[Any]]
    done, pending = await wait(futures)

The crash is clearly a mypy bug, but the signature of wait also looks suspect, since the overload variants are overlapping.

[hellocoldworld]

Great work! Thanks!