Bug(?): GzipFile typing incompatible with io.TextIOWrapper

[jamesbraza]

Please see the below code with Python 3.12:

import gzip
import io


def gzip_to_list(file: gzip.GzipFile) -> list[str]:
    with io.TextIOWrapper(buffer=file) as f:
        return list(f)

Running mypy==1.8.0, I get this error message:

a.py:6:34: error: Argument "buffer" to "TextIOWrapper" has incompatible type "GzipFile"; expected "IO[bytes]"  [arg-type]

However, running this code, it actually works fine.

I am thinking this is a typeshed bug, but I am not 100% positive. #5491 and #6061 are possibly related

[srittau]

I/O classes are a bit of a mess, since they predate protocols. TextIOWrapper should take a protocol as buffer argument. Someone needs to investigate the implementation of TextIOWrapper (found here: https://github.com/python/cpython/blob/main/Modules/_io/textio.c) to determine, which methods and attributes of buffer are actually accessed and then we could use such a protocol.

In fact, TextIOWrapper should probably be generic over the buffer type, but we can't do that before PEP 696 lands.

[srittau]

Here is what I found:

• seekable() -> bool
• readable() -> bool
• writable() -> bool
• tell() -> int
• write(b: bytes, /) -> object
• flush() -> object
• read(size: int = ...) -> bytes
• seek(offset: Literal[0], whence: Literal[2]) -> int
• truncate(size: int, /) -> int
• fileno() -> int
• isatty() -> int
• close() -> object
• name: str
• closed: bool
• Optional: read1(size: int, /) -> bytes (strictly speaking Buffer would do as return type, but bytes is safer)

Some of these are only forwarded, so strictly speaking they are not necessary if the method is not called on the wrapper, but not requiring them would be unsafe.

It also checks for buffer.raw to exist and be a FileIO type under certain conditions, but ignores it if it isn't, so this is not part of the protocol.

[AlexWaygood]

Here is what I found

I found the same.

Links to the source code

• A readable() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L887
• A writable() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L943
• A tell() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L989-L990
• A write() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L1626-L1628
• A flush() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L1759-L1762
• A read() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L1977
• A seekable() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L1228
• A seek() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L2568
• A truncate() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L2943
• A fileno() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L3025
• An isatty() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L3077
• A close() method: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L3137
• A name attribute: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L3195
• A closed attribute: https://github.com/python/cpython/blob/dd5e4d90789b3a065290e264122629f31cb0b547/Modules/_io/textio.c#L3209

[srittau]

I've opened a draft PR: #11420. Let's see what our tests – especially primer – say.