Cannot access member "hex" for type "ByteString"

[anacrolix]

BPO	41564
Nosy	@ericvsmith, @anacrolix, @websurfer5
Files	hex.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-08-17.02:50:33.866>
labels = ['3.8', 'type-bug', 'library']
title = 'Cannot access member "hex" for type "ByteString"'
updated_at = <Date 2020-08-24.02:06:57.496>
user = 'https://github.com/anacrolix'

bugs.python.org fields:

activity = <Date 2020-08-24.02:06:57.496>
actor = 'Jeffrey.Kintscher'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-08-17.02:50:33.866>
creator = 'anacrolix'
dependencies = []
files = ['49423']
hgrepos = []
issue_num = 41564
keywords = []
message_count = 8.0
messages = ['375523', '375528', '375553', '375554', '375831', '375832', '375833', '375834']
nosy_count = 3.0
nosy_names = ['eric.smith', 'anacrolix', 'Jeffrey.Kintscher']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41564'
versions = ['Python 3.8']

[anacrolix]

I get this error when running pyright for a type of typing.ByteString. All the implementations of ByteString (bytes, bytearray, memoryview) have the hex method, so this seems unexpected?

[ericvsmith]

Without some example code that shows the problem we can't help you. Have you considered that this is a bug with pyright, not a bug with python itself?

[anacrolix]

cpython/Lib/_collections_abc.py

Line 953 in 48b069a

XXX Should add all their methods.

cpython/Lib/typing.py

Line 1612 in 0e95bbf

ByteString = _alias(collections.abc.ByteString, 0) # Not generic

[ericvsmith]

We need to know how to trigger the problem you're seeing. You need to provide code we can run that shows the error you're seeing.

[anacrolix]

$ pyright hex.py 
stubPath /Users/anacrolix/src/dht-scraper/typings is not a valid directory.
Assuming Python platform Darwin
Searching for source files
Found 1 source file
/Users/anacrolix/src/dht-scraper/hex.py
  3:9 - error: Cannot access member "hex" for type "ByteString"
  Member "hex" is unknown (reportGeneralTypeIssues)
1 error, 0 warnings 
Completed in 0.562sec
anacrolix@anacrolix-mbp-2018:~/src/dht-scraper$ mypy hex.py 
hex.py:3: error: "ByteString" has no attribute "hex"
Found 1 error in 1 file (checked 1 source file)
anacrolix@anacrolix-mbp-2018:~/src/dht-scraper$ python3 hex.py 
deadbeef

[ericvsmith]

This looks like a problem in pyright, not in CPython.

[anacrolix]

I do not think so. mypy has the same issue. The ByteString type does not include the methods shared by all its implementations. I already linked to this in https://bugs.python.org/msg375553. I also showed that mypy doesn't work in my last comment.

[websurfer5]

hex.py works for me with CPython versions 3.5, 3.7, 3.8, and 3.9, and the master branch.

[martindemello]

based on this note in Lib/_collections_abc.py this might require creating a ByteString subclass:

# ABCs are different from other standard library modules in that they
# specify compliance tests.  In general, once an ABC has been published,
# new methods (either abstract or concrete) cannot be added.
#
# Though classes that inherit from an ABC would automatically receive a
# new mixin method, registered classes would become non-compliant and
# violate the contract promised by ``isinstance(someobj, SomeABC)``.
#
# Though irritating, the correct procedure for adding new abstract or
# mixin methods is to create a new ABC as a subclass of the previous
# ABC.  For example, union(), intersection(), and difference() cannot
# be added to Set but could go into a new ABC that extends Set.

[hauntsaninja]

First, note that static type checkers don't actually look at code in CPython, they look at the definition in typeshed: https://github.com/python/typeshed/blob/1d7ace353a4c96b6a279ed0df9179118467da7c7/stdlib/typing.pyi#L708
As you can see, this means that typing.ByteString is just a glorified Sequence[int].

But it is true that typeshed is just reflecting the code that's in CPython, so decisions should be made here. The interface for ByteString is defined here:

cpython/Lib/_collections_abc.py

Line 1090 in 1ae7abf

XXX Should add all their methods.

. You'll see there's an XXX comment that maybe suggests that this wasn't as thought out as it could be.

I don't think this is well specified. As I mention here https://discuss.python.org/t/pep-688-making-the-buffer-protocol-accessible-in-python/15265/16 , if you intersect bytes, bytearray and memoryview you get a Sequence[int] + .hex and nothing else, so hex is really the only thing missing.

Overall, I vote to close this. It's probably problematic from a compatibility standpoint to change interfaces in collections.abc.

If you want to type your code loosely, just use bytes (type checkers will do the right thing, although PEP 688 proposes removing the shorthand in case you intend the type to also allow bytearray and memoryview: https://peps.python.org/pep-0688/#no-special-meaning-for-bytes )

If you want to type your code exactly, use a custom protocol, e.g.:

class Hex(Protocol):
    def hex(self) -> str: ...
    def __getitem__(self, int) -> int: ...

[hauntsaninja]

Also see the discussion at #91896