Mypy complains about slicing a DateTimeIndex

[gandhis1]

Shortened code:

  reveal_type(start) # Union[datetime.datetime, datetime.date, pandas._libs.tslibs.timestamps.Timestamp, None]"
  reveal_type(end)  # Union[datetime.datetime, datetime.date, pandas._libs.tslibs.timestamps.Timestamp, None]"
  concat_df = concat_df.sort_index().loc[start:end, :]

Error:

error: Slice index must be an integer or None

I don't know yet whether this is a stubs issue or potentially a MyPy issue - could even be an issue with my code - I will investigate further

[gandhis1]

On second glance, this may be something where my code hasn't narrowed down the index type of DataFrame to a DatetimeIndex, and if it could do that, then this code would work. But how do I go about doing that properly?

[Dr-Irv]

I think you'll need a more complete example that can be tested by others.

Having said that, I get the same error on this simple code with mypy, so it seems it is a mypy problem, as pyright allows this:

import pandas as pd
df = pd.DataFrame({"c1":[1,2,3,4], "c2":[10,20,30,40]}, index=["a", "b", "c" , "d"])

x =  df.loc["b":"c", :]

Seems like a long standing issue. See
python/mypy#2410
python/typeshed#8647

I'm going to close this given that it seems to be a known issue with mypy

[Dr-Irv]

reopening so we can track if mypy ever fixes the bug

[gandhis1]

It seems that PEP 696 is going to be deferred indefinitely. I was thinking - given that now typeshed added SupportsIndex to slice, are there any downsides to adding an __index__ method to pd.Timestamp so that at least there's some semblance of support? It would be technically incorrect, but if there are no downsides, it would provide a stop-gap until the PEP moves forward. Of course, this wouldn't be viable with datetime or np.datetime64, and it doesn't fix the test case I put in the original post of this thread, but I suspect a good number of users are using pandas timestamps.

[Dr-Irv]

It seems that PEP 696 is going to be deferred indefinitely. I was thinking - given that now typeshed added SupportsIndex to slice, are there any downsides to adding an __index__ method to pd.Timestamp so that at least there's some semblance of support? It would be technically incorrect, but if there are no downsides, it would provide a stop-gap until the PEP moves forward. Of course, this wouldn't be viable with datetime or np.datetime64, and it doesn't fix the test case I put in the original post of this thread, but I suspect a good number of users are using pandas timestamps.

I'm open to a PR that does this. Should probably add a comment that you're adding __index__ to Timestamp to support slicing.

[Dr-Irv]

Still an issue with mypy 1.14.1. Another user reported the pandas issue at python/mypy#2410 (comment)

[gandhis1]

There was a recent typeshed update to make slice generic: python/typeshed#11637

I'm not sure if the latest Mypy includes that, but if it does, I would think that would allow supporting this

[Dr-Irv]

There was a recent typeshed update to make slice generic: python/typeshed#11637

I'm not sure if the latest Mypy includes that, but if it does, I would think that would allow supporting this

I just tried it with mypy 1.15.0+dev.bac9984a0e2696eed5d65ca49d006c545ba83a54 and slicing on strings doesn't work. If you do s.loc["a":"b"], mypy says that is incorrect. But s.loc[slice("a", "b", None)] is accepted.

[Dr-Irv]

I created #1080 that closes the original issue (slicing on Timestamp). That will close the original issue, but I created a new issue #1080 that is about the string slicing bug still in mypy.