Timestamp and DatetimeIndex construct different timezones

[TomAugspurger]

Question in the form of an issue: is this a bug?

In [37]: pd.Timestamp('2000', tz='US/Central').tz == pd.DatetimeIndex(['2000'], tz='US/Central').tz
Out[37]: False

This leads to strange things like

In [38]: idx = pd.date_range('20170101', periods=4, tz='US/Pacific')

In [39]: idx[0].tz == idx.tz
Out[39]: False

[TomAugspurger]

DatetimeIndex (via, DatetimeArrayMixin) goes

        tz = timezones.maybe_get_tz(tz)
        result._tz = timezones.tz_standardize(tz)

Timestmamp.__new__ seems to use a tzinfo object.

So it seems like I should be using _libs.tslibs.timezones.tz_compare, rather than ==.

[jbrockmendel]

Can you show the tzinfo objects? Tzinfo equality comparisons have weird semantics.

[TomAugspurger]

In [2]: a = pd.Timestamp('2000', tz='US/Central')

In [3]: b = pd.DatetimeIndex(['2000'], tz='US/Central')

In [4]: a.tz
Out[4]: <DstTzInfo 'US/Central' CST-1 day, 18:00:00 STD>

In [5]: b.tz
Out[5]: <DstTzInfo 'US/Central' LMT-1 day, 18:09:00 STD>

Just to be clear, this came from a base ExtensionArray test. If this comes down to "timezones are weird" I'm happy to note that in the docs and skip the test.

[mroeschke]

These are expected to be unequal due to pytz semantics.

For a pytz timezone with a Timestamp, we are only dealing with one date, so we can confidently know which side of DST border the date lies and can use the DST informed tzinfo instance (CST-1 day).

For a pytz timezone with a DatetimeIndex, we are potentially dealing with 1+ dates where a mix of dates can lie on either side of the DST border. Therefore we cannot easily use one pytz timezone instance to represent all these dates and just default to the "first" pytz instance for a timezone (LMT-1 day)

[TomAugspurger]

Fun... I'll try to turn this into a new docs section in http://pandas-docs.github.io/pandas-docs-travis/timeseries.html#working-with-time-zones

Should we add tz_compare to our public API, as the recommended way to compare to tz objects?

[mroeschke]

Yeah clarifying this pytz idiosyncrasy in the docs would be a nice addition.

@jbrockmendel is the spec for tz_compare to see if two timezone objects represent the same timezone or to check if two timezone instances are the same?

[jbrockmendel]

To see if they represent the same timezone.

[mroeschke]

Gotcha. My impression was that tz_compare was a utility for the internals, but I guess if users may find it useful then we can expose it as a toplevel "pd.tz_compare". Probably would need a lot more tests though to nail down the spec.

[jorisvandenbossche]

See also #17572 and #18595 where this has been discussed before.