core/util fails due to pandas 1.3 refactoring ABCIndexClass to ABCIndex

[michaelaye]

in core/util.py we got:

       if pandas_version >= '0.24.0':
            from pandas.core.dtypes.dtypes import DatetimeTZDtype as DatetimeTZDtypeType
            from pandas.core.dtypes.generic import ABCSeries, ABCIndexClass 

but since pandas 1.3 ABCIndexClass is ABCIndex:

https://github.com/pandas-dev/pandas/blame/f00ed8f47020034e752baf0250483053340971b0/pandas/core/dtypes/generic.py#L92

This little change leads to this monstrous traceback trying to do a hvplot on an xarray:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/IPython/core/formatters.py in __call__(self, obj, include, exclude)
    968 
    969             if method is not None:
--> 970                 return method(include=include, exclude=exclude)
    971             return None
    972         else:

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/core/dimension.py in _repr_mimebundle_(self, include, exclude)
   1315         combined and returned.
   1316         """
-> 1317         return Store.render(self)
   1318 
   1319 

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/core/options.py in render(cls, obj)
   1403         data, metadata = {}, {}
   1404         for hook in hooks:
-> 1405             ret = hook(obj)
   1406             if ret is None:
   1407                 continue

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/ipython/display_hooks.py in pprint_display(obj)
    280     if not ip.display_formatter.formatters['text/plain'].pprint:
    281         return None
--> 282     return display(obj, raw_output=True)
    283 
    284 

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/ipython/display_hooks.py in display(obj, raw_output, **kwargs)
    256     elif isinstance(obj, (HoloMap, DynamicMap)):
    257         with option_state(obj):
--> 258             output = map_display(obj)
    259     elif isinstance(obj, Plot):
    260         output = render(obj)

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/ipython/display_hooks.py in wrapped(element)
    144         try:
    145             max_frames = OutputSettings.options['max_frames']
--> 146             mimebundle = fn(element, max_frames=max_frames)
    147             if mimebundle is None:
    148                 return {}, {}

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/ipython/display_hooks.py in map_display(vmap, max_frames)
    204         return None
    205 
--> 206     return render(vmap)
    207 
    208 

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/ipython/display_hooks.py in render(obj, **kwargs)
     66         renderer = renderer.instance(fig='png')
     67 
---> 68     return renderer.components(obj, **kwargs)
     69 
     70 

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/plotting/renderer.py in components(self, obj, fmt, comm, **kwargs)
    408                 doc = Document()
    409                 with config.set(embed=embed):
--> 410                     model = plot.layout._render_model(doc, comm)
    411                 if embed:
    412                     return render_model(model, comm)

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/panel/viewable.py in _render_model(self, doc, comm)
    425         if comm is None:
    426             comm = state._comm_manager.get_server_comm()
--> 427         model = self.get_root(doc, comm)
    428 
    429         if config.embed:

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/panel/viewable.py in get_root(self, doc, comm, preprocess)
    482         """
    483         doc = init_doc(doc)
--> 484         root = self._get_model(doc, comm=comm)
    485         if preprocess:
    486             self._preprocess(root)

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/panel/layout/base.py in _get_model(self, doc, root, parent, comm)
    111         if root is None:
    112             root = model
--> 113         objects = self._get_objects(model, [], doc, root, comm)
    114         props = dict(self._init_params(), objects=objects)
    115         model.update(**self._process_param_change(props))

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/panel/layout/base.py in _get_objects(self, model, old_objects, doc, root, comm)
    101             else:
    102                 try:
--> 103                     child = pane._get_model(doc, root, model, comm)
    104                 except RerenderError:
    105                     return self._get_objects(model, current_objects[:i], doc, root, comm)

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/panel/pane/holoviews.py in _get_model(self, doc, root, parent, comm)
    237             plot = self.object
    238         else:
--> 239             plot = self._render(doc, comm, root)
    240 
    241         plot.pane = self

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/panel/pane/holoviews.py in _render(self, doc, comm, root)
    302                 kwargs['comm'] = comm
    303 
--> 304         return renderer.get_plot(self.object, **kwargs)
    305 
    306     def _cleanup(self, root):

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/plotting/bokeh/renderer.py in get_plot(self_or_cls, obj, doc, renderer, **kwargs)
     71         combining the bokeh model with another plot.
     72         """
---> 73         plot = super(BokehRenderer, self_or_cls).get_plot(obj, doc, renderer, **kwargs)
     74         if plot.document is None:
     75             plot.document = Document() if self_or_cls.notebook_context else curdoc()

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/plotting/renderer.py in get_plot(self_or_cls, obj, doc, renderer, comm, **kwargs)
    241             init_key = tuple(v if d is None else d for v, d in
    242                              zip(plot.keys[0], defaults))
--> 243             plot.update(init_key)
    244         else:
    245             plot = obj

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/plotting/plot.py in update(self, key)
    980     def update(self, key):
    981         if len(self) == 1 and ((key == 0) or (key == self.keys[0])) and not self.drawn:
--> 982             return self.initialize_plot()
    983         item = self.__getitem__(key)
    984         self.traverse(lambda x: setattr(x, '_updated', True))

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/plotting/bokeh/element.py in initialize_plot(self, ranges, plot, plots, source)
   1388             element = self.hmap.last
   1389         key = util.wrap_tuple(self.hmap.last_key)
-> 1390         ranges = self.compute_ranges(self.hmap, key, ranges)
   1391         self.current_ranges = ranges
   1392         self.current_frame = element

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/plotting/plot.py in compute_ranges(self, obj, key, ranges)
    636             if (not (axiswise and not isinstance(obj, HoloMap)) or
    637                 (not framewise and isinstance(obj, HoloMap))):
--> 638                 self._compute_group_range(group, elements, ranges, framewise,
    639                                           axiswise, robust, self.top_level,
    640                                           prev_frame)

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/plotting/plot.py in _compute_group_range(cls, group, elements, ranges, framewise, axiswise, robust, top_level, prev_frame)
    853                     continue
    854                 matching &= (
--> 855                     len({'date' if isinstance(v, util.datetime_types) else 'number'
    856                          for rng in rs for v in rng if util.isfinite(v)}) < 2
    857                 )

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/plotting/plot.py in <setcomp>(.0)
    854                 matching &= (
    855                     len({'date' if isinstance(v, util.datetime_types) else 'number'
--> 856                          for rng in rs for v in rng if util.isfinite(v)}) < 2
    857                 )
    858             if matching:

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/core/util.py in isfinite(val)
    902         return finite
    903     elif isinstance(val, datetime_types+timedelta_types):
--> 904         return not isnat(val)
    905     elif isinstance(val, (basestring, bytes)):
    906         return True

/luna4/maye/miniconda3/envs/py38/lib/python3.8/site-packages/holoviews/core/util.py in isnat(val)
    866     elif pd and val is pd.NaT:
    867         return True
--> 868     elif pd and isinstance(val, pandas_datetime_types+pandas_timedelta_types):
    869         return pd.isna(val)
    870     else:

NameError: name 'pandas_datetime_types' is not defined

How about adding another version edge-case:
So, https://github.com/holoviz/holoviews/blob/master/holoviews/core/util.py#L47-L55

becomes:

        if pandas_version >= '1.3.0':
            from pandas.core.dtypes.dtypes import DatetimeTZDtype as DatetimeTZDtypeType
            from pandas.core.dtypes.generic import ABCSeries, ABCIndex as ABCIndexClass
        elif pandas_version >= '0.24.0':
            from pandas.core.dtypes.dtypes import DatetimeTZDtype as DatetimeTZDtypeType
            from pandas.core.dtypes.generic import ABCSeries, ABCIndexClass
        elif pandas_version > '0.20.0':
            from pandas.core.dtypes.dtypes import DatetimeTZDtypeType
            from pandas.core.dtypes.generic import ABCSeries, ABCIndexClass
        else:
            from pandas.types.dtypes import DatetimeTZDtypeType
            from pandas.types.dtypes.generic import ABCSeries, ABCIndexClass

This has become quite ugly already, I wonder what methods are there (being no computer scientist), to avoid these issues? An API that would stay the same and the API would remap any refactors? That might only be possible to some extent, right?

[kgullikson88]

I just ran into the same thing - had to pin pandas<1.3 in my application for the time being. I would be happy to make a PR with the additional edge-case that @michaelaye suggested if that is all it takes

[hoxbro]

I also just ran into this problem. Hopefully, this can be fixed before the next release.

[philippjfr]

@kgullikson88 I think that's all that's needed, a PR would be massively appreciated.

[kgullikson88]

@philippjfr I'm not seeing any specific developer instructions so I will just start making the change and adding a test case if possible. Please let me know if there are more specific instructions/guidelines for making a PR.

[michaelaye]

fixed by #5013

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.