Fix multiindex level serialization after reset_index

[benbovy]

• Closes objects remain unserializable after reset_index  #8628
• Tests added
• User visible changes (including notable bug fixes) are documented in whats-new.rst

[dcherian]

Don't fully understand but happy to defer to you.

[slevang]

This does not seem to have fixed xarray-contrib/xeofs#148, which worked prior to 2024.1.0

[benbovy]

@slevang could you provide an example (MVCE), please?

[slevang]

An MCVE with xeofs is:

import xarray as xr
import xeofs

data = xr.tutorial.open_dataset('air_temperature').air
model = xeofs.models.EOF(n_modes=10)
model.fit(data, "time")
model.save("test.zarr")

File ~/xarray/xarray/conventions.py:92, in ensure_not_multiindex(var, name)
     90     name = var.name
     91 if var.dims == (name,):
---> 92     raise NotImplementedError(
     93         f"variable {name!r} is a MultiIndex, which cannot yet be "
     94         "serialized. Instead, either use reset_index() "
     95         "to convert MultiIndex levels into coordinate variables instead "
     96         "or use https://cf-xarray.readthedocs.io/en/latest/coding.html."
     97     )

NotImplementedError: variable 'feature' is a MultiIndex, which cannot yet be serialized. Instead, either use reset_index() to convert MultiIndex levels into coordinate variables instead or use https://cf-xarray.readthedocs.io/en/latest/coding.html.

I'm trying to figure out a pure xarray version though. We should just be doing the same stack(feature=[...]) and reset_index("feature") ops as in #8628 but there must be something else in there.

[slevang]

Looks like the key difference is that in the previous issue, isinstance(var, IndexVariable) evaluates to False, then True in my case so name gets set and trips the error. Not sure how we get there.

[benbovy]

Ok, we are still raising that error (and we shouldn't do so) in the case where a reset multi-index level coordinate has been renamed to its dimension:

import pandas as pd
import xarray as xr

midx = pd.MultiIndex.from_product([["a", "b"]], names=["foo"])
coords = xr.Coordinates.from_pandas_multiindex(midx, "x")

ds = xr.Dataset(coords=coords)
ds_reset = ds.reset_index("x").rename_vars(foo="x")

type(ds_reset.x.variable._data)
# xarray.core.indexing.PandasMultiIndexingAdapter

ds_reset.to_netcdf("test.nc")
# error

Instead of adding more checks like in this PR, I think that a proper fix would be to unwrap the multi-index level coordinate variable data from xarray.core.indexing.PandasMultiIndexingAdapter (e.g., convert it to numpy arrays) after resetting or dropping the multi-index.

[slevang]

@benbovy any more guidance on how to go about fixing this edge case? I can try a PR but am pretty unfamiliar with the new indexers.