Problem when using pcolormesh with matplotlib 3.8

[l4nc3l0t]

Description

Problem with colorbar when using pcolormesh with matplotlib 3.8 everything work with 3.7.2 something to do with masked data

(Try to replace pcolormesh by pcolor but another problem ('Colorbar' object has no attribute '_boundaries')

Don't know if it's a Cartopy or Matplotlib problem...

Traceback

File ~/Documents/***/.env/lib/python3.11/site-packages/matplotlib/pyplot.py:2230, in colorbar(mappable, cax, ax, **kwargs)
2225 if mappable is None:
2226 raise RuntimeError('No mappable was found to use for colorbar '
2227 'creation. First define a mappable such as '
2228 'an image (with imshow) or a contour set ('
2229 'with contourf).')
-> 2230 ret = gcf().colorbar(mappable, cax=cax, ax=ax, **kwargs)
2231 return ret

File ~/Documents/***/.env/lib/python3.11/site-packages/matplotlib/figure.py:1295, in FigureBase.colorbar(self, mappable, cax, ax, use_gridspec, **kwargs)
1291 cax.grid(visible=False, which='both', axis='both')
1293 NON_COLORBAR_KEYS = [ # remove kws that cannot be passed to Colorbar
1294 'fraction', 'pad', 'shrink', 'aspect', 'anchor', 'panchor']
-> 1295 cb = cbar.Colorbar(cax, mappable, **{
1296 k: v for k, v in kwargs.items() if k not in NON_COLORBAR_KEYS})
1297 cax.figure.stale = True
1298 return cb

File ~/Documents/***/.env/lib/python3.11/site-packages/matplotlib/colorbar.py:395, in Colorbar.init(self, ax, mappable, cmap, norm, alpha, values, boundaries, orientation, ticklocation, extend, spacing, ticks, format, drawedges, extendfrac, extendrect, label, location)
392 self.ticklocation = ticklocation
394 self.set_label(label)
--> 395 self._reset_locator_formatter_scale()
397 if np.iterable(ticks):
398 self._locator = ticker.FixedLocator(ticks, nbins=len(ticks))

File ~/Documents/***/.env/lib/python3.11/site-packages/matplotlib/colorbar.py:1159, in Colorbar._reset_locator_formatter_scale(self)
1153 def _reset_locator_formatter_scale(self):
1154 """
1155 Reset the locator et al to defaults. Any user-hardcoded changes
1156 need to be re-entered if this gets called (either at init, or when
1157 the mappable normal gets changed: Colorbar.update_normal)
1158 """
-> 1159 self._process_values()
1160 self._locator = None
1161 self._minorlocator = None

File ~/Documents/***/.env/lib/python3.11/site-packages/matplotlib/colorbar.py:1087, in Colorbar._process_values(self)
1084 b = np.hstack((b, b[-1] + 1))
1086 # transform from 0-1 to vmin-vmax:
-> 1087 if self.mappable.get_array() is not None:
1088 self.mappable.autoscale_None()
1089 if not self.norm.scaled():
1090 # If we still aren't scaled after autoscaling, use 0, 1 as default

File ~/Documents/***/.env/lib/python3.11/site-packages/cartopy/mpl/geocollection.py:24, in GeoQuadMesh.get_array(self)
22 # If the input array has a mask, retrieve the associated data
23 if hasattr(self, '_wrapped_mask'):
---> 24 A[self._wrapped_mask] = self._wrapped_collection_fix.get_array()
25 return A

File /usr/lib/python3.11/contextlib.py:81, in ContextDecorator.call..inner(*args, **kwds)
78 @wraps(func)
79 def inner(*args, **kwds):
80 with self._recreate_cm():
---> 81 return func(*args, **kwds)

File ~/Documents/***/.env/lib/python3.11/site-packages/numpy/ma/core.py:3385, in MaskedArray.setitem(self, indx, value)
3383 _data[indx.data] = dval
3384 else:
-> 3385 _data[indx] = dval
3386 _mask[indx] = mval
3387 elif hasattr(indx, 'dtype') and (indx.dtype == MaskType):

Full environment definition

Operating system

Ubuntu 23.04

Cartopy version

0.21.1

Matplotlib version

3.8.0.dev1496+g2c45571872

[rcomer]

Hi @l4nc3l0t thanks for the report. Could you post a self-contained code example that reproduces the problem? I have been doing some testing against the Matplotlib development branch and the above traceback is not reproduced in the tests. So I think we would need to understand what about your example is not covered by this test:

cartopy/lib/cartopy/tests/mpl/test_mpl_integration.py

Lines 270 to 314 in c4858e4

	def test_pcolormesh_get_array_with_mask():
	# make up some realistic data with bounds (such as data from the UM)
	nx, ny = 36, 18
	xbnds = np.linspace(0, 360, nx, endpoint=True)
	ybnds = np.linspace(-90, 90, ny, endpoint=True)
	x, y = np.meshgrid(xbnds, ybnds)
	data = np.exp(np.sin(np.deg2rad(x)) + np.cos(np.deg2rad(y)))
	data = data[:-1, :-1]
	fig = plt.figure()
	ax = fig.add_subplot(2, 1, 1, projection=ccrs.PlateCarree())
	c = ax.pcolormesh(xbnds, ybnds, data, transform=ccrs.PlateCarree())
	assert c._wrapped_collection_fix is not None, \
	'No pcolormesh wrapping was done when it should have been.'
	result = c.get_array()
	assert not np.ma.is_masked(result)
	assert np.array_equal(data.ravel(), result), \
	'Data supplied does not match data retrieved in wrapped case'
	ax.coastlines()
	ax.set_global() # make sure everything is visible
	# Case without wrapping
	nx, ny = 36, 18
	xbnds = np.linspace(-60, 60, nx, endpoint=True)
	ybnds = np.linspace(-80, 80, ny, endpoint=True)
	x, y = np.meshgrid(xbnds, ybnds)
	data = np.exp(np.sin(np.deg2rad(x)) + np.cos(np.deg2rad(y)))
	data2 = data[:-1, :-1]
	ax = fig.add_subplot(2, 1, 2, projection=ccrs.PlateCarree())
	c = ax.pcolormesh(xbnds, ybnds, data2, transform=ccrs.PlateCarree())
	ax.coastlines()
	ax.set_global() # make sure everything is visible
	assert getattr(c, "_wrapped_collection_fix", None) is None, \
	'pcolormesh wrapping was done when it should not have been.'
	result = c.get_array()
	assert not np.ma.is_masked(result)
	assert np.array_equal(data2.ravel(), result), \
	'Data supplied does not match data retrieved in unwrapped case'

[l4nc3l0t]

Seems to be a problem with nan in the rows if I insert np.insert(data, 2, np.nan, axis=1) it works...

def minimal_cb_pbl():
    nx, ny = 36, 18
    xbnds = np.linspace(
        0,
        360,
        nx,
        endpoint=True)
    ybnds = np.linspace(
        -90,
        90,
        ny,
        endpoint=True)

    x, y = np.meshgrid(xbnds, ybnds)
    data = np.exp(np.sin(np.deg2rad(x)) + np.cos(np.deg2rad(y)))
    data = data[:-1, :-1]
# insert nan in row
    data = np.insert(data, 2, np.nan, axis=0)
    fig = plt.figure()

    ax = fig.add_subplot(2, 1, 1, projection=ccrs.PlateCarree())
    c = ax.pcolormesh(x,
                                      y,
                                      data,
                                      transform=ccrs.PlateCarree())
    cb1 = plt.colorbar(c,
                                     ax=ax)

    ax.coastlines()
    ax.set_global()  # make sure everything is visible
    plt.show()

[greglucas]

Matplotlib 3.8 isn't out yet, so just to verify, you are testing this on the main source branch of MPL?

If so, I think we still need @rcomer's PR in Cartopy: #2166 when using that branch and main MPL your examples work for me as expected. @rcomer is that ready for review now?

[l4nc3l0t]

I'm using the nightly build from https://matplotlib.org/stable/users/installing/index.html#installing-a-nightly-build

That's right using the https://github.com/rcomer/cartopy/pcolormesh-rgba branch solves the issue

[rcomer]

Thanks for the clear example. I have rebased #2166 and added a row of nans to the get_array test. I still want to add a basic test for the set_array non-wrapped case. If that passes, then I think it will be ready for review.