downsampling results in 0 or 1 raster without float

[pz-max]

Description

Pictures tell more than words

Expected Behavior

Actual Behavior

Error Message

Your Environment

• The atlite version used:
• How you installed atlite (conda, pip or github):
• Operating System:
• My environment:
  (output of `conda list`)

  ``` ```

[pz-max]

Trying to debug this if others face the same issue

[pz-max]

I am trying to understand why the cutout raster has either a 0 or a 1 availability and nothing in between... ( as the expected image behaviour would suggest). I was looking now into the function shape_availability_reprojected() in gis.py. Here in particular rasterio.warp.reproject() is used to downsample (high-resolution raster -> low-resolution raster).

Looking into the variables:

• masked represents the fine-grained shape e.g. in case of croatia an array/raster of the shape (4876, 5065)
• dst_shape represents the coarse-grained shape e.g. in case of croatia an array of the shape (31, 45)
• resampling represents the method of how we upsample the shape. 5 is method "average". There a many more methods available (here). One can be here also more explicit like replacing "5" with "rasterio.enums.Resampling.average" (tested and works)
• the other cars stuff indicated the CRS of each raster & the transformation (relation between the raster resolutions) is required for upsampling...

So here I am with a bit more knowledge but haven't found the cause of the problem. Maybe it has something to do with the "src_nodata" stuff described here. :)

atlite/atlite/gis.py

Lines 473 to 519 in e3d0865

	def shape_availability_reprojected(
	geometry, excluder, dst_transform, dst_crs, dst_shape
	):
	"""
	Compute and reproject the eligible area of one or more geometries.
	The function executes `shape_availability` and reprojects the calculated
	mask onto a new raster defined by (dst_transform, dst_crs, dst_shape).
	Before reprojecting, the function pads the mask such all non-nodata data
	points are projected in full cells of the target raster. The ensures that
	all data within the mask are projected correclty (GDAL inherent 'problem').
	----------
	geometry : geopandas.Series
	Geometry in which the eligible area is computed. If the series contains
	more than one geometry, the eligble area of the combined geometries is
	computed.
	excluder : atlite.gis.ExclusionContainer
	Container of all meta data or objects which to exclude, i.e.
	rasters and geometries.
	dst_transform : rasterio.Affine
	Transform of the target raster.
	dst_crs : rasterio.CRS/proj.CRS
	CRS of the target raster.
	dst_shape : tuple
	Shape of the target raster.
	masked : np.array
	Average share of available area per grid cell. 0 indicates excluded,
	1 is fully included.
	transform : rasterio.Affine
	Affine transform of the mask.
	"""
	masked, transform = shape_availability(geometry, excluder)
	masked, transform = pad_extent(
	masked, transform, dst_transform, excluder.crs, dst_crs
	)
	return rio.warp.reproject(
	masked,
	empty(dst_shape),
	resampling=5,
	src_transform=transform,
	dst_transform=dst_transform,
	src_crs=excluder.crs,
	dst_crs=dst_crs,
	)

[FabianHofmann]

@pz-max thank you very much for testing this, this was extremely important. I found the error. Rasterio changed the order of the resampling methods (without a warning). I'll push a fix with a safe method selection.

[pz-max]

I can confirm it is still not working. :)

[pz-max]

Fixed the issue PR is coming @FabianHofmann

[pz-max]

Reproduce the error

git clone https://github.com/PyPSA/atlite.git
cd atlite
conda env create -f environment.yaml  # will install rasterio 1.2.10
pip install atlite  # install to test the examples
pytest

Leads to errors

Which I could reproduce in the examples:

[pz-max]

Performing then pip install 'rasterio<1.2.10' fixes the issue (downgrades to 1.2.9).
Testing now the implementation into the atlite envs

[pz-max]

constraining rasterio in the environment.yaml to

- rasterio>=1.0,<1.2.10  # until 1.3.0 is released

solves the issue we experienced (even though the anaconda installation quite a bit longer). The solved rasterio version is 1.2.8 (yes not 1.2.9)

I checked the pytest to confirm everything runs well

One note, the rasterio team has a pre-release in the beta (v.1.3.0) with several improvements that also impact the downsampling/upsampling. This beta version also solves our issues (I tested it). Once fully released we should change our package constraints.