Combine affine transformations

[Spenhouet]

Is your feature request related to a problem? Please describe.

All affine transformations perform an interpolation. In medical imaging this is very crucial since you loose information with every interpolation.
Since most libraries take this into account and collect and combine affine transformations to only perform a single transformation at one timepoint, we assumed that Compose is exactly doing that.
We were quite shocked to discover that MONAI actually does not do this and instead performs an interpolation on every individual transformation step (were an interpolation is required).

MONAI comes with a lot of functionality that we need and is a great tool. The fact above really shocked us since we assumed that this would be taken into account since MONAI is specifically for medical image data.
Was this overlooked during development? How come that this could happen?

Describe the solution you'd like
All spatial transformations should be collected and then combined down to as few transformations as possible. Especially for affine transformations.

[wyli]

thanks, great to see this type of dicussions, yes it is important and has been discussed since day one #112

Currently MONAI provides specific transforms to do the fused spatial transformations:

MONAI/monai/transforms/spatial/dictionary.py

Lines 555 to 573 in a0afa60

	class Affined(MapTransform, InvertibleTransform):
	"""
	Dictionary-based wrapper of :py:class:`monai.transforms.Affine`.
	"""
	def __init__(
	self,
	keys: KeysCollection,
	rotate_params: Optional[Union[Sequence[float], float]] = None,
	shear_params: Optional[Union[Sequence[float], float]] = None,
	translate_params: Optional[Union[Sequence[float], float]] = None,
	scale_params: Optional[Union[Sequence[float], float]] = None,
	spatial_size: Optional[Union[Sequence[int], int]] = None,
	mode: GridSampleModeSequence = GridSampleMode.BILINEAR,
	padding_mode: GridSamplePadModeSequence = GridSamplePadMode.REFLECTION,
	as_tensor_output: bool = False,
	device: Optional[torch.device] = None,
	allow_missing_keys: bool = False,
	) -> None:

MONAI/monai/transforms/spatial/dictionary.py

Lines 826 to 846 in a0afa60

	class Rand2DElasticd(RandomizableTransform, MapTransform):
	"""
	Dictionary-based wrapper of :py:class:`monai.transforms.Rand2DElastic`.
	"""
	def __init__(
	self,
	keys: KeysCollection,
	spacing: Union[Tuple[float, float], float],
	magnitude_range: Tuple[float, float],
	spatial_size: Optional[Union[Tuple[int, int], int]] = None,
	prob: float = 0.1,
	rotate_range: Optional[Union[Sequence[Union[Tuple[float, float], float]], float]] = None,
	shear_range: Optional[Union[Sequence[Union[Tuple[float, float], float]], float]] = None,
	translate_range: Optional[Union[Sequence[Union[Tuple[float, float], float]], float]] = None,
	scale_range: Optional[Union[Sequence[Union[Tuple[float, float], float]], float]] = None,
	mode: GridSampleModeSequence = GridSampleMode.BILINEAR,
	padding_mode: GridSamplePadModeSequence = GridSamplePadMode.REFLECTION,
	as_tensor_output: bool = False,
	device: Optional[torch.device] = None,
	allow_missing_keys: bool = False,

This is a generic discussion, I'm converting it to the discussion tab.

[Spenhouet]

@wyli Good to hear that this is already on the radar.

We are already using the dictionary based transforms in our pipeline. Are you saying that the Affined transform already performs this fusing? It did not look like it to me but I might have misunderstood the code.

[Nic-Ma]

Hi @Spenhouet ,

Glad to see your interest and experiments with MONAI.
For more details of the fused transform(Affine, Elastic deformation, etc.), you can take a look at:
https://github.com/Project-MONAI/tutorials/blob/master/modules/3d_image_transforms.ipynb
https://github.com/Project-MONAI/tutorials/blob/master/acceleration/transform_speed.ipynb

Thanks.