Support instance masks (N,H,W)

[ashnair1]

Changes

Fixes #2855 related to #941

Type of change

• 🧪 Tests Cases
• 🐞 Bug fix (non-breaking change which fixes an issue)

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Did you update CHANGELOG in case of a major change?

[shijianjian]

Meanwhile, what we did for the boxes is to merge them all into one tensor, which is

kornia/kornia/geometry/boxes.py

Lines 20 to 33 in 46a5e40

	def _merge_box_list(boxes: list[torch.Tensor], method: str = "pad") -> tuple[torch.Tensor, list[int]]:
	r"""Merge a list of boxes into one tensor."""
	if not all(box.shape[-2:] == torch.Size([4, 2]) and box.dim() == 3 for box in boxes):
	raise TypeError(f"Input boxes must be a list of (N, 4, 2) shaped. Got: {[box.shape for box in boxes]}.")
	if method == "pad":
	max_N = max(box.shape[0] for box in boxes)
	stats = [max_N - box.shape[0] for box in boxes]
	output = torch.nn.utils.rnn.pad_sequence(boxes, batch_first=True)
	else:
	raise NotImplementedError(f"`{method}` is not implemented.")
	return output, stats

Not sure if we should do the same to the masks. Can anyone benchmark on which approach runs faster? Iteration or batching?

[ashnair1]

Merging a list of tensors into tensors will not work for instance segmentation. This is because each image will have a different number of objects and so different number of masks. Because of this, they cannot be batched.

This is how it would look like:
Single sample: Image (C, H, W) and Mask (N, H, W)
Batched: Image (B, C, H, W) and Mask ([(N₁, H, W), (N₂, H, W) ... (N_b, H, W)])

N (no. of detections) varies from image to image, so they cannot be batched.

Refer to #2417 for a proper example of this use case.

[shijianjian]

Single sample: Image (C, H, W) and Mask (N, H, W)
Batched: Image (B, C, H, W) and Mask ([(N1, H, W), (N2, H, W) ... (Nb, H, W)])

I meant that

N_max = max(N1, ..., Nb)
padding = [N_max - N1, ..., N_max - Nb]
padded_masks = (B, N_max, H, W)

... # After augmentation

Unpad (B, N_max, H, W) => Mask ([(N1, H, W), (N2, H, W) ... (Nb, H, W)])

The current change is fine in this PR I think. We should benchmark the iterative and batching strategies to see how those methods perform. Maybe a Mask class, similar to Boxes, to handle these.