generating instance label map at inference

[yaoliUoA]

Hi

Given a trained mask-rcnn model, I want to geneate an instance label map at inference time, which encodes each instance with an unique label (background with label 0). It is not trivial as sometimes the segmentations generated from mask-rcnn can be overlapping, ie, one pixel can assigned to multiple instances. Does this package support generating instance label map? Are there any functions in the package support this?

[fmassa]

Hi,

Check the demo that we have, in particular

maskrcnn-benchmark/demo/predictor.py

Lines 274 to 298 in 1589ce0

	def overlay_mask(self, image, predictions):
	"""
	Adds the instances contours for each predicted object.
	Each label has a different color.
	Arguments:
	image (np.ndarray): an image as returned by OpenCV
	predictions (BoxList): the result of the computation by the model.
	It should contain the field `mask` and `labels`.
	"""
	masks = predictions.get_field("mask").numpy()
	labels = predictions.get_field("labels")
	colors = self.compute_colors_for_labels(labels).tolist()
	for mask, color in zip(masks, colors):
	thresh = mask[0, :, :, None]
	contours, hierarchy = cv2_util.findContours(
	thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE
	)
	image = cv2.drawContours(image, contours, -1, color, 3)
	composite = image
	return composite

We don't guarantee that the pixels of the different objects is non-overlapping though.
This repo currently only addresses instance segmentation, and not panoptic segmentation.

[fmassa]

I'm closing this issue as we are tracking panoptic segmentation request in #337