About the key point detection

[zhongyingji]

❓ Questions and Help

Hi, I found out that when KEYPOINT_HEAD.SHARE_BOX_FEATURE_EXTRACTOR=True, the feature_extractor of keypoint_head becomes identical to the ones of bbox_head, which composed of pooler+conv.
In roi_heads.py, when keypoint_head shares feature extractor with bbox_head, calling self.keypoint means another roi alignment? I mean self.keypoint(keypoint_features, detections, targets), the keypoint_features is the result of bbox_head extractor.
Here comes the error:

File "/home/zyj/maskrcnn-benchmark-master/maskrcnn_benchmark/layers/roi_align.py", line 59, in forward
input, rois, self.output_size, self.spatial_scale, self.sampling_ratio
File "/home/zyj/maskrcnn-benchmark-master/maskrcnn_benchmark/layers/roi_align.py", line 20, in forward
input, roi, spatial_scale, output_size[0], output_size[1], sampling_ratio
RuntimeError: Dimension out of range (expected to be in range of [-3, 2], but got 3) (maybe_wrap_dim at /opt/conda/conda-bld/pytorch-nightly_1553145022820/work/c10/core/WrapDimMinimal.h:20)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x45 (0x7fcd173ade05 in /home/jianzhong/anaconda3/envs/zyj_py3/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: at::native::size(at::Tensor const&, long) + 0x196 (0x7fcd0d9ceb36 in /home/jianzhong/anaconda3/envs/zyj_py3/lib/python3.6/site-packages/torch/lib/libcaffe2.so)
frame #2: ROIAlign_forward_cuda(at::Tensor const&, at::Tensor const&, float, int, int, int) + 0x30f (0x7fccaf6f9251 in /home/zyj/maskrcnn-benchmark-master/maskrcnn_benchmark/_C.cpython-36m-x86_64-linux-gnu.so)
frame #3: ROIAlign_forward(at::Tensor const&, at::Tensor const&, float, int, int, int) + 0x95 (0x7fccaf6d8105 in /home/zyj/maskrcnn-benchmark-master/maskrcnn_benchmark/_C.cpython-36m-x86_64-linux-gnu.so)
frame #4: + 0x2a924 (0x7fccaf6e5924 in /home/zyj/maskrcnn-benchmark-master/maskrcnn_benchmark/_C.cpython-36m-x86_64-linux-gnu.so)
frame #5: + 0x2a9ae (0x7fccaf6e59ae in /home/zyj/maskrcnn-benchmark-master/maskrcnn_benchmark/_C.cpython-36m-x86_64-linux-gnu.so)

[Smoothing97]

@zhongyingji sorry but I also have this problem,if you have any sloutions?thanks!!

[AlenUbuntu]

Yes. I figure out that in combined_roi_head, we mark the KEYPOINT_HEAD.SHARE_BOX_FEATURE_EXTRACTOR = True, we are using features extracted from box_head, i.e., a tensor. However, if we check the keypoint head code, it will again call a keypoint_feature_extractor which includes a Pooler. This Pooler should receive a List of tensors and proposals. I think it is where the error occurs.

Could you help me to address this issue?

[2294462474]

I think the number of classes is wrong, according to the paper, there are 2 classes, one for background, and one for foreground, which is the key points.
you can check with the annotations keypoints of coco2017.