feat(image_projection_based_fusion): substitute message filter with custom synchronizer

[tzhong518]

Signed-off-by: tzhong518 sworgun@gmail.com

Description

message_filters::sync::ApproximateTime in ROS2 is designed to collect fixed-size sets of messages, which means the callback will not be called unless all messages are subscribed.
As a result, if a roi message is delayed or missed, not only the waiting time will be long but also the set it belongs to might be discarded.
On the other hand, offsets exist between the timestamps of camera and lidar, which is determined by their shutter time. The matching could be more accurate if the offsets are considered.

Therefore, we decided to use a custom synchronizer fusion callback instead of the messge_filter, which

• adds the offsets to the pointcloud message when matching it with the rois messages
• sets timeout in case the message is delayed, so that the painted pointcloud will still be publised even if not all cameras are painted
• starts fuseOnSingleImage as soon as the roi msg is received to avoid waiting time

Related links

ticket: https://tier4.atlassian.net/browse/T4PB-21187?focusedCommentId=98418&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-98418
confluence pages: https://tier4.atlassian.net/wiki/spaces/~459732634/pages/2597454815
https://tier4.atlassian.net/wiki/spaces/~459732634/pages/2599911881
rosbag with yolo's result: https://drive.google.com/drive/folders/1cqXByizuWT22nYx9j_5fB42lyGJk9pzg?usp=sharing

Tests performed

Notes for reviewers

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

• I've confirmed the contribution guidelines.
• The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

• The PR follows the pull request guidelines.
• The PR has been properly tested.
• The PR has been reviewed by the code owners.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

• There are no open discussions or they are tracked via tickets.
• The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

[yukke42]

@tzhong518 Cloud you describe how you checked this PR solves the issue we discussed before and the results of it in the Description?

[codecov]

Codecov Report

Base: 10.82% // Head: 10.79% // Decreases project coverage by -0.02% ⚠️

Coverage data is based on head (ce65279) compared to base (dd52915).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2175      +/-   ##
==========================================
- Coverage   10.82%   10.79%   -0.03%     
==========================================
  Files        1178     1178              
  Lines       84623    84798     +175     
  Branches    19945    19945              
==========================================
  Hits         9158     9158              
- Misses      65755    65930     +175     
  Partials     9710     9710              

Flag	Coverage Δ		*Carryforward flag
differential	0.00% <0.00%> (?)
total	10.79% <0.00%> (ø)		Carriedforward from 28c0658

*This pull request uses carry forward flags. Click here to find out more.

Impacted Files	Coverage Δ
...lude/image_projection_based_fusion/fusion_node.hpp	0.00% <ø> (ø)
.../image_projection_based_fusion/src/fusion_node.cpp	0.00% <0.00%> (ø)
...ion_based_fusion/src/pointpainting_fusion/node.cpp	0.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[yukke42]

Suggested change

	if (!input_offset_ms_.empty() && rois_number_ != input_offset_ms_.size()) {
	RCLCPP_ERROR(get_logger(), "The number of topics does not match the number of offsets.");
	return;
	}
	if (input_offset_ms_.empty() || rois_number_ != input_offset_ms_.size()) {
	throw std::runtime_error("The number of offsets does not match the number of topics.");
	}

[tzhong518]

May I ask what is the difference?

[yukke42]

Your program just returns from the constructor and continues without any error in your code, but we want the program to exit with error if rois_number and the size of offsets don't matches.

[yukke42]

@tzhong518
The output from pointpainting isn't available when I play the bag with the option of --remap /perception/object_recognition/detection/rois0:=/mapped/perception/object_recognition/detection/rois0 to produce the situation that the one of the ROI topics is missed. Cloud you check?

[yukke42]

[debug messages]

[yukke42]

LGTM