feat(tier4_perception_launch): add image segmentation based pointcloud filter #7225

badai-nguyen · 2024-06-03T01:29:46Z

Description

This is to add image semantic segmentation based pointcloud filter into perception pipeline to sold the issue Add a new pointcloud filter based on image semantic segmentation #5594
The default mode is still disable. Make sure the tensorrt_yolox output semantic segmentation mask image is output by merging feat(tensorrt yolox): inference and publish mask image from yolox model with semantic segmentation header #5553 and fix: add download url of yolox with semantic segmentation header autoware#4012 when the filter is enable.

Releated Link

TIER IV INTERNAL LINK

Tests performed

Not applicable.

Effects on system behavior

Not applicable.

Interface changes

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.
The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

The PR follows the pull request guidelines.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

…d filter Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

codecov · 2024-06-24T03:15:49Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 2.64%. Comparing base (507e3f4) to head (18047c0).
Report is 115 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #7225       +/-   ##
==========================================
- Coverage   14.84%   2.64%   -12.21%     
==========================================
  Files        1999     165     -1834     
  Lines      139163   10099   -129064     
  Branches    43716    1579    -42137     
==========================================
- Hits        20661     267    -20394     
+ Misses      95731    9776    -85955     
+ Partials    22771      56    -22715

Flag	Coverage Δ
differential	`2.64% <ø> (?)`
total	`?`

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…_launch

github-actions · 2024-07-05T03:26:02Z

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

You've checked our contribution guidelines.
Your PR follows our pull request guidelines.
All required CI checks pass before marking the PR ready for review.

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

technolojin

LGTM

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

YoshiRi

LGTM

launch/tier4_perception_launch/package.xml

This reverts commit 5adfef6.

yukkysaito

LGTM

…d filter (autowarefoundation#7225) * feat(tier4_perception_launch): add image segmentation based pointcloud filter Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: detection launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: add maintainer Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Revert "chore: add maintainer" This reverts commit 5adfef6. --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Signed-off-by: palas21 <palas21@itu.edu.tr>

…d filter (autowarefoundation#7225) * feat(tier4_perception_launch): add image segmentation based pointcloud filter Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: detection launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: add maintainer Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Revert "chore: add maintainer" This reverts commit 5adfef6. --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

…d filter (#7225) * feat(tier4_perception_launch): add image segmentation based pointcloud filter Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: detection launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: add maintainer Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Revert "chore: add maintainer" This reverts commit 5adfef6. --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

…d filter (autowarefoundation#7225) * feat(tier4_perception_launch): add image segmentation based pointcloud filter Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: detection launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: add maintainer Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Revert "chore: add maintainer" This reverts commit 5adfef6. --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

….29.0 (#1526) * feat(tensorrt yolox): inference and publish mask image from yolox model with semantic segmentation header (autowarefoundation#5553) * add segmentation model Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> fix: add multitask for segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: publish mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * feat: publish colorized mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: resize yolox mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add memory allocate operations Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * refactor: remove underscore for a local variable Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: add condition to check the number of subscriber for newly added topics Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: pre-commit Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add roi overlapping segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: roi overlay segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: refactor Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: update model name Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add utils into tensorrt_yolox Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: launch file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove unnecessary depend Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: fix yaml file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove duplicated param in launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: semantic class Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: update default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add processing time topic Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * chore: fix cspell error Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: yolox default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update model description Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * refactor: unpublish mask for single task Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * style(pre-commit): autofix * docs: update reamde Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: skip mask size as yolox output Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * fix(tensorrt_yolox): add run length encoding for sematic segmentation mask (autowarefoundation#7905) * fix: add rle compress Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: rle compress Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: move rle into utils Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: pre-commit Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/autoware_tensorrt_yolox/src/utils.cpp Co-authored-by: Yukihiro Saito <yukky.saito@gmail.com> * fix: remove unused variable Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/autoware_tensorrt_yolox/src/utils.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * style(pre-commit): autofix * feat: add unit test for utils Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: unit test Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: change to explicit index Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: cuda cmake Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: separate unit test into different PR Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Co-authored-by: Yukihiro Saito <yukky.saito@gmail.com> Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * refactor(tensorrt_yolox): move utils into perception_utils (autowarefoundation#8435) * chore(tensorrt_yolo): refactor utils Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: tensorrt_yolox Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix(perception_utils): install library after build (autowarefoundation#8501) Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * fix(image_projection_based_fusion): resize sematic segmentation mask as input image size (autowarefoundation#7635) Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix(image_projection_based_fusion): segmentation pointcloud fusion param update (autowarefoundation#7858) * fix(image_projection_based_fusion): add run length decoding for segmentation_pointcloud_fusion (autowarefoundation#7909) * fix: add rle decompress Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: use rld in tensorrt utils Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: rebase error Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: dependency Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: skip publish debug mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/autoware_image_projection_based_fusion/src/segmentation_pointcloud_fusion/node.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * style(pre-commit): autofix * Revert "fix: skip publish debug mask" This reverts commit 30fa9ae. --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * feat(tier4_perception_launch): add image segmentation based pointcloud filter (autowarefoundation#7225) * feat(tier4_perception_launch): add image segmentation based pointcloud filter Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: detection launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: add maintainer Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Revert "chore: add maintainer" This reverts commit 5adfef6. --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix(autoware_image_projection_based_fusion): resolve issue with segmentation pointcloud fusion node failing with multiple mask inputs (autowarefoundation#8769) --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> Co-authored-by: Yukihiro Saito <yukky.saito@gmail.com> Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> Co-authored-by: Yi-Hsiang Fang (Vivid) <146902905+vividf@users.noreply.github.com>

github-actions bot added component:perception Advanced sensor data processing and environment understanding. (auto-assigned) component:launch Launch files, scripts and initialization tools. (auto-assigned) labels Jun 3, 2024

badai-nguyen added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Jun 3, 2024

badai-nguyen mentioned this pull request Jun 10, 2024

update perception pipeline node diagram for segmenation_based_filter autowarefoundation/autoware-documentation#569

Closed

4 tasks

badai-nguyen added 2 commits June 24, 2024 12:07

feat(tier4_perception_launch): add image segmentation based pointclou…

952aedd

…d filter Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

chore: typo

18047c0

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

badai-nguyen force-pushed the fix/add_segmentation_based_pointcloud_filter_launch branch from 98ed17e to 18047c0 Compare June 24, 2024 03:07

badai-nguyen marked this pull request as ready for review June 24, 2024 03:17

badai-nguyen requested review from kminoda, ktro2828, miursh, tzhong518, YoshiRi and yukkysaito as code owners June 24, 2024 03:17

badai-nguyen marked this pull request as draft July 2, 2024 05:27

Merge branch 'main' into fix/add_segmentation_based_pointcloud_filter…

f03adeb

…_launch

github-actions bot removed the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label Jul 5, 2024

badai-nguyen removed request for ktro2828 and tzhong518 July 5, 2024 03:28

badai-nguyen marked this pull request as ready for review July 5, 2024 03:37

fix: detection launch

f7df7a3

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

badai-nguyen requested a review from technolojin July 5, 2024 07:03

technolojin approved these changes Jul 5, 2024

View reviewed changes

chore: add maintainer

5adfef6

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

YoshiRi approved these changes Jul 5, 2024

View reviewed changes

yukkysaito reviewed Jul 5, 2024

View reviewed changes

launch/tier4_perception_launch/package.xml Outdated Show resolved Hide resolved

Revert "chore: add maintainer"

3134879

This reverts commit 5adfef6.

yukkysaito approved these changes Jul 5, 2024

View reviewed changes

badai-nguyen merged commit b61db78 into autowarefoundation:main Jul 5, 2024
24 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(tier4_perception_launch): add image segmentation based pointcloud filter #7225

feat(tier4_perception_launch): add image segmentation based pointcloud filter #7225

badai-nguyen commented Jun 3, 2024 •

edited

Loading

codecov bot commented Jun 24, 2024 •

edited

Loading

github-actions bot commented Jul 5, 2024 •

edited

Loading

technolojin left a comment

YoshiRi left a comment

yukkysaito left a comment

feat(tier4_perception_launch): add image segmentation based pointcloud filter #7225

feat(tier4_perception_launch): add image segmentation based pointcloud filter #7225

Conversation

badai-nguyen commented Jun 3, 2024 • edited Loading

Description

Releated Link

Tests performed

Effects on system behavior

Interface changes

Pre-review checklist for the PR author

In-review checklist for the PR reviewers

Post-review checklist for the PR author

codecov bot commented Jun 24, 2024 • edited Loading

Codecov Report

github-actions bot commented Jul 5, 2024 • edited Loading

technolojin left a comment

Choose a reason for hiding this comment

YoshiRi left a comment

Choose a reason for hiding this comment

yukkysaito left a comment

Choose a reason for hiding this comment

badai-nguyen commented Jun 3, 2024 •

edited

Loading

codecov bot commented Jun 24, 2024 •

edited

Loading

github-actions bot commented Jul 5, 2024 •

edited

Loading