feat(image_projection_based_fusion): unrectify image projection based fusion #7053

yukkysaito · 2024-05-17T16:49:31Z

Description

In image_projection_based_fusion, only the internal parameters of the camera were considered, and no distortion correction was applied. As a result, when projecting onto images with significant distortion, misalignments occurred. This Pull Request has been modified to account for distortion.

Tests performed

In roi_cluster_fusion, operation has been confirmed on the actual machine.
before

after

Notes for reviewers

Interface changes

Effects on system behavior

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.
The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

The PR follows the pull request guidelines.
The PR has been properly tested.
The PR has been reviewed by the code owners.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

There are no open discussions or they are tracked via tickets.
The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com>

…ojection-based-fusion

Signed-off-by: Yukihito Saito <yukky.saito@gmail.com>

…ction-based-fusion' into feature/unrecify-image-projection-based-fusion

Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com>

Signed-off-by: Yukihito Saito <yukky.saito@gmail.com>

perception/image_projection_based_fusion/src/utils/utils.cpp

Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com>

github-actions · 2024-06-27T04:16:24Z

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

You've checked our contribution guidelines.
Your PR follows our pull request guidelines.
All required CI checks pass before marking the PR ready for review.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

technolojin · 2024-06-27T04:20:23Z

@yukkysaito I checked there is a compile failure. It was caused by #7224, and it was fixed.

codecov · 2024-06-27T04:25:31Z

Codecov Report

Attention: Patch coverage is 0% with 35 lines in your changes missing coverage. Please review.

Project coverage is 2.64%. Comparing base (507e3f4) to head (1b0e197).
Report is 157 commits behind head on main.

Files	Patch %	Lines
...ction_based_fusion/src/roi_cluster_fusion/node.cpp	0.00%	13 Missing ⚠️
...on_based_fusion/src/roi_pointcloud_fusion/node.cpp	0.00%	6 Missing ⚠️
...fusion/src/segmentation_pointcloud_fusion/node.cpp	0.00%	6 Missing ⚠️
...sed_fusion/src/roi_detected_object_fusion/node.cpp	0.00%	4 Missing ⚠️
.../image_projection_based_fusion/src/utils/utils.cpp	0.00%	4 Missing ⚠️
...ion_based_fusion/src/pointpainting_fusion/node.cpp	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #7053       +/-   ##
==========================================
- Coverage   14.84%   2.64%   -12.21%     
==========================================
  Files        1999     165     -1834     
  Lines      139163   10105   -129058     
  Branches    43716    1589    -42127     
==========================================
- Hits        20661     267    -20394     
+ Misses      95731    9748    -85983     
+ Partials    22771      90    -22681

Flag	Coverage Δ
differential	`2.64% <0.00%> (?)`
total	`?`

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

technolojin · 2024-06-27T04:25:51Z

@kminoda DCO warning seems to be shown by your Co-authored-by. Can you resolve it?

yukkysaito · 2024-06-27T04:35:29Z

@kminoda DCO warning seems to be shown by your Co-authored-by. Can you resolve it?

done

… fusion (autowarefoundation#7053) * unrecify 3d point for image projection based fusion Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com> * style(pre-commit): autofix * add try-catch code for exception handling Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * Update perception/image_projection_based_fusion/src/utils/utils.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * style(pre-commit): autofix * remove litter Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * revert try-catch Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * Update perception/image_projection_based_fusion/src/utils/utils.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * fix: resolve conflict Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp> --------- Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com> Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> Co-authored-by: Taekjin LEE <taekjin.lee@tier4.jp>

… fusion (autowarefoundation#7053) * unrecify 3d point for image projection based fusion Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com> * style(pre-commit): autofix * add try-catch code for exception handling Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * Update perception/image_projection_based_fusion/src/utils/utils.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * style(pre-commit): autofix * remove litter Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * revert try-catch Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * Update perception/image_projection_based_fusion/src/utils/utils.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * fix: resolve conflict Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp> --------- Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com> Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> Co-authored-by: Taekjin LEE <taekjin.lee@tier4.jp> Signed-off-by: palas21 <palas21@itu.edu.tr>

… fusion (autowarefoundation#7053) * unrecify 3d point for image projection based fusion Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com> * style(pre-commit): autofix * add try-catch code for exception handling Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * Update perception/image_projection_based_fusion/src/utils/utils.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * style(pre-commit): autofix * remove litter Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * revert try-catch Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * Update perception/image_projection_based_fusion/src/utils/utils.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * fix: resolve conflict Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp> --------- Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com> Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> Co-authored-by: Taekjin LEE <taekjin.lee@tier4.jp>

… fusion (#7053) * unrecify 3d point for image projection based fusion Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com> * style(pre-commit): autofix * add try-catch code for exception handling Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * Update perception/image_projection_based_fusion/src/utils/utils.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * style(pre-commit): autofix * remove litter Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * revert try-catch Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * Update perception/image_projection_based_fusion/src/utils/utils.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * fix: resolve conflict Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp> --------- Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com> Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> Co-authored-by: Taekjin LEE <taekjin.lee@tier4.jp>

… fusion (autowarefoundation#7053) * unrecify 3d point for image projection based fusion Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com> * style(pre-commit): autofix * add try-catch code for exception handling Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * Update perception/image_projection_based_fusion/src/utils/utils.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * style(pre-commit): autofix * remove litter Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * revert try-catch Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> * Update perception/image_projection_based_fusion/src/utils/utils.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * fix: resolve conflict Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp> --------- Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com> Signed-off-by: Yukihito Saito <yukky.saito@gmail.com> Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> Co-authored-by: Taekjin LEE <taekjin.lee@tier4.jp>

yukkysaito and others added 9 commits April 26, 2024 22:51

unrecify 3d point for image projection based fusion

db8f01b

Signed-off-by: Yukihiro Saito <yukky.saito@gmail.com>

style(pre-commit): autofix

674f34e

Merge branch 'autowarefoundation:main' into feature/unrecify-image-pr…

bd4e54d

…ojection-based-fusion

add try-catch code for exception handling

b58b890

Signed-off-by: Yukihito Saito <yukky.saito@gmail.com>

Merge remote-tracking branch 'yukkysaito/feature/unrecify-image-proje…

1feb030

…ction-based-fusion' into feature/unrecify-image-projection-based-fusion

Update perception/image_projection_based_fusion/src/utils/utils.cpp

a86ed7b

Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com>

style(pre-commit): autofix

af0c7b5

remove litter

2eda35c

Signed-off-by: Yukihito Saito <yukky.saito@gmail.com>

revert try-catch

5f8f0c3

Signed-off-by: Yukihito Saito <yukky.saito@gmail.com>

github-actions bot added the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label May 17, 2024

yukkysaito marked this pull request as ready for review June 1, 2024 08:30

yukkysaito requested review from badai-nguyen, kminoda, ktro2828, miursh, tzhong518 and YoshiRi as code owners June 1, 2024 08:30

kminoda added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Jun 3, 2024

kminoda reviewed Jun 3, 2024

View reviewed changes

perception/image_projection_based_fusion/src/utils/utils.cpp Outdated Show resolved Hide resolved

Update perception/image_projection_based_fusion/src/utils/utils.cpp

1a58f52

Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com>

kminoda approved these changes Jun 3, 2024

View reviewed changes

yukkysaito enabled auto-merge (squash) June 3, 2024 09:06

Merge branch 'main' into feature/unrecify-image-projection-based-fusion

bcc7a26

yukkysaito disabled auto-merge June 4, 2024 15:13

yukkysaito and others added 2 commits June 6, 2024 16:49

Merge branch 'main' into feature/unrecify-image-projection-based-fusion

f275c24

Merge branch 'main' into feature/unrecify-image-projection-based-fusion

c87053b

fix: resolve conflict

1b0e197

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

technolojin force-pushed the feature/unrecify-image-projection-based-fusion branch from 62fde55 to 1b0e197 Compare June 27, 2024 04:17

technolojin requested a review from kminoda June 27, 2024 04:26

yukkysaito merged commit bcf0677 into autowarefoundation:main Jun 27, 2024
31 of 33 checks passed

technolojin mentioned this pull request Jun 28, 2024

fix(image_projection_based_fusion): handle projection errors in image fusion nodes #7747

Merged

technolojin mentioned this pull request Oct 24, 2024

fix(autoware_image_projection_based_fusion): pointpainting bug fix for point projection #9150

Merged

technolojin mentioned this pull request Nov 5, 2024

fix(autoware_image_projection_based_fusion): make optional to consider lens distortion in the point projection #9233

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(image_projection_based_fusion): unrectify image projection based fusion #7053

feat(image_projection_based_fusion): unrectify image projection based fusion #7053

yukkysaito commented May 17, 2024 •

edited by technolojin

Loading

github-actions bot commented Jun 27, 2024 •

edited

Loading

technolojin commented Jun 27, 2024

codecov bot commented Jun 27, 2024 •

edited

Loading

technolojin commented Jun 27, 2024

yukkysaito commented Jun 27, 2024

feat(image_projection_based_fusion): unrectify image projection based fusion #7053

feat(image_projection_based_fusion): unrectify image projection based fusion #7053

Conversation

yukkysaito commented May 17, 2024 • edited by technolojin Loading

Description

Related links

Tests performed

Notes for reviewers

Interface changes

Effects on system behavior

Pre-review checklist for the PR author

In-review checklist for the PR reviewers

Post-review checklist for the PR author

github-actions bot commented Jun 27, 2024 • edited Loading

technolojin commented Jun 27, 2024

codecov bot commented Jun 27, 2024 • edited Loading

Codecov Report

technolojin commented Jun 27, 2024

yukkysaito commented Jun 27, 2024

yukkysaito commented May 17, 2024 •

edited by technolojin

Loading

github-actions bot commented Jun 27, 2024 •

edited

Loading

codecov bot commented Jun 27, 2024 •

edited

Loading