Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(multi_object_tracker): reduce publish delay #6710

Merged

Conversation

technolojin
Copy link
Contributor

@technolojin technolojin commented Mar 29, 2024

Description

This PR suggests a new method to reduce latency about 100 ms.

Background:

Current multi_object_tracker publishes the tracked object triggered by a timer. This timer ensures the topic is published in target frequency. The delay is compensated by predict the tracked objects motion.
However, the input (merged objects from multiple detectors) frequency is unstable (see the following plots, name of input_latency_ms). The instability of the detection latency is absorbed by the multi_object_tracker and adds latency about one cycle (100 ms of TIER IV configuration).

The new method:

there is two triggers to publish the tracked object

  1. on measurement : When the publishing frequency is in the target window, publish right after the process is done. The publishing object will be compensated the delay from the measurement to the publishing time.
  2. on timer : When the input is late, publish tracked object with delay compensation.

Side effect:

The publishing frequency will not be kept precisely. (see the following plots, name of cycle_time_ms )

Related links

TIER IV INTERNAL

Tests performed

Current
MOT_timing_now

Improved
MOT_timing_proposal

Notes for reviewers

This PR will be merged after
PR #6706
PR #6757
PR #6775

This PR is an alternative work of PR #6687

TODO: improve input (merged object) stability

Interface changes

N/A

Effects on system behavior

It may reduces overall latency about one cycle of the perception (100 ms).

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

  • The PR follows the pull request guidelines.
  • The PR has been properly tested.
  • The PR has been reviewed by the code owners.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.
  • The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

@github-actions github-actions bot added the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label Mar 29, 2024
@technolojin technolojin force-pushed the feat/mot-reduce-publish-delay branch 2 times, most recently from 4091973 to 705eb8c Compare April 2, 2024 08:40
@technolojin technolojin marked this pull request as ready for review April 2, 2024 08:50
@technolojin technolojin added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Apr 3, 2024
@technolojin technolojin self-assigned this Apr 3, 2024
Copy link

codecov bot commented Apr 4, 2024

Codecov Report

Attention: Patch coverage is 0% with 19 lines in your changes are missing coverage. Please review.

Project coverage is 14.93%. Comparing base (35642a6) to head (76c47f7).
Report is 8 commits behind head on main.

❗ Current head 76c47f7 differs from pull request most recent head f9b483d. Consider uploading reports for the commit f9b483d to get more accurate results

Files Patch % Lines
...i_object_tracker/src/multi_object_tracker_core.cpp 0.00% 19 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6710      +/-   ##
==========================================
- Coverage   14.93%   14.93%   -0.01%     
==========================================
  Files        1944     1944              
  Lines      134061   134051      -10     
  Branches    39891    39879      -12     
==========================================
- Hits        20027    20025       -2     
+ Misses      91735    91733       -2     
+ Partials    22299    22293       -6     
Flag Coverage Δ *Carryforward flag
differential 0.00% <0.00%> (?)
total 14.94% <ø> (+<0.01%) ⬆️ Carriedforward from 8046005

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@YoshiRi
Copy link
Contributor

YoshiRi commented Apr 4, 2024

Now I get your idea.
The key value in your logic is constexpr double maximum_latency_ratio = 1.11; // 11% margin.

This value represents how large this "time keeper" can endure input delay variance.

If this value is small as 1.11, the time keeper is more strict so that it can cause large delay when larger input latency variance comes.
image

If I choose larger value like 1.5, the time keeper would not bother process and then processing time can be stick to smaller value. Apparently, it has drawback that the worst delay becomes larger since time keeper is loose, so this value should decided by maximum acceptable publish interval. I think.
image

Anyway, I prefer larger value because current tracking already has some variance in the publishing interval.

@technolojin
Copy link
Contributor Author

technolojin commented Apr 5, 2024

@YoshiRi
Your understand is right. If we increase the value maximum_latency_ratio, probability of prediction-without-update will reduced.
However, I do not know users can accept the long silent of 150 ms.

One another key tuning point is
const double minimum_publish_interval = publisher_period_ * 0.7; // 70% of the period
If we allow to publish topics in a short time (lower the minimum_publish_interval), The publisher triggered by message will keep up the latest object.
I adjusted the value to 0.4 and the test result is as following.

Screenshot from 2024-04-05 13-51-33

The cyclic time is occasionally drops near to 40 ms, but the publish interval is kept within 116 ms (111 ms of configured maximum + onTimer maximum latency 5 ms) in my environment.

// check the publish period
const auto elapsed_time = (current_time - last_published_time_).seconds();
// if the elapsed time is over the period, publish objects with prediction
constexpr double maximum_latency_ratio = 1.11; // 11% margin
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[imo]
Since early publish is better for the planning module, I prefer larger value like 1.4 or more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the acceptable publish interval is set to 70% to 111%, as agreed with @yukkysaito

@technolojin technolojin force-pushed the feat/mot-reduce-publish-delay branch 2 times, most recently from 5187953 to 079f34d Compare April 9, 2024 05:04
@technolojin
Copy link
Contributor Author

This PR will be reviewed and merged after a bugfix of the MOT #6775 is merged.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: reduce lower limit of publish interval

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: enlarge publish range upper limit

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: set parameter tested and agreed

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
@technolojin technolojin force-pushed the feat/mot-reduce-publish-delay branch from 03c087e to f9b483d Compare April 9, 2024 06:32
@technolojin technolojin requested a review from YoshiRi April 9, 2024 06:45
Copy link
Contributor

@YoshiRi YoshiRi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Worked within my laptop
Uploading image.png…

@technolojin technolojin merged commit f5c30eb into autowarefoundation:main Apr 9, 2024
22 of 24 checks passed
technolojin added a commit to technolojin/autoware.universe that referenced this pull request Apr 9, 2024
…6710)

* feat: implement a new publish timer

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* chore: add comments for new variables

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: variable name was wrong

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: reduce lower limit of publish interval

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: enlarge publish range upper limit

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: set parameter tested and agreed

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

---------

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
esteve pushed a commit that referenced this pull request Apr 9, 2024
* feat: implement a new publish timer

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* chore: add comments for new variables

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: variable name was wrong

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: reduce lower limit of publish interval

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: enlarge publish range upper limit

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: set parameter tested and agreed

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

---------

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
karishma1911 pushed a commit to Interplai/autoware.universe that referenced this pull request Jun 3, 2024
…6710)

* feat: implement a new publish timer

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* chore: add comments for new variables

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: variable name was wrong

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: reduce lower limit of publish interval

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: enlarge publish range upper limit

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

fix: set parameter tested and agreed

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

---------

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:perception Advanced sensor data processing and environment understanding. (auto-assigned) run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Predicted obstacle box is too far from the actual points, which impacts obstacle stop planner
2 participants