Update deprecated TensorRT 8.5 codes for ROS2 humble

[xmfcx]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I've agreed with the maintainers that I can plan this task.

Description

Currently in CI for humble & Ubuntu 22.04, we are using old (Ubuntu 20.04) packages for TensorRT.

In the latest (currently tensorrt-dev_8.5.1.7-1+cuda11.8) TensorRT version, some old APIs are deprecated.

Purpose

This issue is created to track upgrading the following 4 packages to compile and run Autoware Universe successfully.

Possible approaches

• Wait until we deprecate galactic
• Update the CI and ansible to install latest version of TensorRT packages
• Update traffic_light_classifier fix(tensorrt): update tensorrt code of traffic_light_classifier #2325
• Update traffic_light_ssd_fine_detector fix(tensorrt): update tensorrt code of traffic_light_ssd_fine_detector #2326
• Update tensorrt_yolo fix(tensorrt): update tensorrt code of tensorrt_yolo #2327
• Update lidar_centerpoint fix(tensorrt): update tensorrt code of lidar_centerpoint #2328
• Update tensorrt_yolox (no PR created)

Definition of done

• Make sure all TensorRT packages compile
• Make sure traffic_light_classifier works normally
• Make sure traffic_light_ssd_fine_detector works normally
• Make sure tensorrt_yolo works normally
• Make sure lidar_centerpoint works normally
• Make sure tensorrt_yolox works normally
• CI is updated is updated to install the latest TensorRT packages
• Ansible code is updated to install the latest TensorRT packages
• PRs are merged

[xmfcx]

The ARM64 network repository https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/sbsa/ still doesn't have tensorrt-dev available.

I think we should also wait until this is available too.

Because the new code is not backwards compatible.
(Update on 2022-12-21: I've updated them to be backwards compatible.)

We could also install old packages to ARM64 and new packages to x86_64 too but it would make things complicated.

What do you think, @wep21 @kenji-miyake ?

[kenji-miyake]

I think we should also wait until this is available too.

👍
Using different versions for arm64 and amd64 would be too complicated.

[yukke42]

Nice!
Doesn't lidar_apollo_instance_segmentation need to be updated?

[xmfcx]

@yukke42

Doesn't lidar_apollo_instance_segmentation need to be updated?

With all current PRs applied, everything compiled fine. I also checked lidar_apollo_instance_segmentation/lib/src/TrtNet.cpp and it doesn't look like it's using outdated APIs similar to ones I've changed.

Tomorrow I will try to compile it again just to be sure.

[xmfcx]

Update

• traffic_light_classifier fix was small, it is merged with fix(tensorrt): update tensorrt code of traffic_light_classifier #2325 to support both versions.
• I've updated the PRs to be compatible with both versions. But as I was updating the code, I got stuck in updating enqueueV2 to enqueueV3 and updated versions doesn't work.
• Temporarily I've created fix(tensor-rt): allow deprecated declarations for some tensorrt packages #2559 to temporarily have everything working.
• But I hope someone more experienced on TensorRT can help or take over this task.
• As I was working on this, feat: add tensorrt_yolox package #2370 tensorrt_yolox package was added.
  • It needs to be updated too. I've added it to the list.

Actions

• For now with the fix(tensor-rt): allow deprecated declarations for some tensorrt packages #2559 it works and I will take a break from this task.
• I am waiting for help on the existing open PRs.

cc @wep21 @kenji-miyake @yukkysaito

[xmfcx]

As for guidance in understanding the the enqueueV2 to enqueueV3 transition, following resources can be helpful:

• EnqueueV2 documentation
• EnqueueV3 documentation
• Runtime phase general helper doc: https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#runtime-phase
• https://github.com/NVIDIA/TensorRT/blob/main/samples/common/sampleInference.cpp the only place on the internet I could find where enqueueV3 is used.
• The sampleInference.cpp gets updated in this commit from v2 to v3 (search for sampleInference and expand) NVIDIA/TensorRT@1d6bf36#diff-aae446aa5c914afd563008477f89fafa3cb0f9b58fb4d23b8aeeafb2ba4c693b

Also from my experience on the task:

• I didn't know how to incorporate the buffers from the v2 so I removed them, I think they should be used somehow.
  • My diff on how I did it in lidar_centerpoint
  • Probably bool setTensorAddress(char const* tensorName, void* data) noexcept must be used for this but I couldn't figure how.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity.