build(lidar_apollo_segmentation_tvm and lidar_apollo_segmentation_tvm…

…_nodes): remove download from cmake (autowarefoundation#5431) * add include tier4_autoware_utils and dependency Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * remove downloading logic from Cmake, update documentation Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): remove downloading logic from Cmake, update documentation Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(tvm_utility): fix lint_cmake error Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): format warning message Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): add logic to work with autoware_data folder, add nn config header and test image Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * style(pre-commit): autofix * build(tvm_utility): refactor, update InferenceEngineTVM constructor Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(tvm_utility): add lightweight model and test with it Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): make building yolo_v2_tiny disable by default Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): remove test artifact for yolo_v2_tiny Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): update docs Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): update docs Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(tvm_utility): update namespace in abs_model test Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): rewrite yolo_v2_tiny as example Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): clean comments in yolo_v2_tiny example main.cpp Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): add launch file for yolo_v2_tiny example Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): update yolo_v2_tiny example readme Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(tvm_utility): add model for arm based systems, need to be tested on device Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * style(pre-commit): autofix * build(tvm_utility): update config header for arm Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(tvm_utility): remove debug output Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): add find_package conditional section Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): fix lint_cmake errors Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): remove coping model files during build Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): update readme with new data folder structure Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): fix spell check warnings Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(tvm_utility): add no model files guard to get_neural_network Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(tvm_utility): set back default paths in config headers Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(lidar_apollo_segmentation): wip update launch files Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): add param file, update launch file Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): add schema file, update node name Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(tvm_utility): fix json-schema-check Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): fix json-schema-check Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(tvm_utility): add parameter table to example readme Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(tvm_utility): fix typo-error in description of schema.json Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * buiild(tvm_utility): fix spell-check warning and typo Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * feat(spell-check): add dltype and tvmgen to local dict Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(lidar_apollo_segmentation_tvm): remove test Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(lidar_apollo_segmentation_tvm): add data_path to constructor Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(lidar_apollo_segmentation_tvm_nodes): add data_path to param file Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(lidar_apollo_segmentation_tvm_nodes): add allow_substs to launches Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(lidar_apollo_segmentation_tvm_nodes): update README Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(lidar_apollo_segmentation_tvm): update README Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(lidar_apollo_segmentation_tvm_nodes): fix schema typo Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(lidar_apollo_segmentation_tvm): remove unused import Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(lidar_apollo_segmentation_tvm): suppress cpplint Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(lidar_apollo_segmentation_tvm): suppress cpplint Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(lidar_apollo_segmentation_tvm): return test back and update it Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix * build(lidar_apollo_segmentation_tvm): fix cpplilnt errors Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * build(lidar_apollo_segmenntation_tvm): update checking for models files Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> * style(pre-commit): autofix --------- Signed-off-by: Alexey Panferov <lexavtanke@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
tier4 · Nov 22, 2023 · 3e8c7bf · 3e8c7bf
1 parent 3462f27
commit 3e8c7bf
Show file tree

Hide file tree

Showing 12 changed files with 117 additions and 20 deletions.
diff --git a/perception/lidar_apollo_segmentation_tvm/.gitignore b/perception/lidar_apollo_segmentation_tvm/.gitignore
diff --git a/perception/lidar_apollo_segmentation_tvm/README.md b/perception/lidar_apollo_segmentation_tvm/README.md
@@ -6,9 +6,10 @@
 
 #### Neural network
 
-This package will not build without a neural network for its inference.
-The network is provided by the cmake function exported by the tvm_utility package.
-See its design page for more information on how to enable downloading pre-compiled networks (by setting the `DOWNLOAD_ARTIFACTS` cmake variable), or how to handle user-compiled networks.
+This package will not run without a neural network for its inference.
+The network is provided by ansible script during the installation of Autoware or can be downloaded manually according to [Manual Downloading](https://github.com/autowarefoundation/autoware/tree/main/ansible/roles/artifacts).
+This package uses 'get_neural_network' function from tvm_utility package to create and provide proper dependency.
+See its design page for more information on how to handle user-compiled networks.
 
 #### Backend
 

diff --git a/...ption/lidar_apollo_segmentation_tvm/data/models/baidu_cnn/inference_engine_tvm_config.hpp b/...ption/lidar_apollo_segmentation_tvm/data/models/baidu_cnn/inference_engine_tvm_config.hpp
@@ -0,0 +1,55 @@
+// Copyright 2021 Arm Limited and Contributors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "tvm_utility/pipeline.hpp"
+
+#ifndef PERCEPTION__LIDAR_APOLLO_SEGMENTATION_TVM__DATA__MODELS__BAIDU_CNN__INFERENCE_ENGINE_TVM_CONFIG_HPP_  // NOLINT
+#define PERCEPTION__LIDAR_APOLLO_SEGMENTATION_TVM__DATA__MODELS__BAIDU_CNN__INFERENCE_ENGINE_TVM_CONFIG_HPP_  // NOLINT
+
+namespace model_zoo
+{
+namespace perception
+{
+namespace lidar_obstacle_detection
+{
+namespace baidu_cnn
+{
+namespace onnx_bcnn
+{
+
+static const tvm_utility::pipeline::InferenceEngineTVMConfig config{
+  {3, 0, 0},  // modelzoo_version
+
+  "baidu_cnn",  // network_name
+  "llvm",       // network_backend
+
+  "./deploy_lib.so",        // network_module_path
+  "./deploy_graph.json",    // network_graph_path
+  "./deploy_param.params",  // network_params_path
+
+  kDLCPU,  // tvm_device_type
+  0,       // tvm_device_id
+
+  {{"data", kDLFloat, 32, 1, {1, 4, 864, 864}}},  // network_inputs
+
+  {{"deconv0", kDLFloat, 32, 1, {1, 12, 864, 864}}}  // network_outputs
+};
+
+}  // namespace onnx_bcnn
+}  // namespace baidu_cnn
+}  // namespace lidar_obstacle_detection
+}  // namespace perception
+}  // namespace model_zoo
+// NOLINTNEXTLINE
+#endif  // PERCEPTION__LIDAR_APOLLO_SEGMENTATION_TVM__DATA__MODELS__BAIDU_CNN__INFERENCE_ENGINE_TVM_CONFIG_HPP_
diff --git a/..._segmentation_tvm/include/lidar_apollo_segmentation_tvm/lidar_apollo_segmentation_tvm.hpp b/..._segmentation_tvm/include/lidar_apollo_segmentation_tvm/lidar_apollo_segmentation_tvm.hpp
@@ -141,10 +141,11 @@ class LIDAR_APOLLO_SEGMENTATION_TVM_PUBLIC ApolloLidarSegmentation
   /// \param[in] height_thresh If it is non-negative, the points that are higher than the predicted
   ///                          object height by height_thresh are filtered out in the
   ///                          post-processing step.
+  /// \param[in] data_path The path to autoware data and artifacts folder
   explicit ApolloLidarSegmentation(
     int32_t range, float score_threshold, bool use_intensity_feature, bool use_constant_feature,
     float z_offset, float min_height, float max_height, float objectness_thresh,
-    int32_t min_pts_num, float height_thresh);
+    int32_t min_pts_num, float height_thresh, const std::string & data_path);
 
   /// \brief Detect obstacles.
   /// \param[in] input Input pointcloud.

diff --git a/perception/lidar_apollo_segmentation_tvm/src/lidar_apollo_segmentation_tvm.cpp b/perception/lidar_apollo_segmentation_tvm/src/lidar_apollo_segmentation_tvm.cpp
@@ -108,7 +108,7 @@ std::shared_ptr<DetectedObjectsWithFeature> ApolloLidarSegmentationPostProcessor
 ApolloLidarSegmentation::ApolloLidarSegmentation(
   int32_t range, float score_threshold, bool use_intensity_feature, bool use_constant_feature,
   float z_offset, float min_height, float max_height, float objectness_thresh, int32_t min_pts_num,
-  float height_thresh)
+  float height_thresh, const std::string & data_path)
 : range_(range),
   score_threshold_(score_threshold),
   z_offset_(z_offset),
@@ -118,7 +118,7 @@ ApolloLidarSegmentation::ApolloLidarSegmentation(
   pcl_pointcloud_ptr_(new pcl::PointCloud<pcl::PointXYZI>),
   PreP(std::make_shared<PrePT>(
     config, range, use_intensity_feature, use_constant_feature, min_height, max_height)),
-  IE(std::make_shared<IET>(config, "lidar_apollo_segmentation_tvm")),
+  IE(std::make_shared<IET>(config, "lidar_apollo_segmentation_tvm", data_path)),
   PostP(std::make_shared<PostPT>(
     config, pcl_pointcloud_ptr_, range, objectness_thresh, score_threshold, height_thresh,
     min_pts_num)),

diff --git a/perception/lidar_apollo_segmentation_tvm/test/main.cpp b/perception/lidar_apollo_segmentation_tvm/test/main.cpp
@@ -17,14 +17,18 @@
 
 #include <gtest/gtest.h>
 
+#include <filesystem>
 #include <memory>
 #include <random>
 #include <string>
 #include <vector>
 
 using autoware::perception::lidar_apollo_segmentation_tvm::ApolloLidarSegmentation;
+namespace fs = std::filesystem;
 
-void test_segmentation(bool use_intensity_feature, bool use_constant_feature, bool expect_throw)
+void test_segmentation(
+  const std::string & data_path, bool use_intensity_feature, bool use_constant_feature,
+  bool expect_throw)
 {
   // Instantiate the pipeline
   const int width = 1;
@@ -37,9 +41,10 @@ void test_segmentation(bool use_intensity_feature, bool use_constant_feature, bo
   const float objectness_thresh = 0.5f;
   const int32_t min_pts_num = 3;
   const float height_thresh = 0.5f;
+
   ApolloLidarSegmentation segmentation(
     range, score_threshold, use_intensity_feature, use_constant_feature, z_offset, min_height,
-    max_height, objectness_thresh, min_pts_num, height_thresh);
+    max_height, objectness_thresh, min_pts_num, height_thresh, data_path);
 
   auto version_status = segmentation.version_check();
   EXPECT_NE(version_status, tvm_utility::Version::Unsupported);
@@ -85,8 +90,31 @@ void test_segmentation(bool use_intensity_feature, bool use_constant_feature, bo
 // Other test configurations to increase code coverage.
 TEST(lidar_apollo_segmentation_tvm, others)
 {
-  test_segmentation(false, true, false);
-  test_segmentation(true, true, false);
-  test_segmentation(false, false, false);
-  test_segmentation(true, false, false);
+  std::string home = std::getenv("HOME");
+  fs::path data_path(home);
+  data_path /= "autoware_data";
+  fs::path apollo_data_path(data_path);
+  apollo_data_path /= "lidar_apollo_segmentation_tvm";
+  fs::path deploy_path(apollo_data_path);
+  deploy_path /= "models/baidu_cnn";
+
+  fs::path deploy_graph("deploy_graph.json");
+  fs::path deploy_lib("deploy_lib.so");
+  fs::path deploy_param("deploy_param.params");
+
+  fs::path deploy_graph_path = deploy_path / deploy_graph;
+  fs::path deploy_lib_path = deploy_path / deploy_lib;
+  fs::path deploy_param_path = deploy_path / deploy_param;
+
+  if (
+    !fs::exists(deploy_graph_path) || !fs::exists(deploy_lib_path) ||
+    !fs::exists(deploy_param_path)) {
+    printf("Model deploy files not found. Skip test.\n");
+    GTEST_SKIP();
+    return;
+  }
+  test_segmentation(data_path, false, true, false);
+  test_segmentation(data_path, true, true, false);
+  test_segmentation(data_path, false, false, false);
+  test_segmentation(data_path, true, false, false);
 }
diff --git a/perception/lidar_apollo_segmentation_tvm_nodes/README.md b/perception/lidar_apollo_segmentation_tvm_nodes/README.md
@@ -28,7 +28,7 @@ See the design of the algorithm in the core (lidar_apollo_segmentation_tvm) pack
 
 ### Usage
 
-`lidar_apollo_segmentation_tvm` and `lidar_apollo_segmentation_tvm_nodes` will not build without a neural network.
+`lidar_apollo_segmentation_tvm` and `lidar_apollo_segmentation_tvm_nodes` will not work without a neural network.
 See the lidar_apollo_segmentation_tvm usage for more information.
 
 ### Assumptions / Known limits

diff --git a/...lidar_apollo_segmentation_tvm_nodes/config/lidar_apollo_segmentation_tvm_nodes.param.yaml b/...lidar_apollo_segmentation_tvm_nodes/config/lidar_apollo_segmentation_tvm_nodes.param.yaml
@@ -24,3 +24,4 @@
     objectness_thresh: 0.5
     min_pts_num: 3
     height_thresh: 0.5
+    data_path: $(env HOME)/autoware_data
diff --git a/.../lidar_apollo_segmentation_tvm_nodes/launch/lidar_apollo_segmentation_tvm_nodes.launch.py b/.../lidar_apollo_segmentation_tvm_nodes/launch/lidar_apollo_segmentation_tvm_nodes.launch.py
@@ -21,20 +21,25 @@
 from launch.actions import DeclareLaunchArgument
 from launch.substitutions import LaunchConfiguration
 from launch_ros.actions import Node
-import yaml
+import launch_ros.parameter_descriptions
 
 
 def generate_launch_description():
     param_file = os.path.join(
         get_package_share_directory("lidar_apollo_segmentation_tvm_nodes"),
         "config/lidar_apollo_segmentation_tvm_nodes.param.yaml",
     )
-    with open(param_file, "r") as f:
-        lidar_apollo_segmentation_tvm_node_params = yaml.safe_load(f)["/**"]["ros__parameters"]
+
+    lidar_apollo_segmentation_tvm_node_params = launch_ros.parameter_descriptions.ParameterFile(
+        param_file=param_file, allow_substs=True
+    )
 
     arguments = [
         DeclareLaunchArgument("input/pointcloud", default_value="/sensing/lidar/pointcloud"),
         DeclareLaunchArgument("output/objects", default_value="labeled_clusters"),
+        DeclareLaunchArgument(
+            "data_path", default_value=os.path.join(os.environ["HOME"], "autoware_data")
+        ),
     ]
 
     # lidar segmentation node execution definition.

diff --git a/...lidar_apollo_segmentation_tvm_nodes/launch/lidar_apollo_segmentation_tvm_nodes.launch.xml b/...lidar_apollo_segmentation_tvm_nodes/launch/lidar_apollo_segmentation_tvm_nodes.launch.xml
@@ -23,6 +23,7 @@
     <remap from="points_in" to="$(var input/pointcloud)"/>
     <remap from="objects_out" to="$(var output/objects)"/>
     <param name="z_offset" value="$(var z_offset)"/>
-    <param from="$(var param_file)"/>
+    <!-- cspell: ignore substs -->
+    <param from="$(var param_file)" allow_substs="true"/>
   </node>
 </launch>
diff --git a/...idar_apollo_segmentation_tvm_nodes/schema/lidar_apollo_segmentation_tvm_nodes.schema.json b/...idar_apollo_segmentation_tvm_nodes/schema/lidar_apollo_segmentation_tvm_nodes.schema.json
@@ -61,6 +61,11 @@
           "type": "number",
           "default": 0.5,
           "description": "If it is non-negative, the points that are higher than the predicted object height by height_thresh are filtered out in the post-processing step."
+        },
+        "data_path": {
+          "type": "string",
+          "default": "$(env HOME)/autoware_data",
+          "description": "Packages data and artifacts directory path."
         }
       },
       "required": [
@@ -73,7 +78,8 @@
         "max_height",
         "objectness_thresh",
         "min_pts_num",
-        "height_thresh"
+        "height_thresh",
+        "data_path"
       ]
     }
   },

diff --git a/perception/lidar_apollo_segmentation_tvm_nodes/src/lidar_apollo_segmentation_tvm_node.cpp b/perception/lidar_apollo_segmentation_tvm_nodes/src/lidar_apollo_segmentation_tvm_node.cpp
@@ -41,7 +41,7 @@ ApolloLidarSegmentationNode::ApolloLidarSegmentationNode(const rclcpp::NodeOptio
     declare_parameter<bool>("use_constant_feature"), declare_parameter<double>("z_offset"),
     declare_parameter<double>("min_height"), declare_parameter<double>("max_height"),
     declare_parameter<double>("objectness_thresh"), declare_parameter<int64_t>("min_pts_num"),
-    declare_parameter<double>("height_thresh"))}
+    declare_parameter<double>("height_thresh"), declare_parameter<std::string>("data_path"))}
 {
   // Log unexpected versions of the neural network.
   auto version_status = m_detector_ptr->version_check();