Table of Contents

• ATF Nav Test Config
  • Usage
    • Copy
    • Start Testcases
  • Editing
  • Compare Results
  • Requirements
  • Timing
  • Error Monitoring
    • interface.subscribers.type must be of type str
    • 'std::out_of_range' what(): map::at
    • History

ATF Nav Test Config

Config and environment files for atf_nav_tests to run the simulation using ATF

Usage

Copy

Run the copy.sh bash script. It will show a green progress bar:

'########################################   (100%)'

The listed files will be copied to the right directory for msh- and cob-config:

• xml
• stl
• env-configs
• navigation configs

Dependencies: ROS package has to be installed, otherwise rospack find won't work.

Start Testcases

The start_testcases.py starts the automated testframework and starts all possible testcases.

./start_testcases

The testcases are:

• line_passage
• line_passage_obstacle
• line_passage_person_moving
• line_passage_spawn_obstacle
• narrow_passage_2_cone
• t_passage
• t_passage_obstacle

The script provides some shell commands:

./start_testcases -c 2 -n 1 [-s 140]

Command Line Argument	Description
-h	help
-c	number of PCs on which the simulation is run --> splits the testcases equally
-n	number of the PC on which the simulation is run --> starts the corresponding set for this PC
-s	start value for config files (i.e. start test at 140 instead of 0 / default: 0)

Each testcase starts the corresponding atf_nav_pkg (Link) using the commandline to execute catkin_make atf_$ATF_PKG$.

Editing

Only edit the files in the atf_nav_test_config directory and copy them using copy.sh. This will ensure that there is only one file directory that needs editing, and not dozens of different files in each directory.

Pros	Cons
+ easy to setup new environments	- bash script has to be updated everytime a new package is created
+ much easier workflow	- CI not easily possible
+ direct github repo for all files	- call to copy.sh is needed everytime
+ files are needed in different directories are easily distributed

Compare Results

The compare_results.py script takes a path as input argument (you get asked for it... no hurry) and creates a Heatmap- and Errorbar-Plot for the results ATF has generated and where saved using the start_testcases.py script.

---

Requirements

To get everything running, you have to install the requirements for python using the requirements.txt in the scripts folder.

pip install -r /path/to/requirements.txt

The scripts are written using those requirements, and will maybe crash if you update any of those (e.g. matplotlib.. I tried it and everything crashed, even the IDE).

Timing

Best timing for 2 subscribed topics in application.py for goal_metrics:

    rospy.sleep(.2)
    [...]
    rospy.sleep(3)

Timing with rospy.sleep() is necessary because otherwise there will be a threading error. The bagfile includes the topics needed for goal_metrics.

topics:      /atf/testblock_nav/api          2 msgs    : atf_msgs/Api                     
             /atf/testblock_nav/trigger      2 msgs    : atf_msgs/TestblockTrigger        
             /base_pose_ground_truth      1143 msgs    : nav_msgs/Odometry                
             /move_base/goal                 1 msg     : move_base_msgs/MoveBaseActionGoal

detailed example is shown below.

class Application:
    def __init__(self):
        rp = RvizPublisher()
        filepath = '/home/flg-ma/git/catkin_ws/src/msh/msh_bringup/launch/t_passage.launch'
        rp.main(filepath, True, False, 2.0, 0.0, 0, 0, 0)
        rospy.sleep(.2)                     # improved speed with localisation
        self.atf = ATF()

    def execute(self):
        self.atf.start("testblock_nav")
        # necessary to catch goal published on topic /move_base/goal
        rospy.sleep(3)
        sss.move("base", [4.0, 0.0, 0.0])
        self.atf.stop("testblock_nav")
        self.atf.shutdown()

Error Monitoring

interface.subscribers.type must be of type str

The errorlog shown below occurs often and stops ATF from exiting normally. Therefore no bagfile is written and no output can be generated using the provided metrics.

except TypeError as te: self._check_types(ValueError("%s: '%s' when writing '%s'" % (type(te), str(te), str(locals().get('_x', self)))))
File "/opt/ros/indigo/lib/python2.7/dist-packages/genpy/message.py", line 331, in _check_types
check_type(n, t, getattr(self, n))
File "/opt/ros/indigo/lib/python2.7/dist-packages/genpy/message.py", line 253, in check_type
check_type(field_name+"[]", base_type, v)
File "/opt/ros/indigo/lib/python2.7/dist-packages/genpy/message.py", line 263, in check_type
check_type("%s.%s"%(field_name,n), t, getattr(field_val, n))
File "/opt/ros/indigo/lib/python2.7/dist-packages/genpy/message.py", line 263, in check_type
check_type("%s.%s"%(field_name,n), t, getattr(field_val, n))
File "/opt/ros/indigo/lib/python2.7/dist-packages/genpy/message.py", line 253, in check_type
check_type(field_name+"[]", base_type, v)
File "/opt/ros/indigo/lib/python2.7/dist-packages/genpy/message.py", line 263, in check_type
check_type("%s.%s"%(field_name,n), t, getattr(field_val, n))
File "/opt/ros/indigo/lib/python2.7/dist-packages/genpy/message.py", line 229, in check_type
raise SerializationError('field %s must be of type str'%field_name)
SerializationError: field nodes[].interface.subscribers[].type must be of type str

The Error can be preventend when the lines below in the file /home/flg-ma/git/catkin_ws/src/msh/msh_bringup/launch/application.xml are included! The robot_status_retriever.py is necessary to complete the writing of the output in a bagfile.

<!-- sensorring -->
<group>
    <machine name="$(arg s1)" address="$(arg s1)" env-loader="$(arg env-script)" default="true" timeout="30"/>
    <include ns="people_detection_sensorring" file="$(find cob_people_detection)/ros/launch/people_detection.launch">
        <arg name="camera_namespace" value="sensorring_cam3d_upright"/>
        <arg name="launch_head_detector" value="true"/>
        <arg name="launch_face_detector" value="true"/>
        <arg name="launch_face_recognizer" value="false"/>
        <arg name="launch_detection_tracker" value="false"/>
        <arg name="launch_face_capture" value="false"/>
        <arg name="launch_coordinator" value="false"/>
        <arg name="display_results_with_image_view" value="false"/>
    </include>
    <!--necessary, otherwise error occurs: SerializationError: field nodes[].interface.subscribers[].type must be of type str-->
    <node pkg="msh_bringup" type="robot_status_retriever.py" name="robot_status" output="screen"/>
    <include file="$(find msh_bringup)/launch/poi.launch">
        <arg name="robot" value="$(arg robot)"/>
    </include>
    <node pkg="zbar_ros" type="barcode_reader_node" name="barcode_reader" output="screen">
        <remap from="image" to="/sensorring_cam3d/rgb/image_raw"/>
    </node>
</group>

'std::out_of_range' what(): map::at

The errorlog shown below occurs when the robot can't find a path on the global costmap.

[ WARN] [1509019721.735995968, 7.195000000]: Path Segment ps 12 has partially blocked check the costmap. Illegal point idc: 15
[ INFO] [1509019721.736094996, 7.195000000]: The blocked parts were cleared
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
[move_base-85] process has died [pid 6139, exit code -6, cmd /opt/ros/indigo/lib/move_base/move_base cmd_vel:=/base/twist_mux/command_navigation odom:=/base/odometry_controller/odometry __name:=move_base __log:=/home/flg-ma/.ros/log/4ca1f794-ba46-11e7-ad95-782bcbad4e67/move_base-85.log].
log file: /home/flg-ma/.ros/log/4ca1f794-ba46-11e7-ad95-782bcbad4e67/move_base-85*.log

To fix this error, the path_segments.yaml has to be renewed, using roslaunch ipa_navigation_bringup config_gui.launch pkg_env_config:=rospack find msh_env_config robot_env:=$ROBOT_ENV$ where $ROBOT_ENV$ is the desired robot_env. I.e.:

• line_passage
• line_passage_obstacle
• line_passage_person_moving
• narrow_passage_2_cone
• t_passage
• t_passage_obstacle
• ... (if more environments were added later...)

Copy modified files:

The modified files path_segments.yaml or line_maps/laserline.yaml have to be copied in the atf_nav_test_env_config/envs/ directory, into the matching package.

History

V2.0

• added start_testcases.py
• now all necessary files are copied into the atf_nav_test folder

V1.0:

• first push