Stereo Vision

Computer stereo vision is the extraction of 3D information from digital images that can be used for object identification, tracking and manipulation tasks.

Modules

Stereo Rectification

Stereo Depth Perception - CUDA

ROS Stereo Vision

This is a ROS Package for Jetson CSI Stereo Camera for Computer Vision Tasks.

This package contains the nodes:

• src/stereo_camera_pub: Obtains left and right rectified images from IMX219-83 Stereo Camera.
• src/stereo_camera_sub: Displays the images using OpenCV.
• src/depth_perception_sub: Compute depth based on stereo images.

Hardware Stereo Module

Waveshare Documentation: IMX219-83 Stereo Camera.

Test with Jetson Nano

Hardware Connection

• Connect the camera to the CSI interfaces of Jetson Nano.
• Connect an HDMI LCD to Jetson Nano.
• Connect the I2C interface (only the SDA and SCL pins are required) of the Camera to I2C interface of the Jetson Nano Developer Kit (the Pin3, and Pin5).

Sofware Testing

• Open a Terminal.

• Check the video devices with command:

  ls /dev/video*

• Check if both video0 and video1 are detected

  • video0

  DISPLAY=:0.0 gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=3280, height=2464, format=(string)NV12, framerate=(fraction)20/1' ! nvoverlaysink -e

  • video1

  DISPLAY=:0.0 gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! 'video/x-raw(memory:NVMM), width=3280, height=2464, format=(string)NV12, framerate=(fraction)20/1' ! nvoverlaysink -e

• Test the ICM20948

  wget https://www.waveshare.com/w/upload/a/a4/D219-9dof.tar.gz
  tar zxvf D219-9dof.tar.gz]
  cd D219-9dof/03-double-camera-display
  mkdir build
  cd build
  cmake ..
  make
  ./double-camera-display

• If you find that the image captured is red. You can try to download .isp file and install it:

  wget https://www.waveshare.com/w/upload/e/eb/Camera_overrides.tar.gz
  tar zxvf Camera_overrides.tar.gz
  sudo cp camera_overrides.isp /var/nvidia/nvcam/settings/
  sudo chmod 664 /var/nvidia/nvcam/settings/camera_overrides.isp
  sudo chown root:root /var/nvidia/nvcam/settings/camera_overrides.isp

Installation

Requirements

• JetPack 4.6.1
• OpenCV (Installed with JetPack)
  • Optional
    1. Change the file

    sudo nano /opt/ros/melodic/share/cv_bridge/cmake/cv_bridgeConfig.cmake

    2. Change these lines:

    set(_include_dirs "include;/usr/include;/usr/include/opencv")

    to:

    set(_include_dirs "include;/usr/include;/usr/include/opencv4")

• ZSH (If you are using bash adapt your command in Step 4)

Install ROS

• Follow steps Ubuntu install of ROS Melodic.

  I recommend to install

  sudo apt install ros-melodic-desktop-full

• Initialize ROS Core.

  roscore

Integrate this repo into your ROS Workspace.

1. Clone this repo as a ROS Workspace.

cd ~/
git clone https://github.com/adolfos94/ROS-Stereo-Camera.git

2. Navigate to the new ROS Workspace or you can integrate the source files to another ROS Workspace.

cd ROS-Stereo-Camera

3. Build the package in the catking workspace.

$ adolfo in ROS-Stereo-Camera at jetson-nano
catkin_make

4. Add the workspace to your ROS environment.

$ adolfo in ROS-Stereo-Camera at jetson-nano
source devel/setup.zsh

5. Start the Publisher.

$ adolfo in ROS-Stereo-Camera at jetson-nano
rosrun stereo_camera_pub stereo_camera_pub_node

6. Start the Suscriber.

$ adolfo in ROS-Stereo-Camera at jetson-nano
rosrun stereo_camera_sub stereo_camera_sub_node