multi-camera rendering depth and color images

[dohyeonYoon]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	L515
Firmware Version	(01.05.08.01)
Operating System & Version	Window 10
Kernel Version (Linux Only)
Platform	PC
SDK Version	{ v2.47.0 }
Language	python
Segment	others

Issue Description

<Describe your issue / question / feature request / etc..>

Previously, "camera rendering depth and color images using the help of OpenCV and Numpy" was performed with only one L515 depth camera.

I am going to do camera rendering depth and color images with a total of 3 L515 cameras.

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/opencv_viewer_example.py

I proceeded with camera rendering referring to the code above. To perform rendering with 3 cameras at the same time, how to modify the code compared with the code above?

thank you :)

[MartyG-RealSense]

Hi @dohyeonYoon A tutorial in the link below describes how to create a multi-camera version of a pyrealsense2 stream viewer program like opencv_viewer_example. Scroll down the article to the section titled Now lets write a test code for Multiple Camera Setup to find the scripting.

https://rahulvishwakarma.wordpress.com/2019/08/17/realsense-435i-depth-rgb-multi-camera-setup-and-opencv-python-wrapper-intel-realsense-sdk-2-0-compiled-from-source-on-win10/

The tutorial is focused around Windows 10 but the code should be usable in any pyrealsense2 installation.

---

A RealSense user also recently shared their own Python multiple-camera stream viewer project.

https://github.com/ivomarvan/samples_and_experiments/tree/master/Multiple_realsense_cameras

[dohyeonYoon]

Thanks for your help!
but, the following error occurred while running the (Test 2: Stream images from all the devices and show) code.
I wonder what the reason is.

ps: I already add the realsense_device_manager.py at the same directory :(

Exception has occurred: KeyError
<stream.color: 2>
File "C:\Users\user\Desktop\code.py", line 21, in visualise_measurements
color_image = np.asarray(frame[rs.stream.color].get_data())
File "C:\Users\user\Desktop\code.py", line 51, in
visualise_measurements(frames)

[MartyG-RealSense]

Taking the realsense_device_manager.py code out of box_dimensioner_multicam and integrating it into your own project can be achieved but it can be difficult and can results in errors at first. Would it be possible to post your script in a comment below, please?

[dohyeonYoon]

If I run the following code,
The first camera frame and the second camera frame are displayed alternately in one window
how can I fix this matter? ( I want to display each frame seperately.)

import pyrealsense2 as rs
import numpy as np
import cv2

from realsense_device_manager import DeviceManager

def visualise_measurements(frames_devices):
  
    for (device, frame) in frames_devices.items():
        color_image = np.asarray(frame[rs.stream.color].get_data())
        text_str = "device"
        cv2.putText(color_image, text_str, (50,50), cv2.FONT_HERSHEY_PLAIN, 2, (0,255,0) )
        # Visualise the results
        text_str = 'Color image from RealSense Device Nr: ' + "device"
        cv2.namedWindow(text_str)
        cv2.imshow(text_str, color_image)
        cv2.waitKey(1)


# Define some constants 
resolution_width = 1024 # pixels
resolution_height = 768 # pixels
frame_rate = 15  # fps
dispose_frames_for_stablisation = 30  # frames

try:
    # Enable the streams from all the intel realsense devices
    rs_config = rs.config()
    rs_config.enable_stream(rs.stream.depth, resolution_width, resolution_height, rs.format.z16, frame_rate)
    rs_config.enable_stream(rs.stream.infrared, 1, resolution_width, resolution_height, rs.format.y8, frame_rate)
    rs_config.enable_stream(rs.stream.color, resolution_width, resolution_height, rs.format.bgr8, frame_rate)

    # Use the device manager class to enable the devices and get the frames
    device_manager = DeviceManager(rs.context(), rs_config)
    device_manager.enable_all_devices()
    
    # Allow some frames for the auto-exposure controller to stablise
    while 1:
        frames = device_manager.poll_frames()
        visualise_measurements(frames)

except KeyboardInterrupt:
    print("The program was interupted by the user. Closing the program...")

finally:
    device_manager.disable_streams()
    cv2.destroyAllWindows()

• I want to display depth_color frame not only RGB_frame how can I display the depth_color frame too?

Thanks for your help while you're busy

[MartyG-RealSense]

It's no trouble at all. :)

The first difference between your script and the Rahul Vishwakarma article is that in the text_str line you place 'device' in quotation marks:

Whilst the article does not use quotation marks around 'device':

A more significant difference is in the conversion of the config streams for L515 resolutions. You have set depth, infrared and color to all use the same resolution and FPS. However, 1024x768 is not supported on L515 for the RGB stream. I recommend 1280x720 instead.

Also, the only FPS supported for depth and infrared on L515 is 30, so the depth and infrared FPS should not be set to 15. I would recommend using 30 for depth, infrared and color.

The rest of the script looks fine compared to the Rahul Vishwakarma original.

[dohyeonYoon]

As you said, if I don't use quotes around 'device', I get the following error message

Also after change the fps to 30 and resolution_width, resolution_height to 1280 x 720,
I still have a problem with 3 camera frames appearing take turns in one window. :(

how can I display the each 3 camera frame in each window? like below image

[MartyG-RealSense]

May I confirm please that you have set depth and infrared to 1024x768 and color to 1280x720, please? 1280x720 is not supported for depth and infrared on L515.

Your code could look like this, defining separate variables for storing the color resolution:

# Define some constants 
resolution_width = 1024 # pixels
resolution_height = 768 # pixels
resolution2_width = 1280 # pixels
resolution2_height = 720 # pixels
frame_rate = 30  # fps
dispose_frames_for_stablisation = 30  # frames

try:
    # Enable the streams from all the intel realsense devices
    rs_config = rs.config()
    rs_config.enable_stream(rs.stream.depth, resolution_width, resolution_height, rs.format.z16, frame_rate)
    rs_config.enable_stream(rs.stream.infrared, 1, resolution_width, resolution_height, rs.format.y8, frame_rate)
    rs_config.enable_stream(rs.stream.color, resolution2_width, resolution2_height, rs.format.bgr8, frame_rate)

[dohyeonYoon]

import pyrealsense2 as rs
import numpy as np
import cv2

from realsense_device_manager import DeviceManager

def visualise_measurements(frames_devices):
  
    for (device, frame) in frames_devices.items():
        color_image = np.asarray(frame[rs.stream.color].get_data())

        # Visualise the results
        text_str = 'frame'
        cv2.namedWindow(text_str)
        cv2.imshow(text_str, color_image)
        cv2.waitKey(1)


# Define some constants 
resolution_width = 1024 # pixels
resolution_height = 768 # pixels
resolution2_width = 1280 # pixels
resolution2_height = 720 # pixels
frame_rate = 30  # fps
dispose_frames_for_stablisation = 30  # frames

try:
    # Enable the streams from all the intel realsense devices
    rs_config = rs.config()
    rs_config.enable_stream(rs.stream.depth, resolution_width, resolution_height, rs.format.z16, frame_rate)
    rs_config.enable_stream(rs.stream.infrared, 1, resolution_width, resolution_height, rs.format.y8, frame_rate)
    rs_config.enable_stream(rs.stream.color, resolution2_width, resolution2_height, rs.format.bgr8, frame_rate)

    # Use the device manager class to enable the devices and get the frames
    device_manager = DeviceManager(rs.context(), rs_config)
    device_manager.enable_all_devices()
    
    # Allow some frames for the auto-exposure controller to stablise
    while 1:
        frames = device_manager.poll_frames()
        visualise_measurements(frames)

except KeyboardInterrupt:
    print("The program was interupted by the user. Closing the program...")

finally:
    device_manager.disable_streams()
    cv2.destroyAllWindows()

this is my total code I still suffering same problem that 3 camera frames appearing take turns in one window.

so the frame displayed by >>>>> visualise_measurements(frames)<<<< code

def visualise_measurements(frames_devices):
  
    for (device, frame) in frames_devices.items():
        color_image = np.asarray(frame[rs.stream.color].get_data())

        # Visualise the results
        text_str = 'frame'
        cv2.namedWindow(text_str)
        cv2.imshow(text_str, color_image)
        cv2.waitKey(1)

how about add cv2.namedwindow and cv2.imshow one by one?

[MartyG-RealSense]

Does the error go away if you use device_manager.poll_for_frames() instead of poll_frames()

[dohyeonYoon]

I get the following error :(

[MartyG-RealSense]

Okay, thanks for trying. Going back to the original poll_frames version, then. I guess that the error occurs when inserting a 'for' because it is looking for the code block called 'poll_frames()' in realsense_device_manager.py.

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/box_dimensioner_multicam/realsense_device_manager.py#L219

Maybe it is looking in the folder that your script is in for the script file realsense_device_manager.py from the box_dimensioner_multicam project and cannot find it, so is erroring on the device_manager.poll_frames() command.

I appreciate your patience as we work through this problem!

[MartyG-RealSense]

Hi @dohyeonYoon Do you require further assistance with this case, please? Thanks!

[dohyeonYoon]

oh, I solve the problem whit this issues code! thank you for your help ! :)
#1735

[MartyG-RealSense]

Excellent news @dohyeonYoon - you are very welcome. Thanks very much for the update :)