Frames didn't arrive error - after improper shutdown

[motudor]

Required Info
Camera Model	D435
Firmware Version	05.08.15.00
Operating System & Version	Ubuntu 16.04.3 LTS
Kernel Version (Linux Only)	4.13.0-32-generic
SDK Version	2.9.1

Hi all,

I am working on a project where I am using a realsense D435 sensor to get data about the position of some objects and process that using OpenCV.

Normally, the sensor works fine. But if the power is turned off to the computer (or the computer is reset) while the program is running, when I try to run the program after powerup, I get:

RealSense error calling rs2_pipeline_wait_for_frames(pipe:0x8ceea0):
Frame didn't arrived within 5000

The device is detected by query_devices() and I can read information from it using dev.get_info() but when I try to stream the depth data I get the error.

Plugging the cable out and then in again solves the issue. Just shutting down the computer, waiting a few minutes and powering up again, does not solve it.

Unfortunately, most of the shutdowns that I can expect will be improper ones (power failures) and plugging the cable out is not an option because the sensor is mounted on an automatic crane and it could takes hours until a technician gets to the power cabinet and re-plugs the cable.

When I try to run realsense-viewer, the device is detected, but when I try to stream the depth data it also fails:

 26/01 19:59:28,850 WARNING [140202273707776] (sensor.cpp:313) Unregistered Media formats : [ UYVY ]; Supported: [ ]
 26/01 19:59:28,852 WARNING [140202273707776] (backend-v4l2.cpp:1100) Pixel format 36315752-1a66-a242-9065-d01814a likely requires patch for fourcc code RW16!
 26/01 19:59:28,852 WARNING [140202273707776] (sensor.cpp:313) Unregistered Media formats : [ RW16 ]; Supported: [ ]
 26/01 19:59:44,757 WARNING [140202240136960] (backend-v4l2.cpp:809) Empty frame has arrived.
 (empty frame has arrived repeats 31 times)
 26/01 19:59:49,776 WARNING [140202240136960] (backend-v4l2.cpp:827) Frames didn't arrived within 5 seconds
 26/01 19:59:49,787 WARNING [140202160944896] (backend-v4l2.cpp:827) Frames didn't arrived within 5 seconds
 26/01 19:59:54,781 WARNING [140202240136960] (backend-v4l2.cpp:827) Frames didn't arrived within 5 seconds
 26/01 19:59:54,792 WARNING [140202160944896] (backend-v4l2.cpp:827) Frames didn't arrived within 5 seconds
...

The udevadm monitor does not show anything when I run my program or realsense-viewer.

I have tried to use linux commands to replicate plugging the usb out and in again (like the ones presented here: https://askubuntu.com/questions/342061/power-on-off-usb-ports), but without any sucess.

So what should I try next?

[zivsha]

Hi @motudor
Are you using some sort of USB hub?

[motudor]

Hi @zivsha
I am not using any hub. The camera is plugged directly in the PC, using the original cable that came with the camera.

[zivsha]

Sound like this might be a firmware issue.
Trying to reproduce this on my Windows laptop fails, I will also try on a Linux machine.

As a workaround, can you try to check if calling hardware_reset() restores the camera's ability to stream?

rs2::device dev = device_you_are_using();
dev.hardware_reset(); //Will disconnect the camera and reconnect it. 
//NOTE: All camera settings will be restored to default after this call (same as unplug->plug)
//Don't use device after this line

[motudor]

After using hardware_reset() the depth stream works again.

That means I can use this camera on the real machine now - I can just call hardware reset at the beginning of the program (at least until we find a more elegant solution). Thank you for the help!

[zivsha]

Glad to see that this w/a works. You can combine the hardware reset with rs2::device_hub to have a synchronous flow like so:

rs2::context ctx;
rs2::device dev = get_device_from_context(ctx);
dev.hardware_reset();
rs2::device_hub hub(ctx);
dev = hub.wait_for_device(); //Note that device hub will get any device, if you have more than 1 connected it could return the other device

[motudor]

Thanks for that!

The plan for this project is to use two cameras, and, as I did not find a way for getting wait_for_device() to work for a certain camera, I have added a delay instead. I figured out the reset time for the D435 is between 2 or 3 seconds, so if I sleep for 5 seconds it should be fine. (If the sleep time is too short I get a permission denied error when I try to open the pipe)

Is there a way to wait for a device with a certain serial number?

My code now looks something like this:

    rs2::context ctx;
    auto list = ctx.query_devices();
    for (int i=0;i<list.size();i++)
    {
        rs2::device dev=list[i];
        std::string dev_serial_number(dev.get_info(RS2_CAMERA_INFO_SERIAL_NUMBER));
        if (serial_number==dev_serial_number)
        {
            cout<<"Device found"<<endl;
            dev.hardware_reset();
            boost::this_thread::sleep(boost::posix_time::seconds(5));
            break;
        }
    }

[zivsha]

A general way to handle device connectivity is using:

rs2::context ctx;
ctx.set_devices_changed_callback([dev](rs2::event_information e) {
    e.get_new_devices();
    e.was_removed(dev);
});

Take a look at rs-multicam example to see how we handle it there.

I'm not sure what was the reason not to have hub.wait_for_device(serial), but I will check and maybe we will add such functionality.
Either way, this doesn't look like a software bug, so let's hope we can reproduce it and fix the root cause.

[KeCh96]

I have the same problem with python2 wrapper on Ubuntu. I run the following codes:

import pyrealsense2 as rs
pipeline = rs.pipeline()
pipeline.start()
frames = pipeline.wait_for_frames()

Sometimes I can get frames successfully, but sometimes I get error:

Traceback (most recent call last):
File "test.py", line 33, in
frames = pipeline.wait_for_frames()
RuntimeError: Frame didn't arrived within 5000

When this error occurs, I will run my code again and again, until the frames arrive. But sometimes I have to try many times, it wastes my time. How to solve this problem? @zivsha

[zivsha]

Hi @KeCh96 ,
This doesn't sound like the same issue, please kindly open a new issue with additional details and we will try to assist

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment]
Hi @motudorD

Did you try the sample app of the latest librealsense and still face the issue on your system?
Not see this issue on our test configuration: NUC + Ubuntu 16.04 + kernel 4.14.15. Both sample app for C or python work with D435.

[andophine]

same issue, occurs on D410 module+Intel UP Board with Android OS, if restart streaming after stop streaming.

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment]
Hi @motudor, @andophine,

Still has the issue with the latest librealsense build?

[motudor]

Hi @RealSense-Customer-Engineering
I will check and get back with an answer in a few days - as soon as I start working on a new machine in the workshop.

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment]
Hi @motudor,

Any further update? Is the issue resolved?

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment]
Hi @motudor,

I will close this first. If you still have any issue, you can re-open it.

[motudor]

It did take a longer time to get back to this, but I can now confirm the problem is solved with the latest library and firmware.

[Hubert51]

I use usb hub. Does this matter?

[HippoEug]

Hi @Hubert51 , are you able to provide more details on what USB Hub you're using? Is it USB3? How is the hub powered?

[bgyooPtr]

Is it possible in ROS?

[Algabri]

I have the same problem with astra camera