"Frame Didn't Arrive Within 15000" issue with D435 while I push rs::frame to std::queue

[pangpangshui]

---

Required Info
Camera Model	D435
Firmware Version	05.12.13.50
Operating System & Version	Win 10
Kernel Version (Linux Only)	Win 10
Platform	PC
SDK Version	2.47.0
Language	c++
Segment	other

Issue Description

I want to push rs::frame to queue to save video, and use the origin frame to preview Real-time. But while I push 16 frame to queue, the error occured with using wait_for_frames. the code like this:

    window app(1920, 1080, "realsense_cameras");
    rs2::context                          ctx;        // Create librealsense context for managing devices
    std::queue<rs2::frame> test_frame_queue;

    // Capture serial numbers before opening streaming
    for (auto&& dev : ctx.query_devices())
        m_serials.push_back(dev.get_info(RS2_CAMERA_INFO_SERIAL_NUMBER));

    // Start a streaming pipe per each connected device
    for (auto&& serial : m_serials) {
        rs2::pipeline pipe(ctx);
        rs2::config cfg;
        cfg.enable_device(serial);
        cfg.enable_stream(RS2_STREAM_COLOR, 848, 480, RS2_FORMAT_RGB8, 30);
        rs2::pipeline_profile profile = pipe.start(cfg);
        pipelines.emplace_back(pipe);
        // Map from each device's serial number to a different colorizer
        colorizers[serial] = rs2::colorizer();
    }

    while (app) {
        std::vector<rs2::frame> new_frames;
        // Collect the new frames from all the connected devices
        for (auto &&pipe : pipelines) {
            rs2::frameset fs;
            try {
                fs = pipe.wait_for_frames();
            } catch(std::exception &e) {
                LOG("LOG_DEBUG", "wait_for_frames, error: %s", e.what());
            } catch(...) {
                 LOG("LOG_DEBUG", "wait_for_frames, error");
            }

            new_frames.emplace_back(fs.get_color_frame());
        }

        // Convert the newly-arrived frames to render-friendly format
        for (const auto& frame : new_frames) {
            // Get the serial number of the current frame's device
            auto serial = rs2::sensor_from_frame(frame)->get_info(RS2_CAMERA_INFO_SERIAL_NUMBER);
            if (frame_temp) {
                test_frame_queue.push(frame);
            }

            render_frames[frame.get_profile().unique_id()] = colorizers[serial].process(frame);
        }

        app.show(render_frames);
    }

Please help and thanks!

[MartyG-RealSense]

Hi @pangpangshui The frame queue of the RealSense SDK can process up to 16 frames at a time. So if the frames are not being released then the queue could get jammed up with frames, causing the application to time-out.

The 16-frame limit is addressed in advice in the link below, which recommends using the SDK's Keep() function to resolve the situation. Keep() enables frames to be stored in the computer's memory capacity.

#2711 (comment)

An advantage of Keep() is that instead of constantly writing frames to file, you can store all of the frames in memory capacity (so long as you have enough memory capacity remaining as it gets progressively filled during recording) and then close the pipeline and perform batch operations such as post-processing, align and saving to file on the whole set of frames at the same time.

[pangpangshui]

Thanks. I have a more question, if I use frame.keep(), How should I release thease frames?

[MartyG-RealSense]

There is a C++ example script for Keep() provided in the link below that you can test.

#1942 (comment)

[pangpangshui]

I change frame.reset() from protected to public and use it to release frame. It's OK now.

Example in #1942 (comment) will lead to memory grows crazy.

[MartyG-RealSense]

That's great news that you developed a solution - thanks for the update :)