Does get_frame_number() return value in milliseconds?

[thekenu]

Required Info
Camera Model	N/A
Firmware Version	N/A
Operating System & Version	N/A
Kernel Version (Linux Only)	N/A
Platform	N/A
SDK Version	Latest (v2.47.0)
Language	C
Segment	N/A

Issue Description

This is a question about the API documentation, so that's why I had put N/A for many of the "Required Info".

The documentation for rs2_get_frame_number says the return value is:

"...the frame number of the frame, in milliseconds since the device was started".

My understanding of "frame number" is that it's simply a counter that gets incremented by 1 between consecutive frames. For example, if you have a 30Hz stream, this counter will get incremented 30 times per second (assuming no frame drops, of course). However, I can't wrap my head around what "frame number" has to do with:

"milliseconds since the device was started".

The documentation for get_frame_number, which calls r2_get_frame_number in its implementation, has the same wording for its documentation.

[MartyG-RealSense]

Hi @thekenu A good way to illustrate visually how frame count is based on milliseconds since the device started is to enable a stream in the RealSense Viewer program and left-click on the 'i' icon at the top of the stream's panel. This will overlay live frame information at the top of the stream panel, including the frame number count that rapidly increments like an athletics stopwatch.

It is not quite as straightforward as being purely a time counter though. For example, if there is a 'hiccup' in the stream then the SDK may return the last known good frame before progressing onwards, resulting in repeated frame numbers being reported.

#1686

[thekenu]

Thanks for the timely response Marty. I see that the frame number increments over time. However, does a frame number of 1 mean 1 millisecond has passed since the device started? And a frame number of 2 mean 2 milliseconds have passed since the device started etc? That's what I got from reading the documentation.

[MartyG-RealSense]

It is probably easier to accept the frame number as being whatever value is reported by the SDK, otherwise you can end up exploring a very deep rabbit-hole involving clocks and timestamps. :)

#2188

The majority of frame number references in the SDK documentation and code files that my research could find simply repeat the statement that the frame count represents "the frame number of the frame, in milliseconds since the device was started".

[thekenu]

I agree there's a rabbit hole associated with timestamps, but that's get_frame_timestamp and not get_frame_number. I still don't see how frame numbers are related to the concept of time. Plus, if you look at the comment for the metadata enum, it desn't even mention the millisecond. This inconsistency in the documentation makes me hard to believe that "the frame number of the frame, in milliseconds since the device was started" is correct.

It is probably easier to accept the frame number as being whatever value is reported by the SDK,

I see where you're coming from. But to me this is just basically saying the SDK can return any arbitrary garbage value, and I should never use the frame number for anything?

[MartyG-RealSense]

Generally with frame counts, the issues that are of primary concern to RealSense users in regard to frame counts are (1) whether the frame numbers are repeating, and (2) if more than one stream is being used, are the total number of recorded frames for each stream significantly different. Timestamps tend to be the measure used for comparison of streams rather than frame numbers.

[thekenu]

(2) if more than one stream is being used, are the total number of recorded frames for each stream significantly different

Sorry I didn't quite grasp exactly what you meant here. If I have 2 streams both running at 30fps, why would the total number of recorded frames from each stream be significantly different?

Timestamps tend to be the measure used for comparison of streams rather than frame numbers.

In #1686, the author noted that the hardware timestamp is also repeating too. So wouldn't this be the same problem as repeating frame number?

[MartyG-RealSense]

In a case linked to below where the frame number of depth and color frames was mismatched, it was advised that auto-exposure being enabled could cause this to occur.

#2224 (comment)

Timestamps can repeat as well as frame numbers, yes. Here is an example of such a case.

#5885

The same frame number with a different timestamp would not be considered a valid behavior, according to the advice below.

#2214 (comment)

Thanks very much for your patience in regard to my comments, as I am not a programming specialist!

[thekenu]

Thanks! I think I will close this for the time being since I have no additional questions.

[MartyG-RealSense]

Thanks very much @thekenu for the update!