get_timestamp returning a timestamp that is less than the previous frame timestamp

[GrubbyHalo]

Camera Model D430
Firmware Version 05.12.11.00
OS Raspbian 10
Kernel 5.10.25-v7l
Platform Rasberry Pi 4
SDK Version 2.45.0

I have the following piece of code that runs during a frame callback

else if (f.get_profile().stream_type() == RS2_STREAM_DEPTH)
{

        char* d = (char*)vf.get_data();
        double fts=f.get_timestamp();
        if(fts < last_fts){
            DebugString("Current frame timestamp is smaller than last.. current fts= " +
                                            FloatToString(fts)+" last fts="+FloatToString(last_fts));
        }
        last_fts=fts;
}

And after about 30 seconds I start getting the message that the current frame timestamp is smaller than the last frame timestamp

Is it possible for frames to come out of order ?

[MartyG-RealSense]

Hi @GrubbyHalo Advice provided in #7837 (comment) by a RealSense team member about 'old' frames being returned may be helpful.

[GrubbyHalo]

Hi @MartyG-RealSense

The issue referenced in #7873 seems to be slightly different. The problem there seems to be that the timestamp of the actual call to get_timestamp is being compared to the frame timestamp. Also that users snippet of code is calling wait_for_frames as opposed to a callback being used to retrieve and process the frame.

In my code above, I am comparing the frame time stamp of subsequent frames and finding a discrepancy between successive frames.

I was under the impression that if a frame is not processed by an application that it is dropped or does the backend buffer frames and only send the very latest frame and then start sending older frames ?

[MartyG-RealSense]

This particular subject of timestamps with frame callback is admittedly outside of my personal knowledge. I did though find a case of incorrect timestamps when using callbacks at #5143 in which the same RealSense team member provides advice.

[GrubbyHalo]

It seems this issue does not happen on a Raspberry Pi 3 B plus or a x86-64 platform.

[MartyG-RealSense]

Are there any differences in how your Pi 4 is set up compared to the Pi 3B+. For example, are both using Raspbian 10?

[GrubbyHalo]

There are differences... The pi3 b+ has an OS and kernel from about 3 years ago.. The pi4 has the OS and kernel released in about July this year. I am currently compiling a newer kernel and modules to see if it makes any difference

[GrubbyHalo]

I have recompiled the kernel to v5.10.6 without any luck. I also downloaded the latest version of Raspbian (Release date: May 7th 2021) and also no luck.
I have also added a check to see if the frames come in order using the rs2::frame::get_frame_number() API call and all frames come in order

I am absolutely flummoxed.

[MartyG-RealSense]

When the program is running on Pi 4 do you see any visible negative effects from this issue with frame timestamps, or is it an issue that only appears in the timestamp debug-string and otherwise does not seem to impact performance of the application?

[GrubbyHalo]

The application that utilises the real sense camera depends on having frames arrive in chronological order.
However I have managed to work around this issue. I noticed that the timestamp domain for each frame was using RS2_TIMESTAMP_DOMAIN_GLOBAL_TIME and that this method of obtaining a timestamp uses a combination of system time in combination with the camera hardware clock. I subsequently forced the usage of only the hardware clock as outline in this response and it seems to have solved the problem.

[MartyG-RealSense]

Great to hear that you achieved a solution, @GrubbyHalo - thanks very much for the update :)