D455 timestamp duplicate

[lyfadvance]

Required Info
Camera Model	D455 }
Firmware Version	5.13.0.50
Operating System & Version	Ubuntu22.04
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC/Raspberry Pi
SDK Version	2.55 and 2.53 }
Language	{c++ }
Segment	{Robot/Smartphone/VR/AR/others }

Issue Description

When I turn on two infrared cameras and one RGB camera, the timestamps of the streams from these three cameras are duplicated. However, when I only turn on two infrared cameras and turn off the RGB camera, the timestamps are no longer duplicated.

timestamps as folllow:
1721122914959027968 1721122914959.0280762
1721122914959027968 1721122914959.0280762
1721122914959027968 1721122914959.0280762
1721122914959027968 1721122914959.0280762
1721122914959027968 1721122914959.0280762
1721122915430711296 1721122915430.7111816
1721122915430711296 1721122915430.7111816
1721122915430711296 1721122915430.7111816
1721122915469847552 1721122915469.8476562
1721122915469847552 1721122915469.8476562
1721122915469847552 1721122915469.8476562

code as follow:

config.enable_stream(RS2_STREAM_ACCEL, RS2_FORMAT_MOTION_XYZ32F, d455.accel_fps);
    config.enable_stream(RS2_STREAM_GYRO, RS2_FORMAT_MOTION_XYZ32F, d455.gyro_fps);
    config.enable_stream(RS2_STREAM_INFRARED, 1, 640, 480, RS2_FORMAT_Y8, 30);
    config.enable_stream(RS2_STREAM_INFRARED, 2, 640, 480, RS2_FORMAT_Y8, 30);
    //config.enable_stream(RS2_STREAM_COLOR, 1280, 800, RS2_FORMAT_BGR8, 30);
    config.enable_stream(RS2_STREAM_COLOR, 640, 480, RS2_FORMAT_BGR8, 30);

std::vector<rs2::video_frame> vfs;
		for (int i = 0; i < 3; ++i) {
			rs2::video_frame vf = fs[i].as<rs2::video_frame>();
			if (!vf) {
			  //BASALT_ASSERT(false && "Weird frame");
			  std::cout << "Weird Frame, skipping" << std::endl;
			  return;
			}
			vfs.push_back(vf);
		}

		// skip frames if configured
		if (frame_counter++ % skip_frames != 0) {
			std::cout << "skip frame" << std::endl;
			return;
		}
		cameraGetTime = std::chrono::steady_clock::now();

		const auto& vf = vfs[0];
		int64_t t_ns = vf.get_timestamp() * 1e6;
		std::cout << t_ns << " " << std::setprecision(30) << vf.get_timestamp() << std::endl;
		cam0TimeStampsForSave.emplace_back(t_ns);
		cv::Mat image(vf.get_height(), vf.get_width(), CV_8UC1);
		memcpy(image.data, vf.get_data(), sizeof(unsigned char) * image.rows * image.cols);
		cam0DatasForSave.emplace_back(image);
		cv::imshow("image", image);
		cv::waitKey(1);

		const auto& vf2 = vfs[1];
		int64_t t_ns2 = vf2.get_timestamp() * 1e6;
		cam1TimeStampsForSave.emplace_back(t_ns2);
		cv::Mat image2(vf2.get_height(), vf2.get_width(), CV_8UC1);
		memcpy(image2.data, vf2.get_data(), sizeof(unsigned char) * image2.rows * image2.cols);
		cam1DatasForSave.emplace_back(image2);
		const auto& vf3 = vfs[2];

		int64_t t_ns3 = vf3.get_timestamp() * 1e6;
		cam2TimeStampsForSave.emplace_back(t_ns3);
		cv::Mat image3(vf3.get_height(), vf3.get_width(), CV_8UC3);
		memcpy(image3.data, vf3.get_data(), sizeof(unsigned char) * image3.rows * image3.cols * 3);
		cam2DatasForSave.emplace_back(image3);

Is there a problem with the code above?

[MartyG-RealSense]

Hi @lyfadvance It is likely not a problem with your code.

At #5885 it is explained that if there are dropped frames then there may be repeated, or 'stale', frames. Frame drops could be caused by a high processing burden on the CPU.

In the Required Info list at the top of this issue, your platform is listed as PC/Raspberry Pi. Are you running your script on a Raspberry Pi, please?

If you are using a Pi then that could help to explain your problem with having three streams enabled (RGB, Infrared, Infrared 2). RealSense cameras can have problems on Pi boards. Pi is best suited to streaming two basic RealSense stream types simultaneously (such as Depth and RGB). IMU streams usually also cannot be accessed on Pi.

Having three streams enabled may be over-burdening the Pi. You could test this possibility by only enabling RGB and Infrared, but not Infrared 2. If the frame repeats still occur then this would indicate that the problem is not with the RGB stream but with the number of simultaneously enabled streams.

You might not need the Infrared 2 stream enabled anyway, as it is almost the same view as Infrared but slightly offset horizontally from Infrared's viewpoint.

[lyfadvance]

This could be a transmission rate issue caused by VMware Ubuntu. When I tested it again on the board, only the first 5 frames were repeated and there was no further repetition.

[MartyG-RealSense]

Thanks very much for the update!