Reducing Latency in ffplay with ffmpeg-rockchip Compared to gstreamer for RTSP Streams

[daoan1412]

Hello ffmpeg-rockchip community,

I am experiencing a noticeable delay issue when using ffplay compared to gstreamer for RTSP stream handling. With ffplay, using the following command:

ffplay -fflags nobuffer -flags low_delay -framedrop -strict experimental -rtsp_transport udp rtsp://[ANONYMIZED]

I encounter a delay of about 1-2 seconds compared to gstreamer:

gst-launch-1.0 rtspsrc location=rtsp://[ANONYMIZED] latency=100 ! rtph265depay ! h265parse ! decodebin ! autovideosink

Could you please explain why there is such a delay difference between these two approaches? Is there a way to optimize ffplay performance in this context, especially with ffmpeg-rockchip, to reduce latency and achieve similar performance as gstreamer?

Any insights or guidance would be greatly appreciated.

Thank you!

• Device: Orange pi 5 plus
• OS: Kernel .5.10.160-28

[hbiyik]

I would suggest not to use ffplay at all, since it does not suppport the drm prime frames.

[nyanmisaka]

@daoan1412
FFmpeg does not recommend using ffplay to test latency too.
https://trac.ffmpeg.org/wiki/StreamingGuide#Testinglatency

By default, ffplay (as a receiver for testing latency) introduces significant latency of its own, so if you use it for testing (see troubleshooting section) it may not reflect latency accurately.

So I doubt such a high latency is caused by ffplay/SDL itself and the internal software frame copies.

And the autovideosink in Rockchip gstreamer contains many hacks/optimizations specific to libmali driver. VOP2/GPU can directly access the DRM frames decoded by the MPP decoder through DMA, thus avoiding extra copies and latencies.

[hbiyik]

definetly the frame copies, internal ffmpeg yuv copy method is quite potato.

[daoan1412]

@nyanmisaka @hbiyik
However, my tests on a Linux x64 machine with an Nvidia card, using the same commands, showed almost no latency difference between ffplay and gstreamer. This leads me to believe that the issue does not lie in buffer copying. Moreover, I also conducted tests on the rk3588 platform with an Android version, building FFmpeg from source, and displaying it on the screen, which also did not show any significant real-time delay.

[hbiyik]

I will put PR wiki entry about how to get hw acceleration when rendering, but currently it is a little early, soon will be ready though

[nyanmisaka]

@daoan1412

I did find an issue regarding the high latency. Please try this patch and see if it helps.

[daoan1412]

hi @nyanmisaka it seems that this patch has fixed the high latency issue for ffplay. Currently, I notice only about a 500ms delay compared to gstreamer, which is acceptable for my application.

[nyanmisaka]

Closed by e753665