Video / audio sync problem in RTMP streams ingested from Dahua cameras

[rudyryk]

Which version are you using?

v1.3.1

Which operating system are you using?

• Linux amd64 standard
• Linux amd64 Docker
• Linux arm64 standard
• Linux arm64 Docker
• Linux arm7 standard
• Linux arm7 Docker
• Linux arm6 standard
• Linux arm6 Docker
• Windows amd64 standard
• Windows amd64 Docker (WSL backend)
• macOS amd64 standard
• macOS amd64 Docker
• Other (please describe)

Describe the issue

When publishing RTMP streams from Dahua cameras sometimes audio is delayed for about 1 second (more or less).

Describe how to replicate the issue

1. Start the MediaMTX server
2. Push RTMP stream from camera (Dahua, e.g. N43CG62) to rtmp://<MediaMTX URL>/test-stream
3. Play stream with RTMP player, e.g. ffplay -i rtmp://<MediaMTX URL>/test-stream
4. or record stream with ffmpeg e.g. ffmpeg -i rtmp://<MediaMTX URL>/test-stream -c copy test-stream.mp4

Is the issue is not happening, try again.

Did you attach the server logs?

No

Did you attach a network dump?

Yes (will send by email)

[aler9]

Hello @rudyryk, i took a look at the network dumps through Wireshark, and in particular at the pcap corresponding to the "bad" sample.

For the first two seconds, your camera just sends audio, without video. The first audio sample has a timestamp of "0" and was received at 0.73 seconds:

At 2.1 seconds, video starts to appear, but it has a timestamp of "0" instead of "2.1 seconds":

This video frame is associated with the audio frame that was sent with the same timestamp, that is zero, so the video frame received at 2.1 seconds it associated with the audio frame received at 0.73 seconds, and that is why video and audio are not synced correctly.

This mess is caused by the camera.

In my opinion you can mitigate the issue by using a workaround: in RTSP, contrarily to RTMP, timestamp has a starting value that is random and unique for each track, and a video frame received at X seconds is always associated with the audio frame received at X seconds, regardless of the timestamp field. Therefore, if you publish the stream to the server with RTMP, but you read this same stream with RTSP, the problem will disappear. You can even use FFmpeg to obtain a synced RTMP stream:

ffmpeg -i rtsp://server/unsynced-stream -c copy -f flv rtmp://server/synced-stream

[rudyryk]

@aler9 Thank you so much, appreciate your help on diving into the problem. We'll pass the information to the camera manufacturer.

Btw, the real offset is around 400ms, while the difference between 2.1s and 0.73s is around 1400ms, do you know where is the extra 1s comes from?