AV1 payloader

[mickel8]

This PR unfortunately doesn't work and I ran out of ideas on how to push it further. Current status:

• chromium successfully decodes and displays the first video frame
• I can't see any logs indicating we are doing something wrong. I ran dev build of chromium. The only thing I can see is

[2098991:3:1211/162436.394069:WARNING:dav1d_decoder.cc(135)] Dav1dDecoder::Decode decoding failed with error code -11
[2098991:3:1211/162436.394213:WARNING:generic_decoder.cc(302)] Failed to decode frame with timestamp 2618223154, error code: -1

According to dav1d IIRC this might not be the error but I think it shouldn't be logged

• I assumed that sending every OBU in a separate RTP packet is ok
• I assumed that every frame is a separate Temporal Unit (in the test file this should be true if only the code checking OBU type is correct) and therefore I set marker of the last RTP packet for every IVF frame
• I tried batching OBUs e.g. by 2 or setting W flag to 1 (instead of 0 and using LEB size) - didn't help
• I tried prepending every OBU with sequence header - this gives a massive number of decoder errors (-11)
• Pion send_from_file example also doesn't work but here I am getting

[2077476:12:1211/123520.805336:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 591
[2077476:12:1211/123520.838810:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123520.871969:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 33
[2077476:12:1211/123520.904092:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123520.936935:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 62
[2077476:12:1211/123520.970613:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123520.996106:WARNING:video_receive_stream2.cc(825)] No decodable frame in 230195 us, requesting keyframe.
[2077476:12:1211/123521.003523:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 33
[2077476:12:1211/123521.037086:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123521.069772:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 180
[2077476:12:1211/123521.102389:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123521.135413:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 28
[2077476:12:1211/123521.168366:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123521.201628:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 95
[2077476:12:1211/123521.214954:WARNING:video_receive_stream2.cc(825)] No decodable frame in 218582 us, requesting keyframe.

• if I don't drop Temporal Units, I can see

[2080197:11:1211/130520.654168:WARNING:rtp_seq_num_only_ref_finder.cc(65)] Generic frame with packet range [51605, 51605] has no GoP, dropping frame.

• maybe this has something to do with this paragraph of AV1 RTP format specification (see here)

If a sequence header OBU is present in an RTP packet and operating_points_cnt_minus_1 > 0 then for any number i where 0 <= i < operating_points_cnt_minus_1 the following MUST be true: (operating_point_idc[i] & operating_point_idc[i+1]) == operating_point_idc[i+1].

A sender MAY produce a sequence header with operating_points_cnt_minus_1 = 0 and operating_point_idc[0] = 0xFFF and seq_level_idx[0] = 0. In such case, seq_level_idx[0] does not reflect the level of the operating point.

Note: The intent is to disable OBU dropping in the AV1 decoder. To ensure a decoder’s capabilities are not exceeded, OBU filtering should instead be implemented at the system level (e.g., in a MANE).

but because it sounds pretty complex I didn't dive into it