-
Notifications
You must be signed in to change notification settings - Fork 112
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create av1_sample_buffer_support.go #238
base: master
Are you sure you want to change the base?
Conversation
Great work @alexpokotilo! Mind writing some unit tests for this? I just want to make sure some of the slice indexing can't cause a panic. thank you |
add class to support av1 unmarshaling in SampleBuffer
use fallthrough remove TODOs
c713c07
to
ba5515a
Compare
Hi @Sean-Der! |
🔥 great work @alexpokotilo! I will let CI run and if that passes I will merge |
@Sean-Der I moved my files to prevent cyclic dependencies. I can add something simple to emulate SampleBuffer behavior into this test to remove dependency from rtp repo. |
add tests for AV1PacketSampleBufferSupport
242316b
to
75ee6c6
Compare
@alexpokotilo Could you please explain why this code is necessary? Why is it not enough to remove the aggregation headers and concatenate the resulting payloads? |
Hi @jech, thanks for the question!
Current codecs.AV1Packet{} implementation doesn't have isPartionHead and isPartionHead to be used with samplebuilder. Without samplebuilder from another hand we cannot just clue av1 rtp chanks because of jitter. but samplebuilder helps us with it. Check codecs.H264Packet implementation as example. h264 payload returned in annexB form but not in rtp form. |
I understand that. Please correct me if I'm wrong, however, I don't think you need to create a list of OBUs, I think you can build the packet on the fly. Suppose you take the incoming RTP packets, reorder them, and take the sequence of packets until you find one with the mark bit set. Then you remove the aggregation headers, concatenate the rest of the bodies. You get exactly the sequence of concatenated OBUs, no? With no need to allocate a list of OBUs? So the only change in the Samplebuilder is this line: which will need to be modified to remove the aggregation header. Or am I missing something? |
This could be implemented by adding an interface
and in samplebuilder do something like
PacketAppender would only need to be implemented for AV1. |
To create sample builder we need to call samplebuilder.New and you have to provide depacketizer rtp.Depacketizer for codec you want to depacketize. We don't have any rtp.Depacketizer implementation for Av1. So my version of rtp.Depacketizer for Av1 is AV1PacketSampleBufferSupport. When func (s *SampleBuilder) Pop() called and it calls buildSample where Unmarshal is called.
Please take a look at func (p *AV1Packet) Unmarshal There is is a comment "// If W bit is set the last OBU Element will have no length header" so I don't think OBU list is produced by just removing aggregation header. The last OBU doesn't have length at all and OBU list will be corrupted if you go your way. In my approach I use frame.AV1 to do the job. I'm not sure I do it 100% right. I just added working method to use samplebuffer and av1 which didn't exist. |
the problem here is that there is no av1 depackager implementing rtp.Depacketizer interface and this is why I added my class. you cannot just implement new PacketAppender interface without adding something to fulfill rtp.Depacketizer for av1. You will not be able just cut header because the last OBU in rtp doesn't have size |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #238 +/- ##
==========================================
- Coverage 86.21% 85.97% -0.25%
==========================================
Files 22 23 +1
Lines 1734 1797 +63
==========================================
+ Hits 1495 1545 +50
- Misses 203 211 +8
- Partials 36 41 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
I'm sure you're right and I'm wrong, but I still don't understand why. Perhaps you could give a concrete example where appending the packet bodies (with the aggregation headers removed) would not work?
It would remove the aggregation header and append the rest of the packet to the given buffer. |
Folks, please refrain from merging this commit until Monday, I want to work out why I'm wrong. |
Please see #264. |
Great! |
Please see #264 (comment) for another comment. |
add class to support av1 unmarshaling in SampleBuffer
Description
Now AV1Packet cannot be used with Sample buffer. I implemented new class for av1 assembling with SampleBuffer.
Works fine for receiving AV1 from Chrome and returning AV bitstream
Reference issue
Fixes #189