-
Notifications
You must be signed in to change notification settings - Fork 5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ffmpeg h264_vl42m2m encoded videos do not play back #5150
Comments
@jc-kynesim can you reproduce this? |
I now understand what is going on - the mp4 muxer appears to require the SPS/PPS extradata at init time if it is going to take it from the context, but the h/w encoder won't provide it before it has been given its first frame (so after that). However it can also be passed along attached to the encoded packet as sidedata so I'll fix the encoder to do that and fix the mp4 muxer to respect it. |
Encoder being the FFmpeg encode side, or does it need a kernel change? |
Just ffmpeg - it all works fine if you have a raw h264 stream as the output type from ffmpeg - it is just separating off SPS/PPS for the mp4 muxer that has issues and that is all internal to ffmpeg (I do make the nasty assumption that SPS/PPS turns up on its own in the 1st capture frame which isn't a part of the V4L2 spec but I recall you saying was always true for our encoder). |
There is a V4L2 control V4L2_CID_MPEG_VIDEO_HEADER_MODE. FFmpeg chooses V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE to ask for the the header bytes to be separate from the 1st frame. The default is that they are joined with the first frame. |
Ah! Missed that. Excellent. |
Fix now pushed to my ffmpeg repo (https://github.com/jc-kynesim/rpi-ffmpeg.git branch test/4.3.4/rpi_main if you want to build it yourself). It'll be in the next ffmpeg update. Also fixes the same issue in mkv generation. |
@jc-kynesim What are the build parameters used to build from source? I'm not having any luck. |
You should find build/install instructions in pi-util/BUILD.txt - if they don't work or are unclear please tell me |
Ahh, didn't know that was there. I'll try that and let you know. |
No luck. Ran make celan and manually cleaned up the out folder but it's not working. Not sure what to do about this error:
|
Run "git clean -dxf" - that will get rid of the old in-tree objects etc. |
Well, finally got back around to this. It completed compiling but after I ran a make install but it didn't actually install ffmpeg anywhere. Now its just completely broken (i.e. no ffmpeg in path). I tried both shared and static. What did I miss? Update: NM, I see that this doesn't build something that is installed with |
I just noted that 4.3.4 was up on the repos. Is that this version already or do we need to wait a bit longer? Also, something I did here today completely broke the camera. libcamera-jpeg says:
I ran in to this problem before and reported it but it got closed because I couldn't replicate. Well, here its replicated again. What part of this could have permanently broken (until a new image is loaded) the camera? |
a) I'm pretty sure that the BUILD.txt I pointed you at made it clear that you needed to get rid of the previous version and copy the result if you wanted it globally accessible. |
Yeah, I didn't finish reading the build.txt but noted that afterward. Regarding the version number, I was just trying to clarify since that is also the version number on your repo but I confirmed that that version is also broken still. Regarding the camera, I wasn't expecting you to know, but was hoping maybe something would make sense when you heard it. I'm just reloading.....AGAIN 😠 |
And how update for RPi4 HAOS? |
That's a question for the HAOS devs, surely? |
Does anyone have a binary with this fix incorporated? |
@jc-kynesim Would you have an estimate on how long these releases take and which version will contain this fix? My path to have this working is wait for the ffmpeg update and then for HAOS to uptake that ... I'd appreciate you shedding some light on this for me. Thanks |
There was a recent ffmpeg update in RPiOS (timestamp shows it was built on Nov 7). |
Is there any way to get that ffmpeg update installed using apt on a Pi? I tried installing the deb files manually but it's one dependency after another missing and it was getting a bit crazy, obviously this was what apt was made for. apt sources is the standard: Many thanks Edit: I managed it somehow (7:4.3.5-0+deb11u1+rpt2) |
@jc-kynesim This does not appear to have trickled up, as nightly builds still don't include these patches. They also target a very old version of ffmpeg. Any update for us? |
still doesn't work for some people @jc-kynesim do you have some news? thanks on advance |
When you say 'still does not work for some people' - does that imply that it does work for others? |
Not a developer, but there are extensive threads detailing what doesn't work (here: blakeblackshear/frigate#3780) and an official bug fix request at the ffmpeg page here: https://trac.ffmpeg.org/ticket/10060. The specific case that locks "some people" out of using acceleration is an RPi4 64 bit scenario with ffmpeg packaged from your repository as part of Frigate. Frigate runs as an add-on (in a container) in Home Assistant OS with the affected ffmpeg version (4.3.3?) and acceleration used by cameras does not work. Add-on container images can be manually patched when they launch to fix but this is temporary until the next restart and beyond the hobbyist / retail user (me) capabilities. I'm sure I could pull it off if I dedicated an evening to it but it's a band-aid. Since Home Assistant can be ran in multiple configurations, only the typical HAOS + add-on install mode is not permanently fixable, as far as I know. If you run HA on non-restricted linux distribution then ffmpeg that Frigate uses can be patched easier, if you know what you're doing. More info here, with many messy examples of what works and what doesn't: blakeblackshear/frigate#3780 @jherby2k has a good handle on where it is reproducible and where it isn't, and took the initiative to open the ffmpeg request which in theory will trickle down to your project which will fix the ffmpeg version the Frigate project ultimately uses downstream. I think the last idea here was if you can help influence the ffmpeg project bug fix request, or debug and fix it in parallel in your project. The definition of the issue is obtainable, and I can help get you the info on what I'm running to help, but the Frigate project link above is teeming with examples. Thanks @jc-kynesim |
Is that FFmpeg bug report valid for this issue? It's referencing DECODE, whereas this is ENCODE. Accurate bug reports are critical if you want developers to investigate them. |
I get what you're saying, it would be hilarious if it was the reverse. Though I believe this issue (5150) describes the fix enough for people to take the solution, apply it to their custom compiled versions of ffmpeg and restore the functionality in Frigate. Frigate uses ffmpeg to decode https://docs.frigate.video/configuration/hardware_acceleration but maybe its not the act of decoding that's broken, but some other common requirement for encoding/decoding to take place. I could be totally wrong (disclaimer). Does this provide more of a clue why people think this issue for encoding is linked with accelerated decoding issue in Frigate? jc-kynesim/rpi-ffmpeg#53 (comment) |
That's the sum total of the fix for this issue related to encoding videos. If using https://github.com/jc-kynesim/rpi-ffmpeg.git also fixes something else, then great, but it is not related to this issue regardless of what others think. |
As the frigate maintainer, I can confirm that this issue is not relevant. |
Thanks for taking the time to do this, then. Clarity is good. |
thank you |
Describe the bug
When trying to encode using h264_vl42m2m within ffmpeg, the resulting videos are useless. However, by simply changing to libx264, and nothing else, videos are fine. For example:
ffmpeg -i sample_1920x1080.mp4 -c:v h264_v4l2m2m -b:v 8M -c:a copy sample_1920x1080_test.mp4 - VIDEO NO GOOD
ffmpeg -i sample_1920x1080.mp4 -c:v libx264 -b:v 8M -c:a copy sample_1920x1080_test.mp4 - VIDEO WORKS
Here's the output from the h264_v4l2m2 encoder:
Here's the output from the libx264 encoder:
I'm really not sure what to try from this point. I am all the way up to date packages and firmware. I would expect this encoder to just work like all the others but maybe we're not there yet. Any ideas on how to get this outputting usable hardware encoded videos here?
Thanks!
Steps to reproduce the behaviour
try:
ffmpeg -i sample_1920x1080.mp4 -c:v h264_v4l2m2m -b:v 8M -c:a copy sample_1920x1080_test.mp4
...and watch as it consumes power and outputs garbage.
Device (s)
Raspberry Pi 4 Mod. B
System
Raspberry Pi 4B, 8GB model
Raspberry Pi reference 2022-04-04
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 27a8050c3c06e567c794620394a8c2d74262a516, stage4
Aug 23 2022 14:23:34
Copyright (c) 2012 Broadcom
version bf881c824bac214626a9f78094d614f7db462c34 (clean) (release) (start)
Linux pi5 5.15.61-v8+ #1578 SMP PREEMPT Wed Aug 24 11:51:18 BST 2022 aarch64 GNU/Linux
Logs
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: