Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ffmpeg h264_vl42m2m encoded videos do not play back #5150

Closed
JooJooBee666 opened this issue Aug 29, 2022 · 32 comments
Closed

ffmpeg h264_vl42m2m encoded videos do not play back #5150

JooJooBee666 opened this issue Aug 29, 2022 · 32 comments

Comments

@JooJooBee666
Copy link

JooJooBee666 commented Aug 29, 2022

Describe the bug

When trying to encode using h264_vl42m2m within ffmpeg, the resulting videos are useless. However, by simply changing to libx264, and nothing else, videos are fine. For example:

ffmpeg -i sample_1920x1080.mp4 -c:v h264_v4l2m2m -b:v 8M -c:a copy sample_1920x1080_test.mp4 - VIDEO NO GOOD
ffmpeg -i sample_1920x1080.mp4 -c:v libx264 -b:v 8M -c:a copy sample_1920x1080_test.mp4 - VIDEO WORKS

Here's the output from the h264_v4l2m2 encoder:

ffmpeg -i sample_1920x1080.mp4 -c:v h264_v4l2m2m -b:v 8M -c:a copy sample_1920x1080_test.mp4
ffmpeg version 4.3.4-0+deb11u1+rpt3 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr --extra-version=0+deb11u1+rpt3 --toolchain=hardened --incdir=/usr/include/aarch64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-mmal --enable-neon --enable-v4l2-request --enable-libudev --enable-epoxy --enable-sand --libdir=/usr/lib/aarch64-linux-gnu --arch=arm64 --enable-pocketsphinx --enable-libdc1394 --enable-libdrm --enable-vout-drm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'sample_1920x1080.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
  Duration: 00:00:28.24, start: 0.000000, bitrate: 10836 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 10833 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 47.95 tbc (default)
    Metadata:
      handler_name    : VideoHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_v4l2m2m))
Press [q] to stop, [?] for help
[h264_v4l2m2m @ 0x559c591f60]  <<< v4l2_encode_init: fmt=0/-1
[h264_v4l2m2m @ 0x559c591f60] Using device /dev/video11
[h264_v4l2m2m @ 0x559c591f60] driver 'bcm2835-codec' on card 'bcm2835-codec-encode' in mplane mode
[h264_v4l2m2m @ 0x559c591f60] requesting formats: output=YU12 capture=H264
[h264_v4l2m2m @ 0x559c591f60] Failed to set number of B-frames: Invalid argument
[h264_v4l2m2m @ 0x559c591f60] Failed to set gop size: Invalid argument
Output #0, mp4, to 'sample_1920x1080_test.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
    Stream #0:0(und): Video: h264 (h264_v4l2m2m) (avc1 / 0x31637661), yuv420p, 1920x1080, q=-1--1, 8000 kb/s, 23.98 fps, 24k tbn, 23.98 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.91.100 h264_v4l2m2m
frame=  677 fps= 21 q=-0.0 Lsize=   22094kB time=00:00:28.19 bitrate=6419.5kbits/s speed=0.871x    
video:22091kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.016396%

Here's the output from the libx264 encoder:

ffmpeg -i sample_1920x1080.mp4 -c:v libx264 -b:v 8M -c:a copy sample_1920x1080_test.mp4 
ffmpeg version 4.3.4-0+deb11u1+rpt3 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr --extra-version=0+deb11u1+rpt3 --toolchain=hardened --incdir=/usr/include/aarch64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-mmal --enable-neon --enable-v4l2-request --enable-libudev --enable-epoxy --enable-sand --libdir=/usr/lib/aarch64-linux-gnu --arch=arm64 --enable-pocketsphinx --enable-libdc1394 --enable-libdrm --enable-vout-drm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'sample_1920x1080.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
  Duration: 00:00:28.24, start: 0.000000, bitrate: 10836 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 10833 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 47.95 tbc (default)
    Metadata:
      handler_name    : VideoHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x55bd668f60] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x55bd668f60] profile High, level 4.0, 4:2:0, 8-bit
[libx264 @ 0x55bd668f60] 264 - core 160 r3011 cde9a93 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=23 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=8000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'sample_1920x1080_test.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
    Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x1080, q=-1--1, 8000 kb/s, 23.98 fps, 24k tbn, 23.98 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.91.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/8000000 buffer size: 0 vbv_delay: N/A
frame=  677 fps=5.2 q=-1.0 Lsize=   27765kB time=00:00:28.11 bitrate=8090.9kbits/s speed=0.216x       
video:27756kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.032574%
[libx264 @ 0x55bd668f60] frame I:3     Avg QP:20.58  size:457560
[libx264 @ 0x55bd668f60] frame P:177   Avg QP:26.51  size:116193
[libx264 @ 0x55bd668f60] frame B:497   Avg QP:31.01  size: 13043
[libx264 @ 0x55bd668f60] consecutive B-frames:  0.4%  3.0%  6.2% 90.4%
[libx264 @ 0x55bd668f60] mb I  I16..4:  2.9% 60.5% 36.6%
[libx264 @ 0x55bd668f60] mb P  I16..4:  0.7%  4.5%  1.2%  P16..4: 36.3% 26.2% 22.3%  0.0%  0.0%    skip: 8.7%
[libx264 @ 0x55bd668f60] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8: 52.8%  6.5%  2.1%  direct: 1.4%  skip:37.2%  L0:38.7% L1:50.5% BI:10.7%
[libx264 @ 0x55bd668f60] final ratefactor: 25.27
[libx264 @ 0x55bd668f60] 8x8 transform intra:67.9% inter:56.4%
[libx264 @ 0x55bd668f60] coded y,uvDC,uvAC intra: 77.1% 55.9% 6.6% inter: 17.7% 8.9% 0.0%
[libx264 @ 0x55bd668f60] i16 v,h,dc,p: 18% 21% 22% 39%
[libx264 @ 0x55bd668f60] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 12% 21%  6% 12%  9% 10%  7%  9%
[libx264 @ 0x55bd668f60] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 12% 21%  7% 11%  8% 10%  6%  8%
[libx264 @ 0x55bd668f60] i8c dc,h,v,p: 65% 16% 15%  3%
[libx264 @ 0x55bd668f60] Weighted P-Frames: Y:6.8% UV:0.6%
[libx264 @ 0x55bd668f60] ref P L0: 61.3% 26.3% 10.0%  2.2%  0.2%
[libx264 @ 0x55bd668f60] ref B L0: 95.4%  4.3%  0.4%
[libx264 @ 0x55bd668f60] ref B L1: 98.1%  1.9%
[libx264 @ 0x55bd668f60] kb/s:8052.25

I'm really not sure what to try from this point. I am all the way up to date packages and firmware. I would expect this encoder to just work like all the others but maybe we're not there yet. Any ideas on how to get this outputting usable hardware encoded videos here?

Thanks!

Steps to reproduce the behaviour

try:
ffmpeg -i sample_1920x1080.mp4 -c:v h264_v4l2m2m -b:v 8M -c:a copy sample_1920x1080_test.mp4
...and watch as it consumes power and outputs garbage.

Device (s)

Raspberry Pi 4 Mod. B

System

Raspberry Pi 4B, 8GB model

Raspberry Pi reference 2022-04-04
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 27a8050c3c06e567c794620394a8c2d74262a516, stage4

Aug 23 2022 14:23:34
Copyright (c) 2012 Broadcom
version bf881c824bac214626a9f78094d614f7db462c34 (clean) (release) (start)

Linux pi5 5.15.61-v8+ #1578 SMP PREEMPT Wed Aug 24 11:51:18 BST 2022 aarch64 GNU/Linux

Logs

No response

Additional context

No response

@popcornmix
Copy link
Collaborator

@jc-kynesim can you reproduce this?

@jc-kynesim
Copy link
Contributor

I now understand what is going on - the mp4 muxer appears to require the SPS/PPS extradata at init time if it is going to take it from the context, but the h/w encoder won't provide it before it has been given its first frame (so after that). However it can also be passed along attached to the encoded packet as sidedata so I'll fix the encoder to do that and fix the mp4 muxer to respect it.

@6by9
Copy link
Contributor

6by9 commented Sep 13, 2022

so I'll fix the encoder to do that and fix the mp4 muxer to respect it

Encoder being the FFmpeg encode side, or does it need a kernel change?

@jc-kynesim
Copy link
Contributor

Just ffmpeg - it all works fine if you have a raw h264 stream as the output type from ffmpeg - it is just separating off SPS/PPS for the mp4 muxer that has issues and that is all internal to ffmpeg (I do make the nasty assumption that SPS/PPS turns up on its own in the 1st capture frame which isn't a part of the V4L2 spec but I recall you saying was always true for our encoder).

@6by9
Copy link
Contributor

6by9 commented Sep 13, 2022

There is a V4L2 control V4L2_CID_MPEG_VIDEO_HEADER_MODE. FFmpeg chooses V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE to ask for the the header bytes to be separate from the 1st frame. The default is that they are joined with the first frame.
https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-codec.html

@jc-kynesim
Copy link
Contributor

Ah! Missed that. Excellent.

@jc-kynesim
Copy link
Contributor

Fix now pushed to my ffmpeg repo (https://github.com/jc-kynesim/rpi-ffmpeg.git branch test/4.3.4/rpi_main if you want to build it yourself). It'll be in the next ffmpeg update. Also fixes the same issue in mkv generation.

@JooJooBee666
Copy link
Author

JooJooBee666 commented Sep 15, 2022

@jc-kynesim What are the build parameters used to build from source? I'm not having any luck.

@jc-kynesim
Copy link
Contributor

You should find build/install instructions in pi-util/BUILD.txt - if they don't work or are unclear please tell me

@JooJooBee666
Copy link
Author

Ahh, didn't know that was there. I'll try that and let you know.

@JooJooBee666
Copy link
Author

JooJooBee666 commented Sep 15, 2022

No luck. Ran make celan and manually cleaned up the out folder but it's not working. Not sure what to do about this error:

pi@pi5:~/source/rpi-ffmpeg $ pi-util/conf_native.sh --noshared
Configure for native build
M/C aarch64
Static libs
Destination directory: /home/pi/source/rpi-ffmpeg/out/arm64-bullseye-4.3.4-static-rel
Out of tree builds are impossible with config.h in source dir.

@jc-kynesim
Copy link
Contributor

Run "git clean -dxf" - that will get rid of the old in-tree objects etc.

@JooJooBee666
Copy link
Author

JooJooBee666 commented Sep 15, 2022

Well, finally got back around to this. It completed compiling but after I ran a make install but it didn't actually install ffmpeg anywhere. Now its just completely broken (i.e. no ffmpeg in path). I tried both shared and static. What did I miss?

Update: NM, I see that this doesn't build something that is installed with make install and it just needs to be manually copied. This version produces playable videos! 🥂

@JooJooBee666
Copy link
Author

JooJooBee666 commented Sep 16, 2022

I just noted that 4.3.4 was up on the repos. Is that this version already or do we need to wait a bit longer?

Also, something I did here today completely broke the camera. libcamera-jpeg says:

0:04:09.668224540] [1723]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3866-0c55e522
[0:04:09.688411074] [1727] ERROR V4L2 v4l2_device.cpp:349 'imx477 10-001a': Unable to set controls: Device or resource busy

I ran in to this problem before and reported it but it got closed because I couldn't replicate. Well, here its replicated again. What part of this could have permanently broken (until a new image is loaded) the camera?

@jc-kynesim
Copy link
Contributor

jc-kynesim commented Sep 16, 2022

a) I'm pretty sure that the BUILD.txt I pointed you at made it clear that you needed to get rid of the previous version and copy the result if you wanted it globally accessible.
b) 4.3.4 is the version used by PiOS and in the branch I pointed you at
c) Dunno about the camera issue - maybe its been grabbed by libcamera in some way? not directly my area.

@JooJooBee666
Copy link
Author

JooJooBee666 commented Sep 16, 2022

Yeah, I didn't finish reading the build.txt but noted that afterward.

Regarding the version number, I was just trying to clarify since that is also the version number on your repo but I confirmed that that version is also broken still.

Regarding the camera, I wasn't expecting you to know, but was hoping maybe something would make sense when you heard it. I'm just reloading.....AGAIN 😠

@ViktuK359
Copy link

Fix now pushed to my ffmpeg repo (https://github.com/jc-kynesim/rpi-ffmpeg.git branch test/4.3.4/rpi_main if you want to build it yourself). It'll be in the next ffmpeg update. Also fixes the same issue in mkv generation.

And how update for RPi4 HAOS?

@pelwell
Copy link
Contributor

pelwell commented Sep 28, 2022

That's a question for the HAOS devs, surely?

@roger-
Copy link

roger- commented Oct 28, 2022

Does anyone have a binary with this fix incorporated?

@hackneyb
Copy link

Fix now pushed to my ffmpeg repo (https://github.com/jc-kynesim/rpi-ffmpeg.git branch test/4.3.4/rpi_main if you want to build it yourself). It'll be in the next ffmpeg update.

@jc-kynesim Would you have an estimate on how long these releases take and which version will contain this fix? My path to have this working is wait for the ffmpeg update and then for HAOS to uptake that ... I'd appreciate you shedding some light on this for me. Thanks

@popcornmix
Copy link
Collaborator

popcornmix commented Nov 20, 2022

There was a recent ffmpeg update in RPiOS (timestamp shows it was built on Nov 7).

@LaurenceGough
Copy link

LaurenceGough commented Nov 21, 2022

Is there any way to get that ffmpeg update installed using apt on a Pi? I tried installing the deb files manually but it's one dependency after another missing and it was getting a bit crazy, obviously this was what apt was made for.

apt sources is the standard: deb https://archive.raspberrypi.org/debian/ bullseye main

Many thanks

Edit:

I managed it somehow (7:4.3.5-0+deb11u1+rpt2)
but the same issue remains.

@jherby2k
Copy link

Fix now pushed to my ffmpeg repo (https://github.com/jc-kynesim/rpi-ffmpeg.git branch test/4.3.4/rpi_main if you want to build it yourself). It'll be in the next ffmpeg update. Also fixes the same issue in mkv generation.

@jc-kynesim This does not appear to have trickled up, as nightly builds still don't include these patches. They also target a very old version of ffmpeg. Any update for us?

@mantaalex
Copy link

still doesn't work for some people @jc-kynesim do you have some news? thanks on advance

@jc-kynesim
Copy link
Contributor

When you say 'still does not work for some people' - does that imply that it does work for others?
Assuming so, what are the differences between those it does work for and those it doesn't? I'll happily debug but I need an actual failing example on PiOS.
The initial example given in this thread works for me with the current PiOS distribution ffmpeg.

@hackneyb
Copy link

hackneyb commented Jan 6, 2023

When you say 'still does not work for some people' - does that imply that it does work for others? Assuming so, what are the differences between those it does work for and those it doesn't? I'll happily debug but I need an actual failing example on PiOS. The initial example given in this thread works for me with the current PiOS distribution ffmpeg.

Not a developer, but there are extensive threads detailing what doesn't work (here: blakeblackshear/frigate#3780) and an official bug fix request at the ffmpeg page here: https://trac.ffmpeg.org/ticket/10060. The specific case that locks "some people" out of using acceleration is an RPi4 64 bit scenario with ffmpeg packaged from your repository as part of Frigate. Frigate runs as an add-on (in a container) in Home Assistant OS with the affected ffmpeg version (4.3.3?) and acceleration used by cameras does not work. Add-on container images can be manually patched when they launch to fix but this is temporary until the next restart and beyond the hobbyist / retail user (me) capabilities. I'm sure I could pull it off if I dedicated an evening to it but it's a band-aid. Since Home Assistant can be ran in multiple configurations, only the typical HAOS + add-on install mode is not permanently fixable, as far as I know. If you run HA on non-restricted linux distribution then ffmpeg that Frigate uses can be patched easier, if you know what you're doing.

More info here, with many messy examples of what works and what doesn't: blakeblackshear/frigate#3780

@jherby2k has a good handle on where it is reproducible and where it isn't, and took the initiative to open the ffmpeg request which in theory will trickle down to your project which will fix the ffmpeg version the Frigate project ultimately uses downstream. I think the last idea here was if you can help influence the ffmpeg project bug fix request, or debug and fix it in parallel in your project. The definition of the issue is obtainable, and I can help get you the info on what I'm running to help, but the Frigate project link above is teeming with examples.

Thanks @jc-kynesim

@6by9
Copy link
Contributor

6by9 commented Jan 6, 2023

Is that FFmpeg bug report valid for this issue? It's referencing DECODE, whereas this is ENCODE.

Accurate bug reports are critical if you want developers to investigate them.

@hackneyb
Copy link

hackneyb commented Jan 7, 2023

Is that FFmpeg bug report valid for this issue? It's referencing DECODE, whereas this is ENCODE.

Accurate bug reports are critical if you want developers to investigate them.

I get what you're saying, it would be hilarious if it was the reverse. Though I believe this issue (5150) describes the fix enough for people to take the solution, apply it to their custom compiled versions of ffmpeg and restore the functionality in Frigate. Frigate uses ffmpeg to decode https://docs.frigate.video/configuration/hardware_acceleration but maybe its not the act of decoding that's broken, but some other common requirement for encoding/decoding to take place. I could be totally wrong (disclaimer).

Does this provide more of a clue why people think this issue for encoding is linked with accelerated decoding issue in Frigate? jc-kynesim/rpi-ffmpeg#53 (comment)

@6by9
Copy link
Contributor

6by9 commented Jan 7, 2023

#5150 (comment)

I now understand what is going on - the mp4 muxer appears to require the SPS/PPS extradata at init time if it is going to take it from the context, but the h/w encoder won't provide it before it has been given its first frame (so after that). However it can also be passed along attached to the encoded packet as sidedata so I'll fix the encoder to do that and fix the mp4 muxer to respect it.

#5150 (comment)

Just ffmpeg - it all works fine if you have a raw h264 stream as the output type from ffmpeg - it is just separating off SPS/PPS for the mp4 muxer that has issues and that is all internal to ffmpeg (I do make the nasty assumption that SPS/PPS turns up on its own in the 1st capture frame which isn't a part of the V4L2 spec but I recall you saying was always true for our encoder).

#5150 (comment)

Fix now pushed to my ffmpeg repo (https://github.com/jc-kynesim/rpi-ffmpeg.git branch test/4.3.4/rpi_main if you want to build it yourself). It'll be in the next ffmpeg update. Also fixes the same issue in mkv generation.

That's the sum total of the fix for this issue related to encoding videos.

If using https://github.com/jc-kynesim/rpi-ffmpeg.git also fixes something else, then great, but it is not related to this issue regardless of what others think.
Telling the FFmpeg devs to look at this issue will not help them solve any decode related issue.

@blakeblackshear
Copy link

As the frigate maintainer, I can confirm that this issue is not relevant.

@hackneyb
Copy link

hackneyb commented Jan 7, 2023

If using https://github.com/jc-kynesim/rpi-ffmpeg.git also fixes something else, then great, but it is not related to this issue regardless of what others think. Telling the FFmpeg devs to look at this issue will not help them solve any decode related issue.

Thanks for taking the time to do this, then. Clarity is good.

@hackneyb
Copy link

hackneyb commented Jan 7, 2023

As the frigate maintainer, I can confirm that this issue is not relevant.

thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests