What is jc-kynesim/rpi-ffmpeg? Which changes were made from upstream ffmpeg? What to expect when using it?

[Abdull]

I stumbled upon this repository via https://raspberrypi.stackexchange.com/a/139512/12183 .

Now I'm curious what kind of changes where made compared to upstream https://github.com/FFmpeg/FFmpeg / https://git.ffmpeg.org/ffmpeg.git , if any.

Like, are there additions to leverage hardware features available on Raspberry Pi (1, 2, 3, 4, 5) in order to speed up decoding and encoding?

And how much speedup can be expected from this ffmpeg version compared to a generic version and compared to a Raspian Pi OS-provided ffmpeg version? So far, any other "custom Raspberry Pi ffmpeg" version with supposedly raspi-based hardware acceleration I tried, I didn't notice any kind of speed up, including when instructing ffmpeg to use the *_v4l2m2m codec variaties.

[Abdull]

I found general and build-specific notes on the test/5.1.4/main branch within the pi-util/ directory.

For instance, pi-util/NOTES.txt talks about the hevc_rpi decoder. Though I don't know if this comprises the only changes/additions to the original ffmpeg codebase.

[chewi]

I'm no expert, but in short: Vanilla FFmpeg can decode in hardware on the Pi already, but it cannot efficiently pass the decoded video to the display hardware where you can see it, so you can end up with a lot of stuttering. The changes here primarily address that and make some other playback-related optimisations. The Pi could previously do efficient playback, but that relied on the legacy software stack that is no longer provided by Raspberry Pi OS.

The hevc_pi decoder was an experiment to really push older Pis beyond what was previously thought possible with HEVC playback. I believe it has been abandoned, as it more of a hack than a sustainable solution.

Sorry if any of this is inaccurate.

[jc-kynesim]

OK. As stated above the original bit of work I did was the rpi_hevc decoder which enabled a Pi3 to decode HD H265 at up to ~20Mbits. This relied on OS interfaces to get at the QPU & GPU and for efficient fine-grained cache manipulation which went away in later PiOSs as it became more standardised. That code can still be found in the history here but it is no longer viable.
I've modified the existing V4L2 stateful decoder s.t. it works better with the Pi 0-4 H264 firmware decode.
I've written (or more accurate taken some existing code from the Kodi project and modified it out of almost all recognition - but the starting point should be acknowledged) a stateless V4L2 decode that works with the Pi4-5 H265 decode
There's V4L2 stateful deinterlace & scale
There are a few tweaks to let DRM prime (i.e. h/w) frames pass unmolested through the command line ffmpeg pipeline
Code to deal with the joy that is SAND formatted frames as emitted by the Pi h/w
A few random tweaks
Does that help?

[longo92-vrmedia]

@jc-kynesim when you say "I've modified the existing V4L2 stateful decoder s.t. it works better with the Pi 0-4 H264 firmware decode." can you point out more details? Can you give me a brief difference from vannilla ffmpeg? Thank you!