how does v4l2display_h264 work?

[WiredLife]

i tried v4l2display_h264 -vv -W 480 -H 320 -F 15 /dev/video1
hoping that shows me the video on framebuffer and display, but it dont
it only says waiting

[WiredLife]

v4l2grab_h264 :
grab raspberry pi screen, compress in H264 format using OMX and write to a V4L2 output device

v4l2display_h264 :
read H264 from V4L2 capture device, uncompress and display using OMX

I need to display the video1 raw h264 on my display

I've run a v4l2rtspserver that shares video0 and clone to video1 (v4l2loopback)

[mpromonet]

Hi,

v4l2display_h264 read H264 frames from a V4L2 device and display using OMX.

In order to works, it is important that the V4L2 device repeat the SPS/PPS.
It just makes somehow what does ffplay /dev/video1 with many less feature, and only with an H264 data.

Best Regards,
Michel.

[WiredLife]

@mpromonet

i'm broadcasting the video from raspberry cam (video0) to rtsp and want the direct video on my 3,5" spi tft which reads framebuffer
so i did that

modprobe v4l2loopback devices=5
v4l2rtspserver -O /dev/video1 -H 1232 -W 1640 -F 15 -P 8555 /dev/video0

and now i want to display but this dont work
v4l2display_h264 -vv -W 480 -H 320 -F 15 /dev/video1
gives only
waiting...
first packet
waiting...
waiting...
and so on...

v4l2-ctl --all -d /dev/video1
gives me this
Driver Info:
Driver name : v4l2 loopback
Card type : Dummy video device (0x0000)
Bus info : platform:v4l2loopback-000
Driver version : 4.19.75
Capabilities : 0x85208003
Video Capture
Video Output
Video Memory-to-Memory
Read/Write
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x85208003
Video Capture
Video Output
Video Memory-to-Memory
Read/Write
Streaming
Extended Pix Format
Device Capabilities
Priority: 0
Video input : 0 (loopback: ok)
Video output: 0 (loopback in)
Format Video Capture:
Width/Height : 1640/1232
Pixel Format : 'H264' (H.264)
Field : None
Bytes per Line : 0
Size Image : 8081920
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Full Range)
Flags :
Format Video Output:
Width/Height : 1640/1232
Pixel Format : 'H264' (H.264)
Field : None
Bytes per Line : 0
Size Image : 8081920
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Full Range)
Flags :
Streaming Parameters Video Capture:
Frames per second: 15.000 (15/1)
Read buffers : 8
Streaming Parameters Video Output:
Frames per second: 15.000 (15/1)
Write buffers : 8

User Controls

                keep_format 0x0098f900 (bool)   : default=0 value=0
          sustain_framerate 0x0098f901 (bool)   : default=0 value=0
                    timeout 0x0098f902 (int)    : min=0 max=100000 step=1 default=0 value=0
           timeout_image_io 0x0098f903 (bool)   : default=0 value=0

[mpromonet]

@WiredLife,

The OMX will not be display on the framebuffer, in order to copy screen to framebuffer you could look at https://github.com/tasanakorn/rpi-fbcp or get the decoded frame and write to the framebuffer.

Best Regards,

Michel.

[aweussom]

@WiredLife,

The OMX will not be display on the framebuffer, in order to copy screen to framebuffer you could look at https://github.com/tasanakorn/rpi-fbcp or get the decoded frame and write to the framebuffer.

If it is a chinese SPI display clone, neither v4l2display_h264 or rpi-fbcp will work?
ffplay should, though. Or OMXplayer, perhaps.

For my setup, I streamed rtsp to the PI browser.

[WiredLife]

@mpromonet
@aweussom
i'm using fbcp-ili9341 driver which copy fb0 to fb1
i can see video with cvlc -vvv rtsp://127.0.0.1:8555/unicast --network-caching=0 --fbdev=/dev/fb0
but this sucks because of delay and sometimes pixelating
opening video1 does not work, maybe there is something missing?

i need to view video1 directly

edit:
mplayer says this
Error while decoding frame!
[h264 @ 0xb5d9b118]non-existing PPS 0 referenced
[h264 @ 0xb5d9b118]decode_slice_header error
[h264 @ 0xb5d9b118]no frame!

i dont know much about that
maybe this is not working? v4l2rtspserver -O /dev/video1 -H 1232 -W 1640 -F 15 -P 8555 /dev/video0

[WiredLife]

i've found a solution by @mpromonet after some google
v4l2-ctl -c repeat_sequence_header=1
and then i can do
v4l2display_h264 -vv -W 480 -H 320 -F 15 /dev/video1
this gives me 30% disturbed picture in the lower so i'm using
cvlc -vvv v4l2:///dev/video1 --noaudio --fbdev=/dev/fb0
which works fine

maybe someone knows the parameter to disable buffering?
want near 0ms delay

[mpromonet]

Hi @WiredLife

Yes in my first answer it is pointed out that SPS/PPS should be present, otherwise v4l2display_h264 or vlc cannot decode the H264 stream.

There is no buffering in v4l2display_h264 and a very small in v4l2rtspserver that is control by the parameter -Q.

Best Regards,
Michel.

[WiredLife]

Great its working with nearly no delay!

v4l2rtspserver -O /dev/video1 -Q 1 -H 1232 -W 1640 -F 15 -P 8555 /dev/video0
cvlc -vvv v4l2:///dev/video1 --live-caching=150 --file-caching=5 --sout-mux-caching=5 --noaudio --fbdev=/dev/fb0

150ms is the lowest value, under that its missing frames and becomes delayed too

[mpromonet]

Hi,

I think below -Q3 could brings some problem.
In my experiment reading from V4L2 an H264 stream give in a same call SPS/PPS/IDR, that will be splitted in 3 frames to the queue (because each one is send in a different RTP packet).
This will work for the copy to loopback but it might cause problem for RTP streaming ?

vlc seems an heavy solution to decode the stream and copy it to framebuffer, using OMX to decode the stream and writing to the framebuffer may be not so complex ?

Best Regards,
Michel.

[WiredLife]

vlc is yet the only solution that works
v4l2display_h264 has still disturbed image
mplayer and ffmpeg dont show any image on framebuffer here

[WiredLife]

are there any news on get this working without pixelated image?
i'm runinng
v4l2rtspserver -O /dev/video1 -Q 3 -H 1232 -W 1640 -F 15 -P 8555 /dev/video0,dsnoopedmic -C 1; done
and
v4l2display_h264 /dev/video1
but the image is still pixelating

[mpromonet]

Hi @WiredLife

Did you check your H264 source, it may be also pixelated ? (For example using v4l2display_h264 /dev/video0, without rtsp stremaner)
I just made a test, and I don't see pixelization, it depends on configuration of the v4L2 device.

Best Regards,
Michel