Video rendering support

[laurens-lamberts]

We currently don't have a specific use case for this, but I think it would be nice for react-native-skia to support video rendering. Especially as this could provide the possibility to apply blurs on top of video content, and make for example;

• fancy custom video controls
• full-screen looping video as a backdrop for an app with blurring components on top of it

Also, this could eliminate the need of adding an extra library such as react-native-video, when react-native-skia is already part of your project. This could be practical if you just want some basic video rendering.

Some related code files I found in the Skia source which might be helpful;
Skia video decoder
Skia video decoder ffmpeg

[wcandillon]

Thanks for sending us these pointers. This is indeed a hot topic.
Based on the examples, I'm wondering if there is a way to access the encoded frame via ffmpeg-kit-react-native and then there is maybe not much that would need to be done on our side. By looking at the API at https://github.com/tanersener/ffmpeg-kit/blob/main/react-native/src/index.d.ts it is not clearly yet how to do it. Maybe run the command to get the jpg/png and access it via the filesystem module?

[MadCoderme]

I would suggest and request native android developers to look into this and use android Mediacodec API for hardware accelerated actions instead of ffmpeg.

[BenCoDev]

I'd love to be able to create a video timeline scrub with canvas.

I made one with react native ffmpeg but it feels pretty over-engineered. See here

That'd be a great use case for me. If I can help somehow, is be happy to!

[XantreDev]

I'd love to be able to create a video timeline scrub with canvas.

I made one with react native ffmpeg but it feels pretty over-engineered. See here

That'd be a great use case for me. If I can help somehow, is be happy to!

This article is crazy)

[BenCoDev]

I'd love to be able to create a video timeline scrub with canvas.

I made one with react native ffmpeg but it feels pretty over-engineered. See here

That'd be a great use case for me. If I can help somehow, is be happy to!

This article is crazy)

Crazy good or crazy bad? 😝

[XantreDev]

I think crazy good, but I've performance concerns

[BenCoDev]

I think crazy good, but I've performance concerns

Thanks, you're too kind.
Yes, honestly, it is a bit goofy to use ffmpeg here. when I have some times, I'll do the same using canvas. It surely will be far more efficient.

[hungtooc]

It's been quite a while since this issue was opened, does anyone have any solution yet? @ALL
I need to render a video from Skia animation.
I just learned a little about ffmpeg-kit-react-native, it seems I have to save the entire image frame as a file and execute FFmpeg commands by providing image paths. ffmpeg-kit-test This seems suboptimal because there is only a slight change between frames. Hopefully Skia can support video rendering more effectively when the frames created by skia and reanimated3.

[marcusstenbeck]

I’m exploring options for overlays on videos or ideally using a video as a mask for something else. Has anyone been able to get a video surface into a format that can be consumed by RN Skia?

[XantreDev]

I've used animated images (webp). It looks like video, if preload all frames

[marcusstenbeck]

Thanks for the tip, but my use case requires video. I’m guessing makeimagefromview isn’t fast enough to run on every frame?

[XantreDev]

Don't know, maybe you should check it

[aphillipo]

I would love it if it was possible to render WebRTC video to a react-native-skia video tile as it would allow me to generate beautiful overlays that react to the underlying video - for example the designer I have came up with a graph that blurs video under it, easy to do with react-native-skia but extremely hard to do otherwise. Obviously this part of the work and a format that can be produced by WebRTC or whatever would have to be agreed on, would it just take the raw decoded frames somehow or accept an MP4/VP8 stream directly.

[wcandillon]

We are actively working on this now.

collab.mov

[mlecoq]

Can't wait to test it 😁