Question: Render faster than realtime?

[dotarmin]

Hi,

this is just a plain question f.y.i. Is it possible in some way to render PNG's faster than realtime somehow using this package?

Currently when I run at 50fps, 1920x1080 for 5 seconds it takes about 90 seconds to generate the screens.

Thanks in advance!

[tungs]

In theory, if given the right hardware and the right settings, it should be possible to render faster than realtime. The package overwrites time handling code of a web page, making it animate independent of realtime.

In practice, however, I have yet to see it happen, though I have only run it on low-end hardware. The speed of capture is limited by puppeteer, which currently runs Chromium/Chrome in headless mode (which limits hardware acceleration), and saves each frame to a PNG (which can be time consuming).

I'll look into having better hardware acceleration support in puppeteer and enabling it in timesnap.

[dotarmin]

@tungs, thanks for your quick response.

That's how much I also figured out but needed some sort of confirmation from you. The hardware I have been using (or my team more correctly) was first a Macbook Pro using latest Intel i9 (2 seconds slower on my i7 Macbook Pro) with six cores (12 in practice). We did try to send various settings to puppeteer (headless mode) but it still took around 90 seconds for 250 frames to be rendered and written to disc.

We then tried same approach on a Dell XPS 9550 with Windows 10 LTSB and it was even slower. I can try on a Supermicro and/or a HP Z4 with a Intel Xeon CPU. In this case the GPU is not that relevant due to the lack of hardware acceleration in puppeteer, right?

I'm really interested of using your lib in a bigger project and I (and my team) would love to contribute back to your project.

[tungs]

Thanks for the interest and response @dotarmin! Contributions are welcome, and it's quite helpful to know about performance on a variety of hardware. As it is out-of-the-box, I'd expect the lack of a GPU to not hinder performance on timesnap, because I don't think puppeteer uses the GPU by default (which you can check by using the url chrome://gpu).

There might be some flag to enable GPU acceleration (e.g. --use-gl=desktop), or to run with headless mode off, but I don't have the hardware or a lot of a time to test it out. It might also be more performant to use Chrome instead of Chromium, using the executablePath configuration. Disabling headless mode and executablePath options aren't currently configuration options for timesnap, but can be added easily, and I'll add it on the next release.

I suspect that a significant bottleneck is saving each frame as a png. I'm curious to know how fast puppeteer can take continuous screenshots of a static image. I think that should give a baseline for performance (though there may be some additional ways to squeeze performance out, like switching to lower quality jpgs or using a ramdisk or not saving images to the disk).

This bug on OS X, may also be a contributing factor, though it sounds like it is solved by disabling headless mode or running with a specific flag.

Some more relevant links:
puppeteer/puppeteer#3637
https://bugs.chromium.org/p/chromium/issues/detail?id=765284#c19

[tungs]

Version 0.0.6 introduces canvas capture mode. For some test cases, it captured faster than real-time. This mode is restricted to capturing only one canvas's data (so it doesn't capture anything in front or behind the canvas, including the document background).

Regardless, the performance disparity between canvas capture mode and screenshot capture mode makes me think that puppeteer's current default screenshot API has some inherent bottleneck that prevents it from capturing screenshots faster than a certain rate.

[ahmic]

Any updates on this?

Capturing 5 seconds video, 30 FPS takes

• 40 seconds using default screenshot png mode
• 5 seconds using canvas capture mode

Is there any way to speed up standard screenshot mode since I can not use canvas in my project?

EDIT: I'm using this in timecut, so refering to video length..

[tungs]

As it is by default, I think there is a bottleneck in puppeteer's Chromium/Chrome instance when it comes to taking screenshots that makes it slower than real time in most practical cases. I haven't yet discovered a Chromimum/Chrome launch flag that makes this faster.

If there are people really interested in exploring this issue, they can look into some questions that I don't have the time for:

• How does Puppeteer/Chromium capture screenshots? What's the process and the code?
• Does windowed mode (nonheadless puppeteer) speed up screenshots?
• Does a faster monitor refresh rate speed up screenshots?

Alternatively, puppeteer is aiming to be a general interface for multiple browsers that support the DevTools protocol. Projects like puppeteer-firefox that use Firefox instead of Chromium may have better performance. I haven't had the time to test it out yet.

[rana01645]

@dotarmin
Did you get any solution in this issue! It will be great if you let us know.

Thanks in advance

[dotarmin]

Hi @rana01645

Did you get any solution in this issue!

Unfortunately not, we did try to pass various flags and combinations to puppeteer and really juicy hardware without any success. The lack of hardware acceleration (even if some threads says it should work on Windows) it didn't work for us.

We use it as is and to render about 5 seconds 1920x1080 in 50fps takes about 180 seconds. We will however try to find another rendering mechanism to solve this, we're not there right now.

Is anybody interested to collaborate regarding this question? I could setup a remote meeting with some developers from my team for example and try to talk about the issue?

Best regards,
Armin

[Asheboy]

@rana01645 @dotarmin I've managed to increase the throughput by changing the type of screenshot to jpeg with no obvious drop in quality. See https://pptr.dev/#?product=Puppeteer&version=v2.0.0&show=api-pagescreenshotoptions.

I don't believe this is currently supported by timesnap, but you can easily make the change here https://github.com/tungs/timesnap/blob/master/lib/capture-screenshot.js#L98.

[Asheboy]

PR open to add the functionality I mentioned above: #29. I'm able to record 1920x1080, 50fps, 5s duration in ~43s using jpeg (quality 80).

[dotarmin]

@tungs, we have done a PR #35 to improve rendering time by using multiple instances of timesnap. Can someone review for feedback and eventually merge it if it looks good?

Thanks.

[Vinlic]

Perhaps my latest open-source project, WebVideoCreator, addresses this issue. Would you like to give it a try?