gpu-next: flicker with display-resample and interpolation

[christoph-heinrich]

Important Information

Provide following Information:

• mpv 7f75131 and libplacebo haasn/libplacebo@31082ee
• Arch linux
• compiled from source

I noticed that problem months ago with a file that had a frame with incorrect pts every few seconds, but didn't look into it any further because it usually doesn't cause any problems (also the video wasn't freely available, so providing a sample might have been problematic).
It came up again when testing #12028.
The whole image flackers occasionally (random intervals) when using opengl, while it only flickers around the border when using vulkan.
Doesn't happen with vo=gpu.
Might be related to #11519

Reproduction steps

1. Download https://github.com/ferreum/mpv-skipsilence/blob/master/skipsilence.lua
2. mpv --no-config --script=skipsilence.lua --script-opts=skipsilence-enabled=yes,skipsilence-ramp_constant=4 --vo=gpu-next --video-sync=display-resample --interpolation --gpu-api=opengl "https://www.youtube.com/watch?v=jpKrrchVRGE"

Expected behavior

Normal playback without flickering

Actual behavior

Opengl

flicker_full.mp4

Vulkan

flicker_edge.mp4

Log file

opengl log
vulkan log

Sample files

The video from the linked issue

[llyyr]

Render queue underrun.

Might be related to #12101 and 35a6b26 ?

[christoph-heinrich]

Render queue underrun.

Might be related to #12101 and 35a6b26 ?

It might be, but there was no mention of flickering in that issue.

[Myaanee]

Maybe related to #11465 as well?

[Obegg]

Render queue underrun.

Might be related to #12101 and 35a6b26 ?

It might be, but there was no mention of flickering in that issue.

Correct. When I had that issue - I did not experience flickering.

[llyyr]

This only happens with --tscale=mitchell --tscale-clamp=1 which were the old defaults, since we use oversample now and hermite is recommended over mitchell for blurrier interpolation this shouldn't be an issue anymore.

[christoph-heinrich]

Interesting observation and I can confirm that skipsilence.lua doesn't cause any problems with the current defaults anymore. There is still the occasional weird flicker similar to the edge flicker in the video above with videos that have mistimed frames on vulkan, which I assume is related.

It still happens with --tscale=mitchell --tscale-clamp=1 for vo=gpu-next but not on vo=gpu, so there is still something wrong.

[Dudemanguy]

Playing that video with those script settings causes a gazillion dropped frames for me. Is my computer just bad or something? Without skipsilence-ramp_constant=4, it still drops frames but a lot less. I can confirm that with --tscale=mitchell --tscale-clamp=1 and with skipsilence-ramp_constant=4 both vulkan and opengl have that flickering. Without skipsilence-ramp_constant=4, it doesn't flicker, likely because it doesn't drop so many frames. So to me this just seems like gpu-next doesn't quite handle interpolation when you're dropping a bunch of frames as elegantly? I assume we're probably giving libplacebo bad values which cause the flicker.

[christoph-heinrich]

Playing that video with those script settings causes a gazillion dropped frames for me. Is my computer just bad or something?

Are you on a 100Hz+ display? Because when fps * speed > display-fps it's normal for the dropped frames to go up.

But I have also noticed dropped frames on speed changes.
When fast forwarding with evafast playback is always a more choppy then it should be while speeding up and for a little bit afterwards. I've never looked into why that may be, but it's probably a frame timing thing.

[Dudemanguy]

Are you on a 100Hz+ display? Because when fps * speed > display-fps it's normal for the dropped frames to go up.

Nah just normal 60Hz. And yeah of course the --speed=2 setting drops frames, but that script with those settings specifically makes it drop an absurd amount and gives me the AV desync message. Is it just me?

[christoph-heinrich]

gives me the AV desync message. Is it just me?

I also get that one, so it's not just you. 🤷

[Dudemanguy]

Did a bit more testing and I think the root cause of this flicker is when you drop a bunch of frames in a short burst. The script with these settings happen to cause this on the video. If you watch the dropped frame counter, anytime there's a big jump in the values, it lines up with a flicker. On the other hand if you load up on comically heavy rendering settings that cause regular frame drops, the flickering artifact doesn't occur since there aren't any sudden bursts. So I don't believe this particular issue is related to anything else.

vo_gpu is more robust against this for whatever reason. I suspect vo_gpu_next could use some extra sanity checking somewhere to avoid weird edge cases like this.

[Dudemanguy]

Fixed by #12865.

[Dudemanguy]

And not actually fixed anymore. Previous commit "fixed" this by accident.

[christoph-heinrich]

Fixed by #12890