[d3d9,d3d11] recreate swapchain on VK_SUBOPTIMAL_KHR

[llyyr]

The vulkan wayland wsi returns suboptimal when the window is fullscreened and not directly scanned out, and there are modifiers available that would allow the window to be directly scanned out. Recreate the swapchain if we receive suboptimal result.

This allows us recreate the swapchain to use a modifier that allows direct scan-out under winewayland on wayland compositors.

I only managed to test it on a couple of games, so this doesn't have the greatest test coverage. I could put it behind a config option if this change is likely to break stuff.

[doitsujin]

Why does this change anything? We already recreate the swap chain if present returns SUBOPTIMAL_KHR.

If we need to do this on acquire as well for whatever reason then we should at least ensure that we're not turning the whole thing into an infinite loop if WSI can't give us what we want.

[llyyr]

We already recreate the swap chain if present returns SUBOPTIMAL_KHR.

Yeah I see it now. I'm not too familiar with dxvk but I'll look into why that isn't sufficient. Marking this as draft for now.

if WSI can't give us what we want.

I might be misunderstanding this comment, but from my understanding we're still picking the format but the WSI is responsible picking what modifier to use for the client. It would be a WSI bug if it picks a modifier but it's still suboptimal no?

[doitsujin]

Yeah I see it now. I'm not too familiar with dxvk but I'll look into why that isn't sufficient.

Looking at the mesa code it seems like present always returns VK_SUCCESS and ignores the swap chain's suboptimal status? Not sure if this is intended by the vk spec that return values ping-pong between SUBOPTIMAL and SUCCESS, but this would at least explain the issue.

We can definitely land this change in that case. Does this currently affect XWayland and/or Gamescope environments too?

It would be a WSI bug if it picks a modifier but it's still suboptimal no?

Arguably yes, but Wayland and X11 aren't the only WSI implementations we end up running on even if those are obviously the priority, and we've had weirder issues in the past. I'd prefer to be robust here and do something like if (status == SUBOPTIMAL) break; after acquiring from the recreated swap chain to only allow one iteration in that case.

[llyyr]

Not sure if this is intended by the vk spec that return values ping-pong between SUBOPTIMAL and SUCCESS, but this would at least explain the issue.

Is this worth reporting to mesa?

Does this currently affect XWayland and/or Gamescope environments too?

Not in my testing, no.

I'd prefer to be robust here and do something like if (status == SUBOPTIMAL) break; after acquiring from the recreated swap chain to only allow one iteration in that case.

Done. Thanks for the help, and also for identifying why this was necessary.

[doitsujin]

Is this worth reporting to mesa?

Would probably be good to clarify why SUBOPTIMAL isn't propagated, yes.

Either way, LGTM.

[llyyr]

Would probably be good to clarify why SUBOPTIMAL isn't propagated, yes.

I asked and it appears that QueuePresent doesn't propagate suboptimal on X11 either for performance reasons. https://gitlab.freedesktop.org/mesa/mesa/-/commit/2b885b233f7a6300ae88732c179888c02788493d

So both x11 and wayland WSI only return suboptimal on AcquireNextImage.