Non-deterministic WRITE_AFTER_PRESENT when multiple windows render and present in parallel

[knokko]

Environment:

• OS: Windows (also happens on Ubuntu)
• GPU and driver version: 3070 Ti driver version 561.9.0.0
• SDK or header version if building from repo: 1.3.290
• Options enabled (synchronization, best practices, etc.): synchronization

Describe the Issue

I have a test application that manages 2 (or more) windows in parallel. Each window has its own thread where it records command buffers, submits them, and presents. This causes the validation layer to sometimes spit out the following two errors:

For the 'fill' window (which only fills the window with the color pink):

SYNC-HAZARD-WRITE-AFTER-PRESENT(ERROR / SPEC): msgNum: 1123218669 - Validation Error: [ SYNC-HAZARD-WRITE-AFTER-PRESENT ] Object 0: handle = 0x23aa750c810, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x42f2f4ed | vkQueueSubmit():  Hazard WRITE_AFTER_PRESENT for entry 0, VkCommandBuffer 0x23aa77fad90[FillWindowLoopBuffer1], Submitted access info (submitted_usage: SYNC_IMAGE_LAYOUT_TRANSITION, command: vkCmdPipelineBarrier, seq_no: 1, reset_no: 16). Access info (prior_usage: SYNC_PRESENT_ENGINE_SYNCVAL_PRESENT_PRESENTED_SYNCVAL, write_barriers: 0, queue: VkQueue 0x23aa750c810[], submit: 41, batch: 0, batch_tag: 140, vkQueuePresentKHR present_tag:140, pSwapchains[0]: VkSwapchainKHR 0xcad092000000000d[Swapchain-FillWindow1], image_index: 0image: VkImage 0x967dd1000000000e[SwapchainImage-FillWindow0]).
    Objects: 1
        [0] 0x23aa750c810, type: 4, name: NULL

For the 'spin' window (which renders a spinning triangle):

SYNC-HAZARD-WRITE-AFTER-PRESENT(ERROR / SPEC): msgNum: 1123218669 - Validation Error: [ SYNC-HAZARD-WRITE-AFTER-PRESENT ] Object 0: handle = 0x23aa750c810, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x42f2f4ed | vkQueueSubmit():  Hazard WRITE_AFTER_PRESENT for entry 0, VkCommandBuffer 0x23aa7765e80[SpinWindowLoopBuffer0], Submitted access info (submitted_usage: SYNC_IMAGE_LAYOUT_TRANSITION, command: vkCmdPipelineBarrier, seq_no: 1, reset_no: 20). Access info (prior_usage: SYNC_PRESENT_ENGINE_SYNCVAL_PRESENT_PRESENTED_SYNCVAL, write_barriers: 0, queue: VkQueue 0x23aa750c810[], submit: 40, batch: 0, batch_tag: 139, vkQueuePresentKHR present_tag:139, pSwapchains[0]: VkSwapchainKHR 0xd175b40000000013[Swapchain-SpinWindow1], image_index: 0image: VkImage 0x9fde6b0000000014[SwapchainImage-SpinWindow0]).
    Objects: 1
        [0] 0x23aa750c810, type: 4, name: NULL

The weirdest part is that the errors stop coming as soon as I close 1 of the windows. I think this is a bug because the windows operate independently (and therefor the appearance of validation errors should not depend on whether there is another window):

• they have their own swapchain
• they have their own fences and semaphores
• they have their own command pools

They do share a queue, but it's accessed under mutual exclusion. (And if I remove this mutual exclusion, it reports a completely justified UNASSIGNED-Threading-MultipleThreads-Write.) Also, when they use separate queues, these weird validation errors also appear.

I attached an api dump of my application as file to this issue. You can find the validation errors by searching "WRITE_AFTER_PRESENT". The api dump clearly shows that some vkQueueSubmit calls cause a validation error, but most of them do not. I have not yet managed to figure out why some sequences of calls in the dump lead to validation errors, but others do not.

Since this is a pretty complex issue, it would be an enormous amount of work to make a minimal reproducible example. I can however share my (test) application that causes the errors: https://github.com/knokko/vk-boiler/tree/potential-validation-bug
It unfortunately requires Java (21) to run, but this can easily be installed on Ubuntu with sudo apt install openjdk-21-jdk. The README shows the instructions to run the (pre-built) program.

The problem generates plenty of validation errors per second on both of my computers, but I can't be absolutely sure that this will also happen on other computers.

Expected behavior

• If my application is bugged (I don't think so, but I'm never sure), the validation errors should be consistent and deterministic
• If my application is correct, no validation errors should be reported

dump.txt