[TRACKER] OpenXR in Godot 4

[BastiaanOlij]

We've identified several outstanding issues with OpenXR. Just raising this issue to keep track of them.

Core features

• Auto exposure only works in left eye In XR auto exposure works only for a left eye #69000
• VRS being made available even when not supported Fix VRS object being created even when not supported #69091
• Query OpenXR for a density map once supported
• Render order issue XR: materials rendering in wrong order #68469 (this is likely caused by the depth buffer issues, see below)
• Only background is being rendered Nothing except for world environment renders to XR camera. #65864 (this is likely caused by depth buffer issues, see below)
• Action map save issue OpenXR Action Map loses unsaved changes easily #68041
• Fix various issues around the action map Various fixes for OpenXR action map meta data and editing #68528
• Fix sky orientation issue Fix mobile and gl_compatibility renderers sky_transform operations #69636
• Fix stereo issue in sky VR XR PanoramaSkyMaterial rendering problem #62738

Extensions

• Support implementing OpenXR extensions as GDExtensions Add GDExtension support for OpenXR extension wrappers #68259
• add support for various composition layers (as extra (2D) Viewports rendering to XRSwapChains)
• add support for Dpad bindings in Action map: https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#XR_EXT_dpad_binding
• Port performance settings extension from Godot 3 plugin: https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#XR_EXT_performance_settings
• Implement render models API once core version becomes available
• Implement Metas Scene capture extension: https://github.com/migeran/godot_openxr_meta
• Ensure interfaces avoid multiple inheritance of classes with data members OpenXRExtensionWrapperExtension: Avoid use of multiple inheritance with data members #69694

Depth buffer

OpenXR allows supplying the depth buffer, the XR runtime can use this information to perform more accurate re-projection when framerates are not met. On Quest the depth buffer can be combined with a motion vector buffer to enable application spacewarp, a more advanced version of re-projection. Motion vector support is not yet available on the mobile renderer but solving the issues already found will be paramount to solving this.

• Left eye depth buffer is supplied for the right eye Fix typo in OpenXR depth buffer submission #69648
• Mobile renderer is not resolving MSAA depth buffer to final depth buffer
• Clustered renderer fails to resolve MSAA depth buffer to final depth buffer if compute shaders are used, it's not possible to set the VK_IMAGE_USAGE_STORAGE_BIT
• Make depth buffer support optional so it can be turned off when effects are used that break the solution Make submitting depth buffer in OpenXR optional #69654

Vulkan specific

OpenGL specific

• Image flip issue when presenting preview image on desktop [opengl] Don't flip render target when blitting it to screen if it wasn't rendered upside down #69581

DX12 specific

• Implement multiview support
• Add DX12 extension for OpenXR

Misc

• WMR devices over SteamVR are not always working SteamVR crashes and desktop screen flashes (on HP Reverb G2?) when running VR project in beta 3 and later. Project works fine in Beta 2. #67398 (possible driver issue with MS)

[RadixSeven]

The two issues linked in the "Depth Buffer" section are marked as merged, and the "Submit Depth Buffer" checkbox exists in 4.0.1.stable. Should the "Depth Buffer" section of this issue be checked off? It looks like we can also check off #68469, #65864, and #62738 under "core features".

[BastiaanOlij]

@RadixSeven there were still a few issues with this. #78550 fixes a major issue, I'm still looking into a few resolve issues when MSAA is being used.