[BUG]: Starting a VR setup before starting stream breaks stream

[EricXu1728]

Package version

3.1.0-exp.8

Environment

* OS: Windows 11
* Unity version: 2022.3.23f1
* Graphics API: DX12
* Browser: Google Chrome Version 125.0.6422.141 (Official Build) (64-bit)

Steps To Reproduce

1. Download and open Unity VR core
2. Follow the "Creating the Scene" instructions to set up streaming (https://docs.unity3d.com/Packages/[email protected]/manual/create-scene.html). Put the broadcaster in a separate camera from the VR setup
3. Put on the VR headset BEFORE you hit play on the locally hosted webpage. This should "wake up" the VR set up.

Current Behavior

The stream is black and multiple errors appear in console. I was unable to replicate the issue with normal cameras.

Logs:
Signaling: Connecting WS ws://127.0.0.1
UnityEngine.Logger:Log (object)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSCreate () (at ./Library/PackageCache/[email protected]/Runtime/Scripts/Signaling/WebSocketSignaling.cs:187)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSManage () (at ./Library/PackageCache/[email protected]/Runtime/Scripts/Signaling/WebSocketSignaling.cs:153)
System.Threading.ThreadHelper:ThreadStart ()

Signaling: WS connected.
UnityEngine.Logger:Log (object)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSConnected (object,System.EventArgs) (at ./Library/PackageCache/[email protected]/Runtime/Scripts/Signaling/WebSocketSignaling.cs:265)
WebSocketSharp.Ext:Emit (System.EventHandler,object,System.EventArgs)
WebSocketSharp.WebSocket:open ()
WebSocketSharp.WebSocket/<>c__DisplayClass201_0:b__0 (System.IAsyncResult)
System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()

Signaling: Receiving message: {"from":"8c07cb5d-fdab-4599-aba9-f906b7506d08","to":"","type":"offer","data":{"sdp":"v=0\r\no=- 7846434483137186184 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:5ybb\r\na=ice-pwd:EDQgn7DfLLwALF/QA4AnH2eU\r\na=ice-options:trickle\r\na=fingerprint:sha-256 78:3C:96:94:13:49:30:B9:05:A8:A2:6E:82:99:5E:72:30:51:C1:9B:C8:FE:51:92:75:2D:4A:F6:C5:C9:CB:8C\r\na=setup:actpass\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n","datetime":1717531019119,"polite":false}}
UnityEngine.Logger:Log (object)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSProcessMessage (object,WebSocketSharp.MessageEventArgs) (at ./Library/PackageCache/[email protected]/Runtime/Scripts/Signaling/WebSocketSignaling.cs:194)
WebSocketSharp.Ext:Emit<WebSocketSharp.MessageEventArgs> (System.EventHandler`1<WebSocketSharp.MessageEventArgs>,object,WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:messagec (WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:message ()
WebSocketSharp.WebSocket/<>c__DisplayClass174_0:b__1 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass84_0:b__3 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass77_0:b__0 (byte[])
WebSocketSharp.Ext/<>c__DisplayClass60_0:b__0 (System.IAsyncResult)
System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()

Signaling: Receiving message: {"from":"8c07cb5d-fdab-4599-aba9-f906b7506d08","to":"","type":"candidate","data":{"candidate":"candidate:2823404550 1 udp 2113937151 79e18b59-0f3b-4f02-900b-f37c03ec2013.local 62495 typ host generation 0 ufrag 5ybb network-cost 999","sdpMLineIndex":"0","sdpMid":0,"datetime":1717531019120}}
UnityEngine.Logger:Log (object)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSProcessMessage (object,WebSocketSharp.MessageEventArgs) (at ./Library/PackageCache/[email protected]/Runtime/Scripts/Signaling/WebSocketSignaling.cs:194)
WebSocketSharp.Ext:Emit<WebSocketSharp.MessageEventArgs> (System.EventHandler`1<WebSocketSharp.MessageEventArgs>,object,WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:messagec (WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:message ()
WebSocketSharp.WebSocket/<>c__DisplayClass174_0:b__1 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass84_0:b__3 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass77_0:b__0 (byte[])
WebSocketSharp.Ext/<>c__DisplayClass60_0:b__0 (System.IAsyncResult)
System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()

[impolite-PeerConnection ConnectionState:New IceConnectionState:New SignalingState:Stable GatheringState:New] this candidate can't accept on state.
UnityEngine.Logger:Log (UnityEngine.LogType,object)
Unity.RenderStreaming.PeerConnection:OnGotIceCandidate (Unity.WebRTC.RTCIceCandidate) (at ./Library/PackageCache/[email protected]/Runtime/Scripts/PeerConnection.cs:295)
Unity.RenderStreaming.SignalingManagerInternal:OnIceCandidate (Unity.RenderStreaming.Signaling.ISignaling,Unity.RenderStreaming.CandidateData) (at ./Library/PackageCache/[email protected]/Runtime/Scripts/SignalingManagerInternal.cs:413)
Unity.RenderStreaming.Signaling.WebSocketSignaling/<>c__DisplayClass39_3:b__4 (object) (at ./Library/PackageCache/[email protected]/Runtime/Scripts/Signaling/WebSocketSignaling.cs:248)
UnityEngine.UnitySynchronizationContext:ExecuteTasks ()

Signaling: Receiving message: {"from":"8c07cb5d-fdab-4599-aba9-f906b7506d08","to":"","type":"candidate","data":{"candidate":"candidate:1661771167 1 udp 1677729535 71.121.236.94 62495 typ srflx raddr 0.0.0.0 rport 0 generation 0 ufrag 5ybb network-cost 999","sdpMLineIndex":"0","sdpMid":0,"datetime":1717531019162}}
UnityEngine.Logger:Log (object)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSProcessMessage (object,WebSocketSharp.MessageEventArgs) (at ./Library/PackageCache/[email protected]/Runtime/Scripts/Signaling/WebSocketSignaling.cs:194)
WebSocketSharp.Ext:Emit<WebSocketSharp.MessageEventArgs> (System.EventHandler`1<WebSocketSharp.MessageEventArgs>,object,WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:messagec (WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:message ()
WebSocketSharp.WebSocket/<>c__DisplayClass174_0:b__1 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass84_0:b__3 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass77_0:b__0 (byte[])
WebSocketSharp.Ext/<>c__DisplayClass60_0:b__0 (System.IAsyncResult)
System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()

ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
System.Collections.Generic.List1[T].get_Item (System.Int32 index) (at <b11ba2a8fbf24f219f7cc98532a11304>:0) UnityEngine.Experimental.Rendering.XRPass.GetViewport (System.Int32 viewIndex) (at ./Library/PackageCache/[email protected]/Runtime/XR/XRPass.cs:195) UnityEngine.Rendering.Universal.Internal.DrawObjectsPass.ExecutePass (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.Internal.DrawObjectsPass+PassData data, UnityEngine.Rendering.Universal.RenderingData& renderingData, System.Boolean yFlip) (at ./Library/PackageCache/[email protected]/Runtime/Passes/DrawObjectsPass.cs:186) UnityEngine.Rendering.Universal.Internal.DrawObjectsPass.Execute (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.RenderingData& renderingData) (at ./Library/PackageCache/[email protected]/Runtime/Passes/DrawObjectsPass.cs:156) UnityEngine.Rendering.Universal.ScriptableRenderer.ExecuteRenderPass (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.ScriptableRenderPass renderPass, UnityEngine.Rendering.Universal.RenderingData& renderingData) (at ./Library/PackageCache/[email protected]/Runtime/ScriptableRenderer.cs:1507) UnityEngine.Rendering.Universal.ScriptableRenderer.ExecuteBlock (System.Int32 blockIndex, UnityEngine.Rendering.Universal.ScriptableRenderer+RenderBlocks& renderBlocks, UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.RenderingData& renderingData, System.Boolean submit) (at ./Library/PackageCache/[email protected]/Runtime/ScriptableRenderer.cs:1463) UnityEngine.Rendering.Universal.ScriptableRenderer.Execute (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.RenderingData& renderingData) (at ./Library/PackageCache/[email protected]/Runtime/ScriptableRenderer.cs:1223) UnityEngine.Rendering.Universal.UniversalRenderPipeline.RenderSingleCamera (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.CameraData& cameraData) (at ./Library/PackageCache/[email protected]/Runtime/UniversalRenderPipeline.cs:664) UnityEngine.Rendering.Universal.UniversalRenderPipeline.RenderCameraStack (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Camera baseCamera) (at ./Library/PackageCache/[email protected]/Runtime/UniversalRenderPipeline.cs:848) UnityEngine.Rendering.Universal.UniversalRenderPipeline.Render (UnityEngine.Rendering.ScriptableRenderContext renderContext, System.Collections.Generic.List1[T] cameras) (at ./Library/PackageCache/[email protected]/Runtime/UniversalRenderPipeline.cs:370)
UnityEngine.Rendering.RenderPipeline.InternalRender (UnityEngine.Rendering.ScriptableRenderContext context, System.Collections.Generic.List`1[T] cameras) (at :0)
UnityEngine.Rendering.RenderPipelineManager.DoRenderLoop_Internal (UnityEngine.Rendering.RenderPipelineAsset pipe, System.IntPtr loopPtr, UnityEngine.Object renderRequest, Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle safety) (at :0)

Expected Behavior

Webpage should properly stream camera. This functions normally if you hit play BEFORE putting on the VR headset. Apologies for not being able to track down the source of the issue.

Anything else?

No response

[EricXu1728]

Did more digging. Seems to be an issue with VR not playing well with the existence of multiple cameras. Closing for now.

[EricXu1728]

Woops, turns out this is an issue with Unity Render Streaming. I just messed up reproducing the issue.