NullReferenceException in WindowsMixedRealityXRSDKArticulatedHand.UpdateHandData

[Holo-Krzysztof]

Describe the bug

There's a NRE happening on every frame where hands are visible, leading to severely degraded performance and visual glitches when hand mesh visualization is enabled.

To reproduce

Steps to reproduce the behavior:

1. Create new Unity project with 3D template
2. Import MRTK 2.8 examples via feature tool
3. In MRTK config window, select "builtin unity plugins (non-openxr)"
4. In XR plugin management enable WMR provider for UWP build target
5. In WMR settings check "shared depth buffer"
6. Skip next step in settings, then import TMP Essentials
7. Open package manager, select MRTK examples package and import "Demos - HandTracking"
8. Open HandInteractionExamples scene
9. Switch build target to UWP, target SDK 19041, VS 2022, add open scenes, then build the generated C++ project
10. Deploy to HoloLens 2 with ARM64/Release config and run the app
11. Observe jankiness when using hands
12. Look into UnityPlayer.log, which will contain lots of:

NullReferenceException: Object reference not set to an instance of an object.
  at Microsoft.MixedReality.Toolkit.XRSDK.WindowsMixedReality.WindowsMixedRealityXRSDKArticulatedHand.UpdateHandData (UnityEngine.XR.InputDevice inputDevice) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.MixedReality.Toolkit.XRSDK.WindowsMixedReality.WindowsMixedRealityXRSDKArticulatedHand.UpdateController (UnityEngine.XR.InputDevice inputDevice) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.MixedReality.Toolkit.XRSDK.Input.XRSDKDeviceManager.Update () [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.MixedReality.Toolkit.XRSDK.WindowsMixedReality.WindowsMixedRealityDeviceManager.Update () [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.MixedReality.Toolkit.BaseDataProviderAccessCoreSystem.Update () [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.MixedReality.Toolkit.MixedRealityToolkit+<>c.<UpdateAllServices>b__72_0 (Microsoft.MixedReality.Toolkit.IMixedRealityService service) [0x00000] in <00000000000000000000000000000000>:0 
  at System.Action`1[T].Invoke (T obj) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.MixedReality.Toolkit.MixedRealityToolkit.ExecuteOnAllServicesInOrder (System.Action`1[T] execute) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.MixedReality.Toolkit.MixedRealityToolkit.UpdateAllServices () [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.MixedReality.Toolkit.MixedRealityToolkit.Update () [0x00000] in <00000000000000000000000000000000>:0 

The line at fault is here. The unityJointPoses array in WindowsMixedRealityXRSDKArticulatedHand is initialized to null and stays like that forever since it's private readonly and I assume nobody else writes to it via unusual means (i.e. reflection). In 2.7 this used to be a dictionary that was properly initialized.

Expected behavior

Hands work properly and without performance problems and exceptions being thrown.

Your setup

• Unity 2020.3.31f1
• MRTK 2.8
• WMR Plugin 4.6.3

Target platform

• HoloLens 2

[keveleigh]

Thanks for the report! A fix for this has been merged via #10622 and we'll be releasing it soon in a 2.8.1 release.

[Holo-Krzysztof]

Thanks for the response; I'll be waiting for the next release then.

[Schroedingers-Cat]

@keveleigh I updated my project to MRTK 2.8.1 using the MRTK Feature tool but the file from Packages\MixedReality\com.microsoft.mixedreality.toolkit.foundation-2.8.1.tgz isn't at the state from the 2.8.1 branch. This is what the file should be like, note the added nullcheck in the marked lines: https://github.com/microsoft/MixedRealityToolkit-Unity/blob/main/Assets/MRTK/Providers/WindowsMixedReality/XRSDK/Controllers/WindowsMixedRealityXRSDKArticulatedHand.cs#L139-L142

Inside Packages\MixedReality\com.microsoft.mixedreality.toolkit.foundation-2.8.1.tgz there's this file instead:
WindowsMixedRealityXRSDKArticulatedHand.cs.txt

It's missing the added nullcheck (which causes an NRE spam in my project). Was this PR ultimately not included in the 2.8.1 release?

[Holo-Krzysztof]

Can confirm; looks like a publishing issue, maybe whoever pushed this to MR Feature Tool used the wrong branch?

[keveleigh]

Thanks for the reports! I'll investigate

[keveleigh]

@Holo-Krzysztof @Schroedingers-Cat Apologies for the noise here, but I've just published 2.8.2 which contains the fixes intended for 2.8.1. Please let me know if those still don't work for you!