Version 8.0.401 release broke ability to run app with WebDriverAgent from Appium on iOS

[rolfbjarne]

From @Joost-Jens-Luminis on Fri, 13 Sep 2024 07:26:42 GMT

We have been creating tests using Appium for a few months now for our app on Windows, Android and iOS.
Unfortunately, with the release of .Net 8.0.401 and the released iOS workload it broke our ability to test on iOS.

When the WebDriverAgent from Appium is running, it takes to long for the App to start now and it gets killed by the iOS watchdog.
If the WebDriverAgent from Appium is not running, the app is starting up fine and working properly.

With .Net version 8.0.300 (and related workloads) it is still working properly. Unfortunately, as we are using a release version, build in Azure Pipelines, to run the tests we cannot downgrade to an earlier version of the workloads.

The following installation is still working fine with Appium testing on iOS (This is from my PC):

Is there any information on what has changed in net8.0-ios that could have caused this? And how we could work around this issue?

Note: We are a 100% sure Appium didn't cause this, as we hadn't update Appium when the problems started.
Of course we did try updating Appium and it's drivers to the latest version but to no avail.

Copied from original issue dotnet/runtime#107781

[rolfbjarne]

From @vitek-karas on Fri, 13 Sep 2024 11:29:09 GMT

@rolfbjarne maybe you have some context.
Should we move this to xamarin-macios?

[rolfbjarne]

@Joost-Jens-Luminis it's not clear from your description, but can you reproduce this locally, or does it only happen on CI?

To start diagnosing this, it would be good to get any crash reports from the device: https://github.com/xamarin/xamarin-macios/wiki/Diagnosis#crash-reports

That would reveal what the app is actually doing when it's being terminated.

[Joost-Jens-Luminis]

@rolfbjarne Yes, this can be reproduced locally. A colleague of mine has updated his installation and with his builds (both debug and release builds) the takes to long to start.
To be extra clear, the splashscreen becomes visible, but then the app stops responsing and gets closed by the watchdog on iOS after about 19 seconds.

This also means that there is no crash. It just gets closed.

[Joost-Jens-Luminis]

This is a video of what happens. It's boring, but give you an idea
https://github.com/user-attachments/assets/98f03e76-f922-4fcb-b64f-078fc3c93134

The first document is the console log from the iOS device regarding what is going on at the time for our App (Bronkhorst FlowcontrolApp). The second one is the full log unfiltered (in cases the filter filters something out that is important)
Filtered console log.txt
Full console log.txt

I hope this provides enough information to get a clue as to what is going on here.

[rolfbjarne]

This also means that there is no crash. It just gets closed.

iOS will still create a crash report. This is from the device log:

ReportCrash Formulating fatal 309 report for corpse[379] Bronkhorst.FlowControl.iOS

[Joost-Jens-Luminis]

You are correct, this is a crash log I got from the phone. I generated it this morning though as I couldn't find the crashlogs from last Friday. But I assume it's the same crash.

Crash log.txt

[rolfbjarne]

Yes, that crash report is for the watchdog:

exhausted real (wall clock) time allowance of 19.76 seconds

Unfortunately all the important frames in the stack trace are only memory addresses:

Thread 0 name:  tid_103 Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0   libsystem_kernel.dylib        	       0x1bb181484 __psynch_cvwait + 8
1   libsystem_pthread.dylib       	       0x1dbc09860 _pthread_cond_wait$VARIANT$armv81 + 1224
2   Bronkhorst.FlowControl.iOS    	       0x104851968 0x104390000 + 4987240
3   Bronkhorst.FlowControl.iOS    	       0x10491bb5c 0x104390000 + 5815132
4   Bronkhorst.FlowControl.iOS    	       0x10491ba00 0x104390000 + 5814784
5   Bronkhorst.FlowControl.iOS    	       0x10493bc90 0x104390000 + 5946512
6   Bronkhorst.FlowControl.iOS    	       0x1048da590 0x104390000 + 5547408
7   Bronkhorst.FlowControl.iOS    	       0x104999a6c 0x104390000 + 6330988
8   Bronkhorst.FlowControl.iOS    	       0x104998404 0x104390000 + 6325252
9   Bronkhorst.FlowControl.iOS    	       0x10498cddc 0x104390000 + 6278620
10  Bronkhorst.FlowControl.iOS    	       0x10498a974 0x104390000 + 6269300
11  Bronkhorst.FlowControl.iOS    	       0x104959ca4 0x104390000 + 6069412
12  Bronkhorst.FlowControl.iOS    	       0x1049069dc 0x104390000 + 5728732
13  Bronkhorst.FlowControl.iOS    	       0x1049096fc 0x104390000 + 5740284
14  Bronkhorst.FlowControl.iOS    	       0x104a45c2c 0x104390000 + 7035948
15  Bronkhorst.FlowControl.iOS    	       0x104a48988 0x104390000 + 7047560
16  UIKitCore                     	       0x182e6b5d0 -[UIApplication _handleDelegateCallbacksWithOptions:isSuspended:restoreState:] + 336

This is easily fixable, by adding the following to the csproj, rebuilding and triggering a new crash report:

<PropertyGroup>
    <NoSymbolStrip>true</NoSymbolStrip>
</PropertyGroup>

Can you do that and upload the new crash report?

[Joost-Jens-Luminis]

Here is the file:

Bronkhorst.FlowControl.iOS-2024-09-17-084529.txt

[rolfbjarne]

That didn't change anything, the stack trace still only shows memory addresses.

Can you get an MSBuild binlog so that I can investigate to see why this happens?

[Joost-Jens-Luminis]

Maybe that was because it was a release build. I made one with debug, there is a bit more information in here.
Crash report from debug build.txt

Unfortunately I spend yesterday trying to update my systems (Windows and Mac) so I can reproduce this while debugging, unfortunately that prevented me from debugging al to gather (VS can't connect to my Mac anymore due to a null reference exception), so this is still a build from the Azure Devops Pipeline but with debugging set instead of release. I hope this is enough.

[rolfbjarne]

Yes, that's a bit more helpful at least.

Can you try turning the interpreter off to see if that changes anything?

<PropertyGroup>
    <UseInterpreter>false</UseInterpreter>
</PropertyGroup>

if it still fails the same way, please get an updated crash report.

[Joost-Jens-Luminis]

You can use the latest workload to see it go wrong.

On iOS Workload 17.2.8078 everything is working as expected.

[rolfbjarne]

This works for me, which is the latest as of today:

$ dotnet workload list

Installed Workload Id      Manifest Version       Installation Source
---------------------------------------------------------------------
maui-tizen                 8.0.82/8.0.100         SDK 8.0.300
ios                        18.0.8303/8.0.100      SDK 8.0.300

[Joost-Jens-Luminis]

That is interesting.
With this it was failing: (This is what I have on my PC)

But you have a newer workload so maybe it's fixed with that?

[rolfbjarne]

Even those versions work for me :/

A few more questions:

• Can you reproduce on both simulator and device, or only one of them?
• Do you see the same broken behavior when you tap on the app (as opposed to launching from within the IDE)?
• Do you see the same broken behavior if you use VSCode on macOS directly to run the app?

[Joost-Jens-Luminis]

1. I am only working on a real device
2. Yes
3. I haven't tested that yet. I have always used VS on windows to debug versions. I have used release builds as well.

Do you have the WDA from Appium installed on your test device? Without that installed it works for me as well, but with that installed is when these problems occur.

[rolfbjarne]

Do you have the WDA from Appium installed on your test device? Without that installed it works for me as well, but with that installed is when these problems occur.

No, I don't; how do I get that installed?

[Joost-Jens-Luminis]

The following are the instructions I follow when I installed it (They are from our own internal wiki).

A good tutorial can be found here: https://www.youtube.com/watch?v=kvNnF4Zpgmo&t=2s
Note: it is a bit out of date as you no longer need to install the WDA from the commandline.

Source: https://www.33rdsquare.com/the-complete-step-by-step-guide-to-appium-ios-test-automation/

• Homebrew is a must-have package manager for any Mac device. Open your Mac terminal and install Homebrew:
  /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

• Next, use Homebrew to install Node.js and the Appium server:

brew install node
npm install -g appium

• Appium requires some additional iOS-focused dependencies like ios-deploy and Carthage:

brew install ideviceinstaller 
npm install -g ios-deploy
brew install carthage

• Install the Appium driver:
  appium driver install xcuitest

https://appium.github.io/appium-xcuitest-driver/latest/

• Configure and install the WDA on the mobile device:

You can open the WDA using the following command:

appium driver run xcuitest open-wda

Make sure to set codesign for all packages (else it may not build).

• Use "Product -> destination" to select your device.
• Use "Product -> Scheme -> WebDriverAgentRunner" to select the package to install.
• Use "Product -> Test" to install the agent on your phone.
  Source: https://appium.github.io/appium-xcuitest-driver/latest/guides/run-preinstalled-wda/#using-xcode

[rolfbjarne]

OK, I can reproduce it now, the WDA app must also be running from inside Xcode in order to reproduce.

[rolfbjarne]

I've finally figured out what's happening, and it's an unintended side effect of a new feature we added.

The problem is in with the Window property in the class MvxApplicationDelegate, and how it's declared new:

public abstract class MvxApplicationDelegate : UIApplicationDelegate, IMvxApplicationDelegate
{
    public new virtual UIWindow Window { get; set; }
	// ...
}

Prior to when this broke, this Window property was not exported to Objective-C (it doesn't have an [Export] attribute, and doesn't override a property that does either).

In one of our latest updates, we implemented improved support for binding protocols by taking advantage of default interface members in C#. The consequence is that the Window property is now exported to Objective-C, because:

1. The Objective-C protocol UIApplicationDelegate has a Window property.
2. This Window property is optional, so it's technically not necessary to implement it.
3. The MvxApplicationDelegate class implements the IMvxApplicationDelegate interface, which implements the IUIApplicationDelegate interface (which represents the Objective-C protocol UIApplicationDelegate).
4. We see that MvxApplicationDelegate's Window property matches the Window property on the IUIApplicationDelegate interface (which has an [Export] attribute), and export it to Objective-C.
5. The presence in Objective-C of the Window property makes iOS take a very different code path internally, and that eventually ends up instantiating the DevicesView class twice (the storyboard where the DevicesView class is used is instantiated twice).

Sidenote: it's possible to get replicate the broken behavior with the older version of the workload by making the Window property override the Window property from UIApplicationDelegate (in this case the property is exported to Objective-C, because UIApplication.Window has an [Export] attribute):

public abstract class MvxApplicationDelegate : UIApplicationDelegate, IMvxApplicationDelegate
{
    public override UIWindow Window { get; set; }
	// ...
}

Going forward, I don't think this is something we'll fix/change on our side, because it's such an incredible rare set of circumstances that leads to the eventual deadlock at startup.

I believe the best fix is on your end, by making the DevicesView class not fail when it's instantiated more than once.

Alternatively it might be possible to just rename the Window property, to say MainWindow:

public abstract class MvxApplicationDelegate : UIApplicationDelegate, IMvxApplicationDelegate
{
    public UIWindow MainWindow { get; set; }
    /* Update all the Window references in this class to use MainWindow instead as well! */
}

I'll close this now, but feel free to comment if you disagree or have any further problems, and I'll see what I can do.

FWIW this was one of the most interesting issues I've had to diagnose in a while, so thanks a lot for taking the time to create a test project I could use to reproduce it!

[Joost-Jens-Luminis]

Wow, thanks for the explanation. I am happy you have figured out what went wrong (and that you enjoyed it :D ).

One final question though, is our work around fine? Or will that cause issues in the future?

The "Main.storyboard" file has a property called "initialViewController" which is set to the DevicesView (via it's storyboard ID).
 If we remove that property, it appears to fix the problem we are having. 

edit: spelling

[rolfbjarne]

Wow, thanks for the explanation. I am happy you have figured out what went wrong (and that you enjoyed it :D ).

One final question though, is our work around fine? Or will that cause issues in the future?

The "Main.storyboard" file has a property called "initialViewController" which is set to the DevicesView (via it's storyboard ID).
 If we remove that property, it appears to fix the problem we are having. 

edit: spelling

Yes, that should work fine too I believe.

[Joost-Jens-Luminis]

In that case I consider this issue resolved, the rest lies with the MvvmCross people whom I have informed.
Thank you so much for your hard work on this, it's much appreciated.