feat: tvos 17 workaround with skipDeviceCheck

[KazuCocoa]

[update Mar 1st, 2024]

This skip device check would reduce device preparation time by 1 sec on my local test, so it may be good to add as part of performance improvement for a real device that manages devices on its own.

Note that commands that need to communicate with the device directly (usbmuxd) do not work. This is expected. This PR aims to make tvOS work over WDA at least while it may have issues with communication method not via WDA.

---

This is currently what we could do as a workaround for appium/appium#19343.

appium/appium#19343 (comment) is my findings for my local apple tv. xcodebuild itself did not work (was not able to find the network connected tvOS 17 device), thus the primary usage should be with webDriverAgentUrl

For tvOS 17+: necessary caps:

  "appium:skipDeviceCheck": true,
  "appium:skipLogCapture": true

btw, this skip reduces 1 sec or less's new session creation duration for real devices.
I wondered if this could be similar to appium:skipDeviceInitialization in uia2.

[Update Mar 23th]

• Remove Frameworks/XC** framework from WebDriverAgentRunner_tvOS-Runner.app

$ xcrun devicectl device install --device 9DA168DB-A406-4350-A1AF-125411E55394 /Users/kazu/Library/Developer/Xcode/DerivedData/WebDriverAgent-ezumztihszjoxgacuhatrhxoklbh/Build/Products/Debug-appletvos/WebDriverAgentRunner_tvOS-Runner.app
$ xcrun devicectl device process launch --device 9DA168DB-A406-4350-A1AF-125411E55394 --activate --environment-variables '{"USE_PORT": "8100"}'  com.kazucocoa.WebDriverAgentRunner.xctrunner

Then, set "appium:webDriverAgentUrl": "http://192.168.5.0:8001" capability with this branch's caps. Then, we can start WDA without Xcode.

9DA168DB-A406-4350-A1AF-125411E55394 is identifier by xcrun devicectl list devices.
The path to WebDriverAgentRunner_tvOS-Runner.app is the result of Xcode. It is signed properly already, so it should be already confirmed the build works on a tvOS device.

[mykola-mokhnach]

To me it looks like we are just trying to workaround a xcodebuild bug. Perhaps, it has also to do with appium/appium#19381

Maybe it would be better to just wait for a fix from Apple rather then adding workarounds thus increasing the tech debt?

[KazuCocoa]

Yes, it's reasonable. Ideally, (and hopefully) Apple will expose udid to the macOS system wide as same as USB connected devices (as same as older tvOS versions such as 16).

Or at least once Apple fixes xcodebuild, we can also use this pr at least... (I know of this is actually not good idea)

[mykola-mokhnach]

I was thinking about doing something like

if (xcodeVersion.major === 17 && xcodeVersion.minor >= 0 && xcodeVersion.minor < 2 && isTvOS) {
  skipIdCheck;
  runXcodebuildWithoutId;
}

but even by having the above ugly workaround many other endpoints that work with udid would most likely fail

[quidow]

Hey, thank you so much for publishing this workaround, it literally saved the start of my project. I also wanted to mention one thing that I noticed, maybe you will find it helpful.
So if we go to Settings -> Remotes and Devices -> Remote App and Devices on the device with tvOS 17 installed, then it's IPv6 becomes available and I can see it in the idevice_id output. And as long as we stay on that screen, we can use its address as udid and Appium is able to establish a session without the workaround. But the issue is that as soon as we leave that screen, the device becomes unavailable again.

[KazuCocoa]

note: i wondered if we could get device info via WDA instead of host in some cases for webDriverAgentUrl usage for example. Maybe we checked device os version and a few we've got via appium-ios-device

[adriandumitrascu19]

@KazuCocoa this workaround would work with a physical device ? I am currently blocked with 17.2 on my apple tv

[KazuCocoa]

This change itself yes, would work.

xcodebuild itself had an issue that could not find a wireless connected tvOS device.
appium/appium#19381 I haven't tested recent Xcode versions, but possibly even with this change, xcodebuild (Apple's command) itself won't work.

[quidow]

I don't really get what xcodebuild issue you are talking about.
I was able to automate the test session launch and run WDA on AppleTV 4K using this:

xcodebuild build-for-testing test-without-building -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner_tvOS -destination 'id={self.__device_udid}'

[KazuCocoa]

Then, you could use this branch with local installation.
https://appium.io/docs/en/2.4/cli/extensions/#install

e.g.

appium driver install --source=local /path/to/this/repository

which is regular usage of Appium v2

[adriandumitrascu19]

@quidow are you able to start a session with the appium inspector on a physical device ?

[quidow]

@quidow are you able to start a session with the appium inspector on a physical device ?

No, in order to use Appium Inspector I need to run WDA first.

[adriandumitrascu19]

@quidow Can you help me out please?

1. Run WDA first.
2. Open appium inspector but... I am not able to start a session with the physical because I receive the error: Failed to create session. An unknown server-side error occurred while processing the command. Original error: Unknown device or simulator UDID: '9ec9b7b0de4f70dbe2e6bb93d75e957ef047b711'

I am using:
appium 2.3.0
[email protected]
apple tv tvOS version 17.2
Xcode 15.1

[quidow]

@quidow Can you help me out please?

1. Run WDA first.
2. Open appium inspector but... I am not able to start a session with the physical because I receive the error: Failed to create session. An unknown server-side error occurred while processing the command. Original error: Unknown device or simulator UDID: '9ec9b7b0de4f70dbe2e6bb93d75e957ef047b711'

I am using: appium 2.3.0 [email protected] apple tv tvOS version 17.2 Xcode 15.1

1. Can you share your Appium capabilities?
2. Did you build and install Appium driver from this PR?

[adriandumitrascu19]

@quidow Can you help me out please?

1. Run WDA first.
2. Open appium inspector but... I am not able to start a session with the physical because I receive the error: Failed to create session. An unknown server-side error occurred while processing the command. Original error: Unknown device or simulator UDID: '9ec9b7b0de4f70dbe2e6bb93d75e957ef047b711'

I am using: appium 2.3.0 [email protected] apple tv tvOS version 17.2 Xcode 15.1

1. Can you share your Appium capabilities?
2. Did you build and install Appium driver from this PR?

I have added the appium:skipDeviceCheck and set it to true
can you help please?

1. {
   "platformName": "tvOS",
   "appium:platformVersion": "17.2",
   "appium:bundleId": "my-app",
   "appium:automationName": "XCUITest",
   "appium:noReset": true,
   "appium:deviceName": "Apple TV 4K",
   "appium:derivedDataPath": "/Users/adriandumitrascu/Library/Developer/Xcode/DerivedData/WebDriverAgent-atygjtlzhmclcddyzjqfzjhqdaoh",
   "appium:udid": "9ec9b7b0de4f70dbe2e6bb93d75e957ef047b711",
   "appium:skipDeviceCheck": "true"
   }

checked out appium-xcuitest-driver
switched to this branch and pulled

[quidow]

@adriandumitrascu19 I see that you run WDA from this branch, but did you install the Appium driver from this branch as well?

Then, you could use this branch with local installation. https://appium.io/docs/en/2.4/cli/extensions/#install

e.g.

appium driver install --source=local /path/to/this/repository

which is regular usage of Appium v2

[quidow]

I have added the appium:skipDeviceCheck and set it to true but I am not familiar with the second solution proposed to use with appium:webDriverAgentUrl to avoid device communication. can you help please?

1. {
   "platformName": "tvOS",
   "appium:platformVersion": "17.2",
   "appium:bundleId": "de.telekom.magenta.tv.PRODUCTION5",
   "appium:automationName": "XCUITest",
   "appium:noReset": true,
   "appium:deviceName": "Apple TV 4K",
   "appium:derivedDataPath": "/Users/adriandumitrascu/Library/Developer/Xcode/DerivedData/WebDriverAgent-atygjtlzhmclcddyzjqfzjhqdaoh",
   "appium:udid": "9ec9b7b0de4f70dbe2e6bb93d75e957ef047b711",
   "appium:skipDeviceCheck": "true"
   }

checked out appium-xcuitest-driver switched to this branch and pulled

"appium:webDriverAgentUrl": "http://192.168.1.104:8100",
"appium:skipDeviceCheck": true,

192.168.1.104 - the local IP of my AppleTV
8100 - the default WDA port
"appium:skipDeviceCheck": true, - make sure that you set true as a boolean value, but not a string

[adriandumitrascu19]

@adriandumitrascu19 I see that you run WDA from this branch, but did you install the Appium driver from this branch as well?

Then, you could use this branch with local installation. https://appium.io/docs/en/2.4/cli/extensions/#install
e.g.

appium driver install --source=local /path/to/this/repository

regular usage of Appium v2

[quidow]

@adriandumitrascu19 so now it works?

[adriandumitrascu19]

@adriandumitrascu19 so now it works?

so with the actual setup (from above) I am able to start the appium session only if I remove the appium:bundleId. I can then start my app and inspect the elements but if I want to run a test suite that needs the bundleId it is not working. Any thoughts ?

[quidow]

Should I set any additional capability if want to use this workaround to test in parallel?

[KazuCocoa]

With webDriverAgentUrl as this description, then no. The URL should be handled properly

[quidow]

I have some weird issue that I cannot resolve for a long time....
When I run the tests on a single AppleTV device, everything works fine, but if I try to do parallel execution, WDA black screen can randomly appear during the test execution. I tried almost everything that came in my mind, but nothing helps.
I implemented some logic that makes a query from the list of the tests and the devices start to act as workers to perform these tests until the query is empty. It work fine with every platform, including iOS, but on AppleTV I get this.
Maybe you have any thoughts how to resolve it?

[KazuCocoa]

I guess the black screen is when your WDA is running foreground. How did you start the process? For example when the WDA process restarted, such foreground behavior could occur I assume.

[quidow]

Before every test I try to establish the session, and if it fails, I restart WDA with
f"xcrun devicectl device process launch --device {self.__device_udid} --activate --terminate-existing --environment-variables '{{\"USE_PORT\": \"8100\"}}' {self.device_info['updatedWDABundleId']}"

I don't understand how, but when 2 devices are working at the same time, it somehow affects the WDA behaviour on the devices. Every device is managed by a separate process. I tried to use the same Appium Server, separate servers, with and without --session-override, doesn't matter. It happens not on every test, but randomly. Let's say, 10 tests out of 100 fail because of this, but if only 1 device is working, it doesn't reproduce.