Testdriver: tabs switch frequently when invokes multiple actions

[Honry]

I encountered this issue when trying to automate test at https://github.com/w3c/web-platform-tests/pull/10396/files#diff-52f806f8477bda3b3246226c740e6cb9

When invokes multiple test_driver actions, the runner will switch tab pages between test page and testharness_runner.html immediately and frequently(one action one switch).

[kereliuk]

Good catch, I'll take a look at this tomorrow.

[foolip]

@kereliuk, should this be priority:roadmap, and has the state of things changed in the intervening months? Seems like this is the same window switching that's the reason that fullscreen can't be automated still?

[kereliuk]

Will put on the list for this week, I think the priority should stay the same for now.

[foolip]

I came across this in #12621. It makes it hard to see what's happening, and all that switching back and forth must have effects that may interfere with the tests. It's also the reason for #7281.

@gsnedders, I'm wondering if we should bump the priority of this to "real soon now" and perhaps do together with #13183. Can you describe what it would take to change this?

[gsnedders]

@foolip We'd need to change how testharness.js tests are run to not run them within a new window (see #12885 (comment)), given WebDriver can only deal with one window at a time. So:

1. Get rid of the testharness.js parent window (this will involve changing the executors and tools/wptrunner/wptrunner/testharnessreport*)
2. Change tools/wptrunner/wptrunner/testdriver-extra.js to deal with the current window and not the parent

[jgraham]

I"m not 100% sure that's necessary. What's undoubtedly required is to stop sending commands to the parent window as the test runs. But it looks like that's possible without merging the windows into one. Maybe I'm missing something though.

[gsnedders]

@jgraham if we're sending all the messages to the current window, what use does the parent window have?

[foolip]

FWIW, if we could just not have the extra window that seems much preferable, since the use of window.open() means that we're not testing plain new tabs, which was a problem in the PR @gsnedders already linked to: #12885

[jgraham]

Not having the window is possible but it's a larger change than required to fix this, with a different set of tradeoffs.

[gsnedders]

@jgraham the parent window is used to receive the results of the testsuite, which would then need to go to the test window (given WebDriver would be listening to messages on the test window), so why do we still need the parent window?

[gsnedders]

@jgraham responded to the above question at https://w3.logbot.info/testing/20181008#c446175

[mdittmer]

Ping from your friendly neighbourhood ecosystem infra rotation

If this is priority:roadmap should it have an assignee?

[foolip]

This was fixed by #13419