A11y audits take too long - axe-core 2.1 update needed

[paullewis]

https://www.moonpig.com/uk/

Presumably such a big DOM that axe has a tough time auditing it. But eventually the driver.evaluateAsync call times out, which means all the a11y audits bail as they expect a violations array.

Suggestions:

1. Fail more gracefully within the gatherer, i.e. always set a violations array and set a debug message.
2. Update the individual audits to bail more completely if there's no violations array passed.

Thoughts?

[paullewis]

Actually the DOM size is pretty meagre at 750 elements or so. Wonder why axe is having such a time of it.

[paullewis]

Ahhhhh looks like axe might be crashing:

We should make evaluateAsync account for errors in execution.

[paulirish]

Moving to the new proper async evaluate (#580) will probably give us this for free.

It's probably also a good time to report some bugs upstream to Axe. 1) slowness 2) exceptions sometimes. :)

[brendankenny]

yes, the new API would require the script to evaluate to a promise, so any exceptions should be automatically caught by the error handler.

We could get an early start on this by changing the script passed to evaluateAsync be wrapped in a promise that when resolved or caught calls __returnResults, and have a specific structure for an error message. Should be an easy switch out to just return the promise when m54 hits stable and we have the real deal.

[KyleAMathews]

Noticed Lighthouse freezing on this at the Lodash docs page https://lodash.com/docs/4.15.0 which also has a very large DOM

[wardpeet]

Probably a good time to go forward with #580 as brendankenny specifies above.

/cc @paulirish @brendankenny

[wardpeet]

Ran https://www.moonpig.com/uk/ on latest master and no error.

@brendankenny @paulirish Can I close?

[paulirish]

Let's keep open. I'll retitle.

[paulirish]

While we now recover from the crash, the a11y run does take a long time, as pointed out on dequelabs/axe-core#240
We will need to upgrade to 2.1 when they ship it.

We aren't running 2.0, but presumably there aren't significant perf improvements to make it worthwhile to upgrade to that. Can just wait for 2.1