Allow lists of nested selectors and allow styles after nested CSS blocks

[rodrigogiraoserrao]

Fixes #3969 and fixes #3999.

Changes the tokenizer to accept lists of selectors in a nested context and also to accept styles after nested scopes.

While working on the above, we opened #4039 because the tokenizer isn't yet clever enough to distinguish a selector with a pseudo-class from a rule with a value in a nested context, if the selector + pseudo-class doesn't use the & operator. (See my comment and the interactions under it.)

[willmcgugan]

@rodrigogiraoserrao Please add some detail on what you fixed (and how). The issues you linked to don't make it clear what work is outstanding. The title is also rather vague -- what was improved?

[rodrigogiraoserrao]

@willmcgugan changed the title, expanded the description, and clarified the outstanding issue 4039.

[darrenburns]

I've added a couple more comments - sorry! EDIT: Ignore 😆

Oops!

[rodrigogiraoserrao]

@darrenburns ignored! But did you review at all? It doesn't show neither an accepted nor a change requested.

[darrenburns]

Nope - I requested changes on the wrong PR. I didn't review this one.

[willmcgugan]

@rodrigogiraoserrao Your tests are essentially snapshot tests. They are as likely to catch any change to the tokenizer, as they are to catch errors.

Suggest we have a test which takes CSS then checks that the expected styles are set on the widget. If that breaks in the future we know that it is a genuine issue, and not just that we have made an inconsequential changed to the tokenizer.

[rodrigogiraoserrao]

@rodrigogiraoserrao Your tests are essentially snapshot tests. They are as likely to catch any change to the tokenizer, as they are to catch errors.

Isn't that true of all tokenizer tests?

Suggest we have a test which takes CSS then checks that the expected styles are set on the widget.

Then, doesn't that mean the test can fail for many different reasons that are completely orthogonal to the thing I'm trying to test, which is that the tokenizer knows what to do with these tokens?

[willmcgugan]

Isn't that true of all tokenizer tests?

Yes, which is why we don't need more.

Then, doesn't that mean the test can fail for many different reasons that are completely orthogonal to the thing I'm trying to test, which is that the tokenizer knows what to do with these tokens?

That's not what we should be testing. We should be testing the CSS that didn't work before, and now does. The output from the tokenizer does not prove that the CSS is processes as expected.

[rodrigogiraoserrao]

@willmcgugan got it. See 3c18e47.
This is ready to be reviewed.

[willmcgugan]

Better!

[darrenburns]

Looks good!