feat: add noIrregularWhitespace rule

[michellocana]

Summary

Implementation of the no-irregular-whitespace rule from ESLint in Biome.
I also had to do a change in the biome_diagnostics crate to validate the range of irregular whitespaces properly and show them correctly in the output.

Closes #53

Valid

const foo = '\\\u2028';
const foo = '\\\u2029';
/* \u{b}    */ const foo = '�';
/* \u{c}    */ const foo = '';
/* \u{20}   */ const foo = ' ';
/* \u{85}   */ const foo = '�';

Invalid

/* \u{b}    */ const�foo�=�'thing';
/* \u{c}    */ constfoo='thing';
/* \u{85}   */ const�foo�=�'thing';
/* \u{feff} */ constfoo='thing';
/* \u{a0}   */ const foo = 'thing';
/* \u{1680} */ const foo = 'thing';

Test Plan

Added snapshots, for valid and invalid cases according to the original rule tests.

[codspeed-hq]

CodSpeed Performance Report

Merging #3333 will degrade performances by 7.73%

_{Comparing michellocana:feat/no-irregular-whitespace (1229ea2) with main (8eb0e8a)}

Summary

⚡ 1 improvements
❌ 2 regressions
✅ 105 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	main	michellocana:feat/no-irregular-whitespace	Change
❌	js_analyzer[lint_11654203871763747189.ts]	26.4 ms	28.6 ms	-7.6%
⚡	react.production.min_3378072959512366797.js[cached]	2 ms	1.9 ms	+6.82%
❌	db_2930068967297060348.json[cached]	12.5 ms	13.5 ms	-7.73%

[ematipico]

I believe we can evaluate a different approach. As you might notice in this example, the irregular whitespaces are inside trivia.

This information is already present inside the nodes, inside the trivia attached to those nodes. You can retrieve trivia from tokens. You have multiple ways to retrieve the tokens: you directly query all tokens via Ast<JsSyntaxToken>, use some methods from syntax (there are many depending on the situation).

[michellocana]

oh, I didn't know we can query all tokens like that, that definitely makes more sense, I will look into that :)

[ematipico]

Here's the thing about this query, and why it doesn't play well with the logic of the rule. This returns any statement, and in this example, you have multiple statements:

const f = function f() {
	if (true) {} else {}
}

The query will trigger for all the statements of this snippet. However, for each statement, you retrieve ALL tokens, which means that, for example, the else token is checked multiple times because it is returned by the majority of the statements you're querying.

That's why you have this massive regression in performance; the rule is doing repeated work, A LOT. I believe the best strategy is to query the root node of the CST, so you're sure that you aren't doing repeated work on the same tokens.

[michellocana]

I see, I can make the change to query only the root node (I assume it's the JsModule module one?), but that would mean that we can only have one error report about irregular whitespaces per file, right? As far as I know there's no way to report multiple error ranges on a lint rule

[ematipico]

I see, I can make the change to query only the root node (I assume it's the JsModule module one?), but that would mean that we can only have one error report about irregular whitespaces per file, right? As far as I know there's no way to report multiple error ranges on a lint rule

Yes, there is: https://github.com/biomejs/biome/blob/main/crates/biome_analyze/CONTRIBUTING.md#multiple-signals

It gets more challenging to work because you can't do things like ok()?, but it's possible and many rules this approach already.

[michellocana]

oooh, okay, I'm gonna dig into that, thanks!

[michellocana]

@ematipico just did this change, worked like a charm

[michellocana]

nevermind, the benchmarks still failed, I will keep working on that

[unvalley]

@michellocana Thank you, could you resolve the conflicts please?

[michellocana]

@michellocana Thank you, could you resolve the conflicts please?

@unvalley done!

[ematipico]

Thank you @michellocana for this rule. If you have the chance, can you send a PR to add a changelog entry?

[michellocana]

Thank you @michellocana for this rule. If you have the chance, can you send a PR to add a changelog entry?

sure, here it is @ematipico :) #3418