Use smol_str for identifiers

[Razican]

Currently, we use a partially optimised version of String for identifiers and function names parsing and lexing. While it's OK, we could greatly benefit from using small strings that will usually be stored inline.

The smol_str crate was made for this, and it's being used on rust-analyzer, for example. It will store up to 22-bytes (ASCII characters) inline, in the stack, and then do some optimisations for whitespace strings. It will then store bigger strings in Rc'd heap allocations. It uses immutable strings, though, but identifiers don't change.

A nice review on this and other crates was done by Amos here.

This should especially help with minified code, that we will be benchmarking in #564.

This is waiting for #564 and #559.

[neeldug]

After looking into lasso's usage, specifically for node identifiers, at which scope would you want the cache to be kept at?

[Razican]

After looking into lasso's usage, specifically for node identifiers, at which scope would you want the cache to be kept at?

lasso would be an interner (related to #279). In any case, I think the lexer should have the interner, and then, once the parser finishes parsing, it shoulg return that interner for the interpreter to use.

But in fact, this might be easier if this was a property of a Boa structure, that would have the lexer, the parser and the interpreter too.

[neeldug]

After looking into lasso's usage, specifically for node identifiers, at which scope would you want the cache to be kept at?

lasso would be an interner (related to #279). In any case, I think the lexer should have the interner, and then, once the parser finishes parsing, it shoulg return that interner for the interpreter to use.

But in fact, this might be easier if this was a property of a Boa structure, that would have the lexer, the parser and the interpreter too.

Was looking into the property of a Boa structure, in doing so would the constructor have to call the constructor of the lexer, the parser and the interpreter, or would those be passed into the constructor to hold as a pointer. I also was trying to make the structure for the interner less convoluted by generating its own method, but can't access the Key type as it's private, if you have any suggestions to get around this, it would be much appreciated.

[neeldug]

@Razican this can be unblocked now right? As both changes have now landed.

[neeldug]

Also @Razican currently the type being used is RcString for tokens correct?

[Razican]

This will no longer be implemented, as interned strings will be used instead.