repl: Handle unicode escape sequence

[princejwesley]

Fix for #3971

[princejwesley]

/cc @bnoordhuis

[jasnell]

CI: https://ci.nodejs.org/job/node-test-pull-request/1636/
(the CI run won't be publicly visible for the time being, one of the collaborators will have to post the results after it runs)

[jasnell]

LGTM if CI is green and @bnoordhuis is happy.

[princejwesley]

@jasnell CI turned green ?

[silverwind]

@princejwesley it's green on all machines, the overall run is in 'cancelled' status, but I guess that's something about the CI, not this PR.

[princejwesley]

@silverwind thanks 😃
/cc @bnoordhuis

[jasnell]

@bnoordhuis ...ping

[bnoordhuis]

Sorry for the delay. Is there a reason you're using regular expressions? They're not super-readable, an literal.indexOf('\\') followed by a simple pattern scanner is arguably a little easier to follow.

[princejwesley]

@bnoordhuis if we use pattern scanner instead of regular expression, we have to create two scanner pattern functions and have to maintain the lastIndex(for multiple match) state like RegExp object.

If it improves readability, I'll build indexOf based RegExp like simple scanner object and update the PR.

[princejwesley]

@bnoordhuis , Here is the comparison code with and without regular expression. I'll update PR if literal version looks good to you.

// ------------------------------------------------------------------------
// With regex
// ------------------------------------------------------------------------
const unicodePointLikePattern = /\\u{([^}]+)}/mg;
const unicodeLikePattern = /\\u(.{0,4})/mg;
const hexLikePattern = /\\x(.{0,2})/mg;
// ------------------------------------------------------------------------

// ------------------------------------------------------------------------
// with simple scanner
// ------------------------------------------------------------------------
class LiteralPattern {

  constructor(prefix, suffixOrMaxLength) {
    this._prefix = prefix;
    this._prefixLength = prefix.length;
    if (typeof suffixOrMaxLength === 'string') {
      this._suffix = suffixOrMaxLength;
      this._suffixLength = suffixOrMaxLength.length;
    } else {
      this._maxLength = suffixOrMaxLength;
    }
    this.lastIndex = 0;
  }

  exec(input) {
    input = input.substring(this.lastIndex);
    let idx = input.indexOf(this._prefix);
    if (idx !== -1) {
      let start = this._prefixLength + idx;
      if (this._suffix) {
        input = input.substring(start);
        let end = input.indexOf(this._suffix);
        if (end !== -1) {
          this.lastIndex += start + end + this._suffixLength;
          return [null, input.substring(0, end)];
        }
      } else {
        this.lastIndex += start + this._maxLength;
        return [null, input.substring(this._prefixLength + idx, this._maxLength)];
      }
    }
    this.lastIndex = 0;
    return null;
  }
}

const unicodePointLikePattern = new LiteralPattern('\\u{', '}');
const unicodeLikePattern = new LiteralPattern('\\u', 4);
const hexLikePattern = new LiteralPattern('\\x', 2);
// ------------------------------------------------------------------------

[princejwesley]

@bnoordhuis shall I update PR with pattern scanner?

[princejwesley]

@bnoordhuis Updated PR with simple pattern scanner.

[jasnell]

@bnoordhuis ... ping

[princejwesley]

@bnoordhuis Fixed merge conflicts!

[princejwesley]

Ping @bnoordhuis

[princejwesley]

Working fine in v6.3.0

~ 🙈 ₹ node
> '\u{110000}'
SyntaxError: Unexpected token ILLEGAL
    at Object.exports.createScript (vm.js:47:10)
    at REPLServer.defaultEval (repl.js:255:25)
    at bound (domain.js:280:14)
    at REPLServer.runBound [as eval] (domain.js:293:12)
    at REPLServer.<anonymous> (repl.js:489:10)
    at emitOne (events.js:101:20)
    at REPLServer.emit (events.js:188:7)
    at REPLServer.Interface._onLine (readline.js:232:10)
    at REPLServer.Interface._line (readline.js:574:8)
    at REPLServer.Interface._ttyWrite (readline.js:851:14)
> process.version
'v6.3.0'