Bug: Breaks when the source code it found an unclosed multiline comment.

[ajinabraham]

>>> x = u"\t\t\t// with unwanted files\n\t\t\tfs.unlink(tmp_path, function() {\n\t\t\t\tif (err) {\n\t\t\t\t\tthrow err\n\t\t\t\t}\n\t\t\t\tres.send('File uploaded to: ' + target_path + ' - ' + req.files.thumbnail.size + ' bytes')\n\t\t\t});\n\t\t});\n\t});\n\t*/"
>>> x
u"\t\t\t// with unwanted files\n\t\t\tfs.unlink(tmp_path, function() {\n\t\t\t\tif (err) {\n\t\t\t\t\tthrow err\n\t\t\t\t}\n\t\t\t\tres.send('File uploaded to: ' + target_path + ' - ' + req.files.thumbnail.size + ' bytes')\n\t\t\t});\n\t\t});\n\t});\n\t*/"
>>> len(x)
214
>>> import jsbeautifier
>>> jsbeautifier.beautify(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ajin/Code/x/venv/lib/python2.7/site-packages/jsbeautifier/__init__.py", line 246, in beautify
    return b.beautify(string, opts)
  File "/Users/ajin/Code/x/venv/lib/python2.7/site-packages/jsbeautifier/__init__.py", line 429, in beautify
    self.tokens = Tokenizer(input, self.opts, self.indent_string).tokenize()
  File "/Users/ajin/Code/x/venv/lib/python2.7/site-packages/jsbeautifier/__init__.py", line 1584, in tokenize
    token_values = self.__tokenize_next()
  File "/Users/ajin/Code/x/venv/lib/python2.7/site-packages/jsbeautifier/__init__.py", line 1748, in __tokenize_next
    if self.input[self.parser_pos] == '*': # peek /* .. */ comment
IndexError: string index out of range

[ajinabraham]

Smaller PoC:

>>> x = "sd=1;*/"                                                                                                                                                                  >>> jsbeautifier.beautify(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ajin/Code/NodeJSCloudScan/venv/lib/python2.7/site-packages/jsbeautifier/__init__.py", line 246, in beautify
    return b.beautify(string, opts)
  File "/Users/ajin/Code/NodeJSCloudScan/venv/lib/python2.7/site-packages/jsbeautifier/__init__.py", line 429, in beautify
    self.tokens = Tokenizer(input, self.opts, self.indent_string).tokenize()
  File "/Users/ajin/Code/NodeJSCloudScan/venv/lib/python2.7/site-packages/jsbeautifier/__init__.py", line 1584, in tokenize
    token_values = self.__tokenize_next()
  File "/Users/ajin/Code/NodeJSCloudScan/venv/lib/python2.7/site-packages/jsbeautifier/__init__.py", line 1748, in __tokenize_next
    if self.input[self.parser_pos] == '*': # peek /* .. */ comment
IndexError: string index out of range

[ajinabraham]

The ideal way is to gracefully skip line