Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parser: Optimize JSON-attribute parsing #11369

Merged
merged 3 commits into from
Nov 4, 2018
Merged

Parser: Optimize JSON-attribute parsing #11369

merged 3 commits into from
Nov 4, 2018

Commits on Nov 2, 2018

  1. Parser: Optimize JSON-attribute parsing 

    Alternate approach to #11355
Fixes: #11066

Parsing JSON-attributes in the existing parsers can be slow when the
attribute list is very long. This is due to the need to lookahead at
each character and then backtrack as the parser looks for the closing
of the attribute section and the block comment closer.

In this patch we're introducing an optimization that can eliminate
a significant amount of memory-usage and execution time when parsing
long attribute sections by recognizing that _when inside the JSON
attribute area_ any character that _isn't_ a `}` can be consumed
without any further investigation.

I've added a test to make sure we can parse 100,000 characters inside
the JSON attributes. The default parser was fine in testing with more
than a million but the spec parser was running out of memory. I'd
prefer to keep the tests running on all parsers and definitely let our
spec parser define the acceptance criteria so I left the limit low for
now. It'd be great if we found a way to update php-pegjs so that it
could generate the code differently. Until then I vote for a weaker
specification.

The default parser went from requiring hundreds of thousands of steps
and taking seconds to parse the attack to taking something like 35
steps and a few milliseconds.

There should be no functional changes to the parser outputs and no
breaking changes to the project. This is a performance/operational
optimization.
    @dmsnell
    dmsnell committed Nov 2, 2018
    Configuration menu
    Copy the full SHA
    448b653 View commit details
    Browse the repository at this point in the history

  2. update generated parser

    @dmsnell
    dmsnell committed Nov 2, 2018
    Configuration menu
    Copy the full SHA
    4bc0f2e View commit details
    Browse the repository at this point in the history

  3. PEGs don't backtrack, take out this (false) optimization

    @dmsnell
    dmsnell committed Nov 2, 2018
    Configuration menu
    Copy the full SHA
    07eb18c View commit details
    Browse the repository at this point in the history