Commits on Feb 15, 2016

1. Add a lazy DFA. …

   A lazy DFA is much faster than executing an NFA because it doesn't
   repeat the work of following epsilon transitions over and and over.
   Instead, it computes states during search and caches them for reuse. We
   avoid exponential state blow up by bounding the cache in size. When the
   DFA isn't powerful enough to fulfill the caller's request (e.g., return
   sub-capture locations), it still runs to find the boundaries of the
   match and then falls back to NFA execution on the matched region. The
   lazy DFA can otherwise execute on every regular expression *except* for
   regular expressions that contain word boundary assertions (`\b` or
   `\B`). (They are tricky to implement in the lazy DFA because they are
   Unicode aware and therefore require multi-byte look-behind/ahead.)
   The implementation in this PR is based on the implementation in Google's
   RE2 library.
   
   Adding a lazy DFA was a substantial change and required several
   modifications:
   
   1. The compiler can now produce both Unicode based programs (still used by the
      NFA engines) and byte based programs (required by the lazy DFA, but possible
      to use in the NFA engines too). In byte based programs, UTF-8 decoding is
      built into the automaton.
   2. A new `Exec` type was introduced to implement the logic for compiling
      and choosing the right engine to use on each search.
   3. Prefix literal detection was rewritten to work on bytes.
   4. Benchmarks were overhauled and new ones were added to more carefully
      track the impact of various optimizations.
   5. A new `HACKING.md` guide has been added that gives a high-level
      design overview of this crate.
   
   Other changes in this commit include:
   
   1. Protection against stack overflows. All places that once required
      recursion have now either acquired a bound or have been converted to
      using a stack on the heap.
   2. Update the Aho-Corasick dependency, which includes `memchr2` and
      `memchr3` optimizations.
   3. Add PCRE benchmarks using the Rust `pcre` bindings.
   
   Closes #66, #146.

   

   BurntSushi committed Feb 15, 2016
   Configuration menu

   • View commit details
   • Copy full SHA for 2aa1727
   • Browse repository at this point

   Copy the full SHA

   2aa1727 View commit details

   Browse the repository at this point in the history