SIMD Optimizations

[epage]

Some past nom suggestions:

• Feature request - Open to SIMD instructions? rust-bakery/nom#653 to use jetscii for take_while / take_until
• testing aho-corasick based combinator to look for multiple patterns? rust-bakery/nom#1248 which is related to FindSubslice

The question is what should the API look like, how do we handle people who don't want the compile time hit, etc

[epage]

• https://crates.io/crates/sliceslice
• https://crates.io/crates/aho-corasick
• https://crates.io/crates/jetscii

[VorpalBlade]

It would be nice to see aho-corasick used for take_till with a set of literals, not just memchr for take_until. That would be a breaking change (without specialisation, at least) though.

[epage]

Hadn't noted it here but I've found that any discussion of SIMD or advanced algorithms will need a prototype with performance numbers before we can consider how we might want to redesign things to support it.

The problem we had was we thought "why use memchr to look for a string and instead uses memmem which was designed for this"? While memmem works well for a lot of regex-like cases, for parsing it tanked our performance numbers, see #86 and #317.