Skip to content

Commit

Permalink
feat(parser): Treat contractions as a word
Browse files Browse the repository at this point in the history
This should be safe.  Rarely is `'` used as syntax in a language that
separates literals.

- `'` is used within hex literals in C++ but we want to treat them as
  one word
- `'` is used for lifetimes in Rust but there are other symbols on the
  left side.
  • Loading branch information
epage committed Jul 14, 2019
1 parent 9d242fe commit a728b2f
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/tokens.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ impl<'t> Identifier<'t> {
lazy_static::lazy_static! {
// Getting false positives for this lint
#[allow(clippy::invalid_regex)]
static ref SPLIT: regex::bytes::Regex = regex::bytes::Regex::new(r#"\b(\p{Alphabetic}|\d|_)+\b"#).unwrap();
static ref SPLIT: regex::bytes::Regex = regex::bytes::Regex::new(r#"\b(\p{Alphabetic}|\d|_|')+\b"#).unwrap();
}
SPLIT.find_iter(content).filter_map(|m| {
let s = std::str::from_utf8(m.as_bytes()).ok();
Expand Down

0 comments on commit a728b2f

Please sign in to comment.