Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(input)!: Consolidate Input traits #107

Merged
merged 20 commits into from
Feb 1, 2023
Merged

fix(input)!: Consolidate Input traits #107

merged 20 commits into from
Feb 1, 2023

Conversation

epage
Copy link
Collaborator

@epage epage commented Feb 1, 2023

This is modeled off of rust-bakery/nom#1612 and rust-bakery/nom#1610 with a mix of combine::Stream.

  • Chose Token, like combine, as that is the term I've been generally
    standardizing on and is clearer in intent than Item
  • Baking in Slice support as that is important for wrapper types. I chose
    Slice over combines Range as this is more focused on Rust integration,
    rather than grammar integration, and we are generally using slice. This helped simplify things having next_slice return Input::Slice rather than having to do into_output().
  • next_token is really what some of nom's existing traits are trying to do
    but awkwardly. combine does this through StreamOnce::uncons
  • iter_offsets instead of iter_tokens as the only time you should care
    about the tokens, you should be looking at the offsets unless using something
    like AsBytes.
  • next_slice is a parallel to next_token but the return type differs as the
    lookup is done in a different function call. I'm tempted to add an
    unchecked variant but I want to benchmark first.

I considered following combine and doing separate InputToken and
InputSlice traits but figured we'd see how this goes first.

As part of this, a streaming bug in hex_u32 was fixed

BREAKING CHANGE: Input traits changed for most parsers and the old traits were removed

Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clippy found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

This is modeled off of rust-bakery/nom#1612 with a mix of `combine::Stream.
- Chose `Token`, like `combine`, as that is the term I've been generally
  standardizing on and is clearer in intent than `Item`
- Baking in `Slice` support as that is important for wrapper types.  I
  chose `Slice` over `combine`s `Range` as this is more focused on Rust
  integration, rather than grammar integration, and we are generally using
  `slice`.
- `next_token` is really what some of nom's existing traits are trying to do
  but awkwardly. `combine` does this through `StreamOnce::uncons`
- `iter_offsets` instead of `iter_tokens` as the only time you should care
  about the tokens, you should be looking at the offsets unless using
  something like `AsBytes`.
- `next_slice` is a parallel to `next_token` but the return type differs
  as the lookup is done in a different function call.  I'm tempted to
  add an `unchecked` variant but I want to benchmark first.

I considered following `combine` and doing separate `InputToken` and
`InputSlice` traits but figured we'd see how this goes first.
Couldn't touch `length_*` because it depends on other parsers
This also demonstrates a bug with the existing code
BREAKING CHANGE: trait bounds for `hex_u32` changed
@coveralls
Copy link

coveralls commented Feb 1, 2023

Pull Request Test Coverage Report for Build 4068391968

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 380 of 574 (66.2%) changed or added relevant lines in 13 files are covered.
  • 31 unchanged lines in 8 files lost coverage.
  • Overall coverage increased (+1.3%) to 58.094%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/bits/mod.rs 4 5 80.0%
src/parser.rs 2 4 50.0%
src/combinator/mod.rs 12 19 63.16%
src/bytes/complete.rs 52 65 80.0%
src/character/complete.rs 72 90 80.0%
src/number/streaming.rs 32 51 62.75%
src/number/complete.rs 30 52 57.69%
src/character/streaming.rs 66 92 71.74%
src/bytes/streaming.rs 30 60 50.0%
src/input.rs 49 105 46.67%
Files with Coverage Reduction New Missed Lines %
src/combinator/mod.rs 1 58.2%
src/number/streaming.rs 1 62.55%
src/bytes/streaming.rs 2 54.02%
src/character/mod.rs 2 35.23%
src/character/streaming.rs 2 74.24%
src/bytes/complete.rs 3 68.02%
src/parser.rs 5 41.32%
src/input.rs 15 37.88%
Totals Coverage Status
Change from base Build 4048669216: 1.3%
Covered Lines: 1676
Relevant Lines: 2885

💛 - Coveralls

@epage epage merged commit de335bb into winnow-rs:main Feb 1, 2023
@epage epage deleted the trait branch February 1, 2023 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants