Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specialize TupleCombinations::fold #775

Merged

Conversation

Philippe-Cholet
Copy link
Member

Related to #755.

[8.2569 ms 8.2733 ms 8.2907 ms]             // Benchmark `TupleCombinations::fold`
[3.2749 ms 3.2875 ms 3.3007 ms]             // `TupleCombinations::fold`
[3.3237 ms 3.3439 ms 3.3661 ms] (same perf) // `Tuple1Combinations::fold`
[237.52 µs 238.81 µs 240.59 µs]             // `Tuple*Combinations::fold` (macro)

This is so much faster that I'm wandering if I'm missing something here!

I previously added the specialization test: a9aaeb4

I'm unsure we can fold the while let loop as we need to clone iter at every step of it.
Both attempts below successfully pass the tests but are not faster.

Maybe there is a solution involving std::cell::??? but I'm not familiar enough with this, are you?
I tried with itertools::rciter but it was 60% slower (and Rc means the use_alloc feature). And with a basic RcIter::fold specialization, I even had an (expected) error at runtime.

let iter = $crate::rciter(iter);
iter.clone().fold(init, |acc, z| {
    let iter_ref = iter.rciter.borrow();
    <$P<I>>::from(iter_ref.clone())
        .map(|($($X),*,)| (z.clone(), $($X),*))
        .fold(acc, &mut f)
})

I eventually found a way to rely on iter.fold using unsafe only (to hold &mut I and &I at the same time) but it's the same performance as the current fold specialization.
I'm not familiar with unsafe Rust so it might just be dumb in the first place.

let iter_mut = &mut iter;
let iter_ref: &I = unsafe {
    std::mem::transmute_copy(&iter_mut)
};
iter_mut.fold(init, |acc, z| {
    let c: $P<I> = iter_ref.clone().into();
    c.map(|($($X),*,)| (z.clone(), $($X),*))
        .fold(acc, &mut f)
})

@Philippe-Cholet Philippe-Cholet force-pushed the fold-tuple-combinations branch 2 times, most recently from 25fceae to 0e15566 Compare October 6, 2023 12:46
`I::Item` and `A` are the same so the "where condition" is not really changed, but it is apparently needed here.
We fold `c` then for each item of `iter`, we fold the cloned `iter`.
The `while let` loop should probably be converted to `fold` somehow later (if possible) but the core logic is done and it is already really faster.
@Philippe-Cholet
Copy link
Member Author

Philippe-Cholet commented Oct 24, 2023

Maybe I should merge .map(...).fold(init, &mut f) into a single .fold(init, |...| ...)?!

EDIT: It would be 6% slower.

Copy link
Member

@phimuemue phimuemue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Philippe-Cholet, sorry for the delay. Nice that's is already faster. Feel free to merge if you want.

Is this specialization already tested in our specialization tests?

src/adaptors/mod.rs Outdated Show resolved Hide resolved
src/adaptors/mod.rs Outdated Show resolved Hide resolved
src/adaptors/mod.rs Outdated Show resolved Hide resolved
@Philippe-Cholet
Copy link
Member Author

Philippe-Cholet commented Oct 26, 2023

As mentionned in the too big message, the specialization test was previously added (by me).
I'm unsure I can merge this myself though. I wanted a feedback first anyway, thanks for it.
EDIT: I guess now I can since you "approved those changes".

@Philippe-Cholet Philippe-Cholet added this pull request to the merge queue Oct 27, 2023
Merged via the queue into rust-itertools:master with commit baaf852 Oct 27, 2023
8 checks passed
@Philippe-Cholet Philippe-Cholet deleted the fold-tuple-combinations branch October 27, 2023 07:34
@Philippe-Cholet Philippe-Cholet added this to the next milestone Nov 3, 2023
@jswrenn jswrenn mentioned this pull request Nov 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants