Raise IterationError on StopIteration #473

groutr · 2019-10-18T22:57:08Z

Resolves #465.

eriknw · 2019-11-02T16:17:12Z

Thanks Ryan! I think I'd prefer to create our own IterationError subclassed from RuntimeError. I would also prefer to have an error message on the thrown errors. Once this is in, your Python 3 PR should do raise IterationError("some message") from exc where exc is the original StopIteration exception so we keep around the original cause.

groutr · 2019-11-04T19:16:23Z

@eriknw I'll update the python 3 PR when this gets merged into master.

groutr · 2020-01-06T21:18:05Z

@eriknw does this need anything else before it can be merged?

eriknw · 2020-01-14T14:22:36Z

This looks pretty good. As a small nit, I would prefer different error messages. For example, nth(5, [1, 2, 3]) shouldn't say "Received empty sequence", because the sequence passed to nth isn't empty.

groutr · 2020-01-15T23:15:01Z

@eriknw the exception is raised in first because nth created a sequence with no first element. In that case, nth should probably catch the error from first and generate a more informative message.

def nth(n, seq):
    """ The nth element in a sequence

    >>> nth(1, 'ABC')
    'B'
    """
    if isinstance(seq, (tuple, list, Sequence)):
        return seq[n]
    else:
        try:
            return first(itertools.islice(seq, n, None))
        except IterationError:
            raise IterationError("Element {} does not exist".format(n))

With the exception chaining in Python 3, this is handled nicely.

Catches the IterationError from first and produces a more meaningful exception message.

eriknw · 2020-03-13T20:47:52Z

LGTM!

groutr · 2020-03-17T17:57:52Z

What the errors look like now. @eriknw if this is fine with you, I believe this is also merge-able.

In [2]: itertoolz.first([])
---------------------------------------------------------------------------
IterationError                            Traceback (most recent call last)
<ipython-input-2-6b062b701b06> in <module>
----> 1 itertoolz.first([])

~/projects/toolz/toolz/itertoolz.py in first(seq)
    377     for rv in seq:
    378         return rv
--> 379     raise IterationError("Received empty sequence")
    380
    381

IterationError: Received empty sequence

In [3]: itertoolz.second([1])
---------------------------------------------------------------------------
IterationError                            Traceback (most recent call last)
~/projects/toolz/toolz/itertoolz.py in second(seq)
    388     try:
--> 389         return first(itertools.islice(seq, 1, None))
    390     except IterationError as exc:

~/projects/toolz/toolz/itertoolz.py in first(seq)
    378         return rv
--> 379     raise IterationError("Received empty sequence")
    380

IterationError: Received empty sequence

The above exception was the direct cause of the following exception:

IterationError                            Traceback (most recent call last)
<ipython-input-3-38b92c6791c0> in <module>
----> 1 itertoolz.second([1])

~/projects/toolz/toolz/itertoolz.py in second(seq)
    389         return first(itertools.islice(seq, 1, None))
    390     except IterationError as exc:
--> 391         raise IterationError("Lenth of seq is < 2") from exc
    392
    393

IterationError: Lenth of seq is < 2
In [6]: itertoolz.nth(3, iter([1, 2]))
---------------------------------------------------------------------------
IterationError                            Traceback (most recent call last)
~/projects/toolz/toolz/itertoolz.py in nth(n, seq)
    403         try:
--> 404             return first(itertools.islice(seq, n, None))
    405         except IterationError as exc:

~/projects/toolz/toolz/itertoolz.py in first(seq)
    378         return rv
--> 379     raise IterationError("Received empty sequence")
    380

IterationError: Received empty sequence

The above exception was the direct cause of the following exception:

IterationError                            Traceback (most recent call last)
<ipython-input-6-0efd83b833d7> in <module>
----> 1 itertoolz.nth(3, iter([1, 2]))

~/projects/toolz/toolz/itertoolz.py in nth(n, seq)
    404             return first(itertools.islice(seq, n, None))
    405         except IterationError as exc:
--> 406             raise IterationError("Length of seq is < %d" % n) from exc
    407
    408

IterationError: Length of seq is < 3

eriknw · 2020-03-17T18:16:59Z

Thanks again.

I don't think it's necessary or informative to chain exceptions. In fact, I may actually prefer a little more verbose by using the for trick instead of using first or next. For example, interpose and peek could use

for _ in it:
    break. # skip the first element in `it`
else:
    raise IterationError()

I suspect this is faster too.

groutr · 2020-03-17T20:01:18Z

@eriknw I've think you've done it again! 👍 Using for loops seems to be faster on my machine. I'll update the PR.

groutr · 2020-03-17T20:20:26Z

Would this mean that second would now be:

def second(seq):
    it = iter(seq)
    for item in it:
        break
    else:
        raise IterationError
    for item in it:
        return item
    else:
        raise IterationError

eriknw · 2020-03-17T20:50:29Z

Yeah, that's a good implementation of second. Here's a small variant (probably about the same speed):

def second(seq):
    it = iter(seq)
    for first_element in it:
        for second_element in it:
            return second_element
        raise IterationError('only 1 in seq blah blah')
    raise IterationError('empty seq blah blah')

toolz/itertoolz.py

groutr · 2020-03-18T00:09:03Z

@eriknw I went with the non-nested implementation for second. While they have very similar performance, the nested version seemed to display a greater variance in performance than the non-nested version on my machine.

groutr · 2020-03-18T00:12:49Z

BTW, there were small performance improvements for most of the functions involved in this PR. 👍 I can post benchmarks if needed.

groutr · 2020-03-18T00:30:23Z

All timing show the original function first followed by the modified function.

first

In [6]: %timeit itertoolz.first([1])
341 ns ± 2.47 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [14]: %timeit itertoolz.first([1])
226 ns ± 1.75 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

second

In [7]: %timeit itertoolz.second([1, 2])
449 ns ± 16.5 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [15]: %timeit itertoolz.second([1, 2])
348 ns ± 4.11 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

interpose

In [18]: %timeit itertoolz.interpose(None, [1]*50)
1.14 µs ± 39.1 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [33]: %timeit itertoolz.interpose(None, [1]*50)
1.08 µs ± 17 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

peek

In [17]: %timeit itertoolz.peek([1, 2, 3, 4])
632 ns ± 16.6 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [32]: %timeit itertoolz.peek([1, 2, 3, 4])
596 ns ± 20.6 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

peekn

In [14]: %timeit itertoolz.peekn(4, range(100))
1.35 µs ± 33.5 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [28]: %timeit itertoolz.peekn(4, range(100))
1.32 µs ± 70.6 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

nth

In [15]: %timeit itertoolz.nth(15, range(20))
1.61 µs ± 31 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [30]: %timeit itertoolz.nth(15, range(20))
1.54 µs ± 19 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

eriknw · 2020-03-18T01:31:43Z

I think this looks good.

One more thing: should we put IterationError in __init__.py?

groutr · 2020-03-18T15:32:37Z

@eriknw What do you think about putting the exceptions module in __init__.py, similar to curried and sandbox?

I'm working on a cytoolz PR that syncs the changes in this PR.

eriknw · 2021-10-28T04:02:39Z

I think this needs to go in. @groutr, since it's been a while, would you care to take another quick look at this PR and the discussion in #465?

I think to close #465, we may want to add default=no_default argument to these functions as an alternative to raising.

groutr added 2 commits October 18, 2019 17:44

Convert StopIteration into RuntimeError.

c61b35e

Update tests.

111f5ca

groutr changed the title ~~Raise RuntimeError on StopIteration~~ [WIP] Raise RuntimeError on StopIteration Oct 18, 2019

groutr added 4 commits November 4, 2019 12:41

Merge branch 'master' into catch_stopiteration

c621703

Add IterationError exception.

5a5c5ca

Use IterationError instead of RuntimeError.

40b133a

Satisfy pycodestyle.

7df34c8

groutr changed the title ~~[WIP] Raise RuntimeError on StopIteration~~ [WIP] Raise IterationError on StopIteration Nov 4, 2019

wrobell mentioned this pull request Jan 3, 2020

StopIteration from first() caught by a generator #465

Open

Produce better exception messages.

01d00ae

Catches the IterationError from first and produces a more meaningful exception message.

groutr added 2 commits March 17, 2020 12:45

Merge branch 'master' into catch_stopiteration

6f4d866

Raise from IterationError.

030b223

groutr changed the title ~~[WIP] Raise IterationError on StopIteration~~ Raise IterationError on StopIteration Mar 17, 2020

eriknw reviewed Mar 17, 2020

View reviewed changes

toolz/itertoolz.py Outdated Show resolved Hide resolved

Raise IterationError where necessary.

3fefff5

Add test for empty sequence in second.

6435b08

Add exceptions module to toolz namespace.

994820a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Raise IterationError on StopIteration #473

Raise IterationError on StopIteration #473

groutr commented Oct 18, 2019

eriknw commented Nov 2, 2019

groutr commented Nov 4, 2019

groutr commented Jan 6, 2020

eriknw commented Jan 14, 2020

groutr commented Jan 15, 2020

eriknw commented Mar 13, 2020

groutr commented Mar 17, 2020 •

edited

Loading

eriknw commented Mar 17, 2020

groutr commented Mar 17, 2020 •

edited

Loading

groutr commented Mar 17, 2020 •

edited

Loading

eriknw commented Mar 17, 2020 •

edited

Loading

groutr commented Mar 18, 2020

groutr commented Mar 18, 2020 •

edited

Loading

groutr commented Mar 18, 2020

eriknw commented Mar 18, 2020

groutr commented Mar 18, 2020 •

edited

Loading

eriknw commented Oct 28, 2021

Raise IterationError on StopIteration #473

Are you sure you want to change the base?

Raise IterationError on StopIteration #473

Conversation

groutr commented Oct 18, 2019

eriknw commented Nov 2, 2019

groutr commented Nov 4, 2019

groutr commented Jan 6, 2020

eriknw commented Jan 14, 2020

groutr commented Jan 15, 2020

eriknw commented Mar 13, 2020

groutr commented Mar 17, 2020 • edited Loading

eriknw commented Mar 17, 2020

groutr commented Mar 17, 2020 • edited Loading

groutr commented Mar 17, 2020 • edited Loading

eriknw commented Mar 17, 2020 • edited Loading

groutr commented Mar 18, 2020

groutr commented Mar 18, 2020 • edited Loading

groutr commented Mar 18, 2020

eriknw commented Mar 18, 2020

groutr commented Mar 18, 2020 • edited Loading

eriknw commented Oct 28, 2021

groutr commented Mar 17, 2020 •

edited

Loading

groutr commented Mar 17, 2020 •

edited

Loading

groutr commented Mar 17, 2020 •

edited

Loading

eriknw commented Mar 17, 2020 •

edited

Loading

groutr commented Mar 18, 2020 •

edited

Loading

groutr commented Mar 18, 2020 •

edited

Loading