Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change reversed() to [::-1] for speed #13

Closed
wants to merge 1 commit into from
Closed

Change reversed() to [::-1] for speed #13

wants to merge 1 commit into from

Conversation

scragg0x
Copy link

@scragg0x scragg0x commented Oct 2, 2012

No description provided.

@jeffwidman
Copy link

Skimming your PR made me curious... [::-1] only replaces reversed().

Why did you drop the ''.join() from your version?

I haven't dug into the surrounding code so I'm unaware of the overall calling context.

@scragg0x
Copy link
Author

Ahh, such an old commit of mine. It would appear that I just neglected it. Good thing this wasn't pulled.

@davidism davidism closed this Apr 10, 2016
@davidism
Copy link
Member

I timed both versions and there was no significant change, so just going to close this and leave it as is.

@scragg0x
Copy link
Author

https://gist.github.com/scragg0x/97d2927ce20b0766710bcee43795c2a2

Interestingly, I just did a bench and reversed it much faster anyway with larger lists. 25x faster when using the example gist.

@ThiefMaster
Copy link
Member

That benchmark is incorrect - reversed() doesn't actually reverse anything when called; it just provides an iterator that starts at the end of the collection

In [1]: lst = ['x'] * 1000
In [2]: %timeit lst[::-1]
100000 loops, best of 3: 4.37 µs per loop
In [3]: %timeit list(reversed(lst))
100000 loops, best of 3: 8.27 µs per loop

With strings:

In [1]: s = 'x' * 1000
In [2]: %timeit s[::-1]
1000000 loops, best of 3: 883 ns per loop
In [3]: %timeit ''.join(reversed(s))
10000 loops, best of 3: 20.7 µs per loop

@scragg0x
Copy link
Author

@ThiefMaster I see, thanks for clarifying. I figured I missed something to have such a large disparity.

@jeffwidman
Copy link

Now that I understand what's happening here, I'd vote to reconsider and merge this.

BTW, thanks @ThiefMaster for taking the time to explain, I hadn't realized that reversed() only returns an iterator.

@davidism
Copy link
Member

After a (very silly) discussion about this micro-optimization, http://chat.stackoverflow.com/transcript/message/29868921#29868921, it looks like it would be better to replace the whole thing with the following, which outperforms everything else on py2 and py3 and doesn't require compat imports.

import struct

to_int64 = struct.Struct('>Q').pack

def int_to_bytes(num):
    return to_int64(num).lstrip(b'\x00')

timeit int_to_bytes(ts)
1000000 loops, best of 3: 398 ns per loop

@davidism
Copy link
Member

090e9e2

@davidism davidism added this to the 1.0.0 milestone Sep 28, 2018
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Nov 10, 2018
Version 1.1.0
-------------

Released 2018-10-26

-   Change default signing algorithm back to SHA-1. (`#113`_)
-   Added a default SHA-512 fallback for users who used the yanked 1.0.0
    release which defaulted to SHA-512. (`#114`_)
-   Add support for fallback algorithms during deserialization to
    support changing the default in the future without breaking existing
    signatures. (`#113`_)
-   Changed capitalization of packages back to lowercase as the change
    in capitalization broke some tooling. (`#113`_)

.. _#113: pallets/itsdangerous#113
.. _#114: pallets/itsdangerous#114


Version 1.0.0
-------------

Released 2018-10-18

YANKED

*Note*: This release was yanked from PyPI because it changed the default
algorithm to SHA-512. This decision was reverted in 1.1.0 and it remains
at SHA1.

-   Drop support for Python 2.6 and 3.3.
-   Refactor code from a single module to a package. Any object in the
    API docs is still importable from the top-level ``itsdangerous``
    name, but other imports will need to be changed. A future release
    will remove many of these compatibility imports. (`#107`_)
-   Optimize how timestamps are serialized and deserialized. (`#13`_)
-   ``base64_decode`` raises ``BadData`` when it is passed invalid data.
    (`#27`_)
-   Ensure value is bytes when signing to avoid a ``TypeError`` on
    Python 3. (`#29`_)
-   Add a ``serializer_kwargs`` argument to ``Serializer``, which is
    passed to ``dumps`` during ``dump_payload``. (`#36`_)
-   More compact JSON dumps for unicode strings. (`#38`_)
-   Use the full timestamp rather than an offset, allowing dates before
    2011. (`#46`_)
-   Detect a ``sep`` character that may show up in the signature itself
    and raise a ``ValueError``. (`#62`_)
-   Use a consistent signature for keyword arguments for
    ``Serializer.load_payload`` in subclasses. (`#74`_, `#75`_)
-   Change default intermediate hash from SHA-1 to SHA-512. (`#80`_)
-   Convert JWS exp header to an int when loading. (`#99`_)

.. _#13: pallets/itsdangerous#13
.. _#27: pallets/itsdangerous#27
.. _#29: pallets/itsdangerous#29
.. _#36: pallets/itsdangerous#36
.. _#38: pallets/itsdangerous#38
.. _#46: pallets/itsdangerous#46
.. _#62: pallets/itsdangerous#62
.. _#74: pallets/itsdangerous#74
.. _#75: pallets/itsdangerous#75
.. _#80: pallets/itsdangerous#80
.. _#99: pallets/itsdangerous#99
.. _#107: pallets/itsdangerous#107
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 9, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants