Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't use .get() with defaultdicts #9918

Merged

Conversation

correctmost
Copy link
Contributor

Type of Changes

Type
βœ“ πŸ”¨ Refactoring

Description

ASTWalker.walk was calling get ~1.42 million times when linting yt-dlp. This is a small speed-up that also helps unclutter profiler output.

timeit data

$ python -m timeit -s 'from collections import defaultdict; visit_events = defaultdict(list)' -n 1000000 'x=visit_events.get("x", ())'
1000000 loops, best of 5: 50.7 nsec per loop

$ python -m timeit -s 'from collections import defaultdict; visit_events = defaultdict(list)' -n 1000000 'x=visit_events["x"]'
1000000 loops, best of 5: 29.4 nsec per loop

Stats

Before

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  2123486    0.529    0.000    0.529    0.000 {method 'get' of 'dict' objects}
Command Mean [s] Min [s] Max [s] Relative
pylint --recursive=y . 33.597 Β± 0.157 33.444 33.839 1.00

After

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   697806    0.204    0.000    0.204    0.000 {method 'get' of 'dict' objects}
Command Mean [s] Min [s] Max [s] Relative
pylint --recursive=y . 33.536 Β± 0.134 33.308 33.731 1.00

This provides a small speed-up on large codebases.
@jacobtylerwalls jacobtylerwalls added performance Skip news πŸ”‡ This change does not require a changelog entry labels Sep 7, 2024
Copy link
Contributor

github-actions bot commented Sep 7, 2024

πŸ€– According to the primer, this change has no effect on the checked open source code. πŸ€–πŸŽ‰

This comment was generated for commit 8a1ec04

@Pierre-Sassoulas Pierre-Sassoulas merged commit b7eeccb into pylint-dev:main Sep 7, 2024
44 of 46 checks passed
@correctmost correctmost deleted the cm/remove-defaultdict-gets branch September 7, 2024 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Skip news πŸ”‡ This change does not require a changelog entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants