Skip to content

Commit

Permalink
optimize _find_most_recently_used_file for exact profile
Browse files Browse the repository at this point in the history
When reading cookies from the browser, the user is able to give either just the browser name, or also provide profile/container information.

If an exact profile is provided, there is no need to find the latest profile with `os.walk` which is very expensive.

This change optimizes that case and the performance increase is significant (~8 sec to 0.6 sec).

```
$ time gallery-dl --config-ignore -d . -D . --cookies-from-browser FIREFOX https://imgur.com/OO4UNqJ
[cookies][info] Extracted 16 cookies from Firefox
 ./imgur_OO4UNqJ.jpg

real    0m8.429s
user    0m0.216s
sys     0m0.431s

$ time gallery-dl --config-ignore -d . -D . --cookies-from-browser FIREFOX:bgamf5r6.default-release https://imgur.com/OO4UNqJ
[cookies][info] Extracted 16 cookies from Firefox
 ./imgur_OO4UNqJ.jpg

real    0m0.456s
user    0m0.183s
sys     0m0.011s

$ gallery-dl --version
1.26.9
```
  • Loading branch information
unfo committed May 1, 2024
1 parent 7fd31aa commit c313871
Showing 1 changed file with 15 additions and 9 deletions.
24 changes: 15 additions & 9 deletions gallery_dl/cookies.py
Original file line number Diff line number Diff line change
Expand Up @@ -1001,15 +1001,21 @@ class DATA_BLOB(ctypes.Structure):


def _find_most_recently_used_file(root, filename):
# if there are multiple browser profiles, take the most recently used one
paths = []
for curr_root, dirs, files in os.walk(root):
for file in files:
if file == filename:
paths.append(os.path.join(curr_root, file))
if not paths:
return None
return max(paths, key=lambda path: os.lstat(path).st_mtime)
# if the provided root points to an exact profile path
# check if it contains the wanted filename
first_choice = os.path.join(root, filename)
if os.path.exists(first_choice):
return first_choice
else:
# if there are multiple browser profiles, take the most recently used one
paths = []
for curr_root, dirs, files in os.walk(root):
for file in files:
if file == filename:
paths.append(os.path.join(curr_root, file))
if not paths:
return None
return max(paths, key=lambda path: os.lstat(path).st_mtime)


def _is_path(value):
Expand Down

0 comments on commit c313871

Please sign in to comment.