Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gallery-dl fails to download more than 2000 pages in website nozomi.la #2754

Closed
panda-cat opened this issue Jul 13, 2022 · 1 comment
Closed

Comments

@panda-cat
Copy link

[gallery-dl][debug] Version 1.22.3 - Executable
[gallery-dl][debug] Python 3.7.9 - Windows-10-10.0.19041
[gallery-dl][debug] requests 2.28.0 - urllib3 1.26.9
[gallery-dl][debug] Starting DownloadJob for 'https://nozomi.la/search.html?q=uncensored%20pussy%20-pokemon#1'
[nozomi][debug] Using NozomiSearchExtractor for 'https://nozomi.la/search.html?q=uncensored%20pussy%20-pokemon#1'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): j.nozomi.la:443
[urllib3.connectionpool][debug] https://j.nozomi.la:443 "GET /nozomi/uncensored.nozomi HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://j.nozomi.la:443 "GET /nozomi/pussy.nozomi HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://j.nozomi.la:443 "GET /index.nozomi HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://j.nozomi.la:443 "GET /nozomi/pokemon.nozomi HTTP/1.1" 200 None
[nozomi][error] An unexpected error occurred: MemoryError - . Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[nozomi][debug]
Traceback (most recent call last):
File "gallery_dl\job.pyc", line 77, in run
File "gallery_dl\extractor\nozomi.pyc", line 33, in items
File "gallery_dl\extractor\nozomi.pyc", line 208, in posts
MemoryError

@panda-cat panda-cat changed the title gallery-dl can't download more than 2000 pages in site nozomi.la Gallery-dl fails to download more than 2000 pages in website nozomi.la Jul 13, 2022
mikf added a commit that referenced this issue Jul 13, 2022
only load and use the entire 'index.nozomi' database
if there are only negative search terms
@mikf
Copy link
Owner

mikf commented Jul 13, 2022

The page number is not necessarily the problem, but the way of how applying negative search terms (-pokemon) was implemented and the huge amount of memory it required.

This got fixed and improved in commit 62cc477.

For a workaround with the current release, you can omit -pokemon from the search and use --filter "'pokemon' not in tags" instead:

gallery-dl --filter "'pokemon' not in tags" https://nozomi.la/search.html?q=uncensored%20pussy#1

@mikf mikf closed this as completed Jul 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants