Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terminate option does not stop extractor as expected #2863

Closed
doublewelp opened this issue Aug 25, 2022 · 2 comments
Closed

Terminate option does not stop extractor as expected #2863

doublewelp opened this issue Aug 25, 2022 · 2 comments
Labels

Comments

@doublewelp
Copy link

I'm trying to automate the download of some Hitomi.la galleries, but I've noticed that I get a few 404 errors every now and again. Once they start appearing all downloads from that point on will fail with the same error.
I'm okay with stopping the process and trying again later, but I noticed that's not what happens by default - gallery-dl will continue to download everything on my list until it reaches the end of my text file. I looked into the options listed in --help, and I changed my command to this:

gallery-dl -T 1 -i <path_to_textfile>

But gallery-dl continues to execute even with sequential 404 errors:
[download][error] Failed to download hitomi_XXX.webp
[downloader.http][warning] '404 Not Found' for 'https://ba.hitomi.la/webp/...'
[download][error] Failed to download hitomi_XXX.webp
[downloader.http][warning] '404 Not Found' for 'https://ba.hitomi.la/webp/...'
[download][error] Failed to download hitomi_XXX.webp
[downloader.http][warning] '404 Not Found' for 'https://aa.hitomi.la/webp/...'

Is this not the intended behaviour for the terminate option? If not, how can I get gallery-dl to completely stop as soon as it encounters an error for the first time?

mikf added a commit that referenced this issue Aug 26, 2022
cached values become invalid after 1-2 hours
@mikf mikf added the bug label Aug 26, 2022
@mikf
Copy link
Owner

mikf commented Aug 26, 2022

You've encountered a bug caused by gallery-dl caching certain values indefinitely, even though those expire after 1-2h. This is fixed in 946643c.

-T/--terminate works more or less like -A/--abort in that it stops when a file is already present on disk or in a download archive and its download gets skipped.

There is currently no option to stop after an error.

@doublewelp
Copy link
Author

Makes sense, I'd only start getting those errors after a couple of hours of downloading. Thank you for clarifying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants