Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fewer downloads of zerochan. #3669

Closed
Coca836 opened this issue Feb 18, 2023 · 4 comments
Closed

Fewer downloads of zerochan. #3669

Coca836 opened this issue Feb 18, 2023 · 4 comments

Comments

@Coca836
Copy link

Coca836 commented Feb 18, 2023

The number of downloads is less than the number shown in the gallery.
That does not change when I log in.

@Alem0012
Copy link

Set metadata to true in the config file, under the zerochan section.

@mikf
Copy link
Owner

mikf commented Feb 19, 2024

Looking at the code, enabling metadata does not make a difference in the number of extracted files.
There was a fix regarding this in the v1.26.8 release (42335ea).

@sudo-meister
Copy link

Looking at the code, enabling metadata does not make a difference in the number of extracted files. There was a fix regarding this in the v1.26.8 release (42335ea).

Thanks for the fix, but it still seems to be skipping files. For example: there are supposed to be around 5800 pictures with the tag 'Raiden Shogun', but only around 2300 get downloaded.

mikf added a commit that referenced this issue Feb 25, 2024
add 'pagination' option
@mikf
Copy link
Owner

mikf commented Feb 25, 2024

Fetching posts by parsing HTML pages is limited to 100 pages * 24 posts per page, it turns out.
(And gallery-dl doesn't support child posts like this one and would only download the top-level post)

This is now fixed in cc6b9e4 at the cost of having to guess the filename extension of each post. Every .png download in tag search mode now looks like the following when metadata is not enabled.

[downloader.http][warning] '404 Not Found' for 'https://static.zerochan.net/.full.3872603.jpg'
[download][info] Trying fallback URL #1
zerochan/Raiden Shogun/3872603.png

It is still necessary to be logged in to grab all posts. "Raiden Shogun" returns ~5500 posts when logged in and only ~3300 when not.

@mikf mikf closed this as completed Feb 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants