Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AO3/ArchiveOfOurOwn] [Feature Request] Add the ability to download from the site in all its formats pdf,epub,html etc #6013

Closed
AtomicTEM opened this issue Aug 13, 2024 · 6 comments

Comments

@AtomicTEM
Copy link

There isn't any good option to simply one click download all fics from a single writer or genre/tag.

Gallery-dl has been extremely effective with other sites, so I would love to also have AO3 supported please.

mikf added a commit that referenced this issue Sep 15, 2024
@ClosedPort22
Copy link
Contributor

Looks like we've got some duplication of efforts here...
https://github.com/ClosedPort22/gallery-dl-googledrive/blob/main/extractor/archiveofourown.py

@mikf If you find anything useful, feel free to copy the code and adapt it to gallery-dl's coding conventions. The whole repo is licensed under the MIT license.

@ClosedPort22
Copy link
Contributor

ClosedPort22 commented Sep 16, 2024

I would recommend extracting the updated_at parameter from download URLs and adding that to the default archive format. From my experience, updated_at is guaranteed to change when the file content changes, even if it's just from the author changing their pen name.

Or you could use a combination of Updated (in-progress works only) and Completed (completed works only) to only detect changes to the story text.

@WarmWelcome
Copy link

Would this implementation support epub? This would be a fantastic feature since I don't believe there is any robust solution for backing up AO3. Other fanfiction sites, such as wattpad, fanfiction, fimfiction, quotev, or any of the others have no robust backup solution either, and an implementation in gdl would be a fantastic addition.

@AtomicTEM
Copy link
Author

Would this implementation support epub? This would be a fantastic feature since I don't believe there is any robust solution for backing up AO3. Other fanfiction sites, such as wattpad, fanfiction, fimfiction, quotev, or any of the others have no robust backup solution either, and an implementation in gdl would be a fantastic addition.

It does
Simply put the following this in your config. defulat is pdf
"formats": "epub",

@ClosedPort22
Copy link
Contributor

Other fanfiction sites, such as wattpad, fanfiction, fimfiction, quotev, or any of the others have no robust backup solution either, and an implementation in gdl would be a fantastic addition.

For fanfiction.net, you can check out https://github.com/JimmXinu/FanFicFare. FFN is notoriously hostile toward scrapers and downloaders and uses Cloudflare anti-bot to prevent story text from being scraped, and as a result the developer has been recommending people to use the "browser cache" feature instead. That feature relies on user interaction so mass archival isn't really feasible I'm afraid.

@WarmWelcome
Copy link

check out https://github.com/JimmXinu/FanFicFare

How in the world haven't I seen this before? Appears in no searches, never seen any recommendations for it... Well, at least I have it now. Thank you kindly for it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants