Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[documentation] custom site support #1658

Open
mikf opened this issue Jun 29, 2021 · 2 comments
Open

[documentation] custom site support #1658

mikf opened this issue Jun 29, 2021 · 2 comments

Comments

@mikf
Copy link
Owner

mikf commented Jun 29, 2021

Manually adding certain custom domains to the list of gallery-dl's recognized URLs is possible since version 1.17.0, but hasn't been documented anywhere except in the commit message of 595bdaa.

To describe this functionality:

gallery-dl can support any site that fits into one of its "basecategories" (gelbooru_v01, gelbooru_v02, moebooru, philomena, foolfuuka, foolslide, mastodon, shopify). Several sub-sites are already supported by default (see the bottom entries in supportedsites.md), but it is also possible to get it to recognize even more with the right config entries.

The general way is to add a new entry with arbitrary name (this will be used as category) inside a block named after the basecategory, which in turn is inside the "extractor" block. These entries must specify the root URL for this site, and can optionally set a custom pattern that gets used to match the site's domain. pattern must not contain a capture group.

For example the following would add support for two additional Gelbooru Beta 0.1.11 sites as well as two Shopify instances in addition to what is already supported by default:

{
    "extractor": {
        "gelbooru_v01": {
            "rozenmaidenbooru": {"root": "http://rm.booru.org"},
            "azureblade"      : {"root": "https://tab.booru.org"}
        },
        "shopify": {
            "hiutdenim": {"root": "https://hiutdenim.co.uk"},
            "ptfoods"  : {"root": "https://partakefoods.com",
                          "pattern": "(?:www\\.)?partakefoods\\.com"}
        }
    }
}

This could definitely be described / documented in a better, easier to understand way, but I'm not very good at this. So if anyone could help with improving this, I'd be much obliged.
I'm also not entirely sure where this information should go. Maybe in the Configuration section of the README? Maybe as its very own section below Authentication?

@Hrxn
Copy link
Contributor

Hrxn commented Jun 30, 2021

There is also still this: https://github.com/mikf/gallery-dl/wiki

Maybe that's a good point to start. I was a bit reluctant at recommending first, because everything that is on that wiki is not part of the repository itself (a wiki is its own repo, basically - still publicly accessible via git, of course), and proper documentation can and should absolutely be part of the reference repo, I think.

But..
This can be considered a benefit: A starting point, user-centric guides and how-tos, like first steps to get the hang of using gallery-dl. Simply to address the issue of getting the same questions in the discussions, or worse, as actual project issues.

The wiki is basically just a collection of pages (searchable, thankfully), with one home page as the initial landing point, which basically can be used to link to all topic-specific pages. There's also a navigation tree at the right hand side, and it apparently is possible to group and nest the actual pages there a bit (An example of a wiki which does this that I've found: uBO)

In short, having this wiki as a separate place means that it can be considered as a draft version for "proper" documentation, basically. So, it does not have to meet any standard of formal writing, and it does not even have to be stylistically consistent (only factually correct, of course) or particularly concise. Just a stick in the ground from which one can start. And as soon as any content on that wiki meets a required threshold, wherever that may be, it can still be lifted and added to the main repo, solely at mikf's discretion.

@SpiffyChatterbox
Copy link

Draft of this added here: Custom Site Support ‐ HowTo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants