Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Nozomi] Database feature doesn't work properly for Nozomi #1523

Closed
oxi7589 opened this issue May 2, 2021 · 2 comments · Fixed by #1529
Closed

[Nozomi] Database feature doesn't work properly for Nozomi #1523

oxi7589 opened this issue May 2, 2021 · 2 comments · Fixed by #1529
Labels

Comments

@oxi7589
Copy link

oxi7589 commented May 2, 2021

Looks like Nozomi submissions are not uniquely represented in the database, making this feature largely useless for this extractor. The problem is reproducible on current stable version, 1.17.3

Config:

    "nozomi": {
        "archive" : "./db/nozomi.sqlite3",
        "filename": "{postid}-{num} {dataid}.{extension}"
    },

Sample output (assuming ./db/nozomi.sqlite3 is blank or absent):

PS U:\gallery-dl> .\gallery-dl "https://nozomi.la/search.html?q=pixiv_id_4169977"
* .\gallery-dl\nozomi\pixiv_id_4169977\27699823-1 8e3c15aa266fc581f72d24af68ffd26770c948e4ef8effa8610c229cf540f15b.png
# .\gallery-dl\nozomi\pixiv_id_4169977\27699823-2 75275890de3e21cbe23bc5021a5085c2523d34a9eb3b85586214e5405441d642.png
# .\gallery-dl\nozomi\pixiv_id_4169977\27699823-3 b50aaae9ad690f90b1d9f0a0c91333a974d2dd012f3b6c5ed155988ef39d7fe2.png
* .\gallery-dl\nozomi\pixiv_id_4169977\27652373-1 82af69c9438597b190ee0b43e6d74a2a8e313ebeb997d66ca40944083cf08367.mp4
* .\gallery-dl\nozomi\pixiv_id_4169977\27616177-1 0b1c1ae162ba1085808c6043fe0992aed7407f0723421f002157488c0982d44c.png
# .\gallery-dl\nozomi\pixiv_id_4169977\27616177-2 cd911f8e297cfe85f70974bd575f443bdfa475343969cd36b59920505fac1d5e.png
# .\gallery-dl\nozomi\pixiv_id_4169977\27616177-3 763fd9c528616be3428e092c90c0c4e197833019080ab68dc9e5b0d81adebb52.png
# .\gallery-dl\nozomi\pixiv_id_4169977\27616177-4 d416ab9c958f8b267087c2d981ec39b299ef558c3ff890fee0b0baea9e7217bd.png
# .\gallery-dl\nozomi\pixiv_id_4169977\27616177-5 0689d6ec7f93cbb848db2ed19b6fa57d8015fad6fb3bb85c6c2f1095e3a8b12a.png
* .\gallery-dl\nozomi\pixiv_id_4169977\27549156-1 459245d1401e62476bd4841cb638a30b4708af7982cf9a3a791a5a1123a902d4.png
# .\gallery-dl\nozomi\pixiv_id_4169977\27549156-2 596020ee52ec01fb13651a58b1b1979c66c58fe0cb9d163af9a9a58f2068d133.png
...

Notice how only the 1st file in a sequence is processed properly.

@Hrxn
Copy link
Contributor

Hrxn commented May 3, 2021

Might be caused by the archive format setting for Nozomi.

Note: Filename settings are entirely irrelevant here.

I think in your example, NozomiSearchExtractor gets used, which has a default archive format of "t_{search_tags}_{postid}"

Maybe that's the problem, because if I recall it correctly, Nozomi is a site that breaks with typical booru conventions, so that one post can have multiple entries/images?

Could you try it with a config like this, for example?

"nozomi": {
        "search": {
              "archive" : "./db/nozomi.sqlite3",
              "filename": "{postid}-{num} {dataid}.{extension}",
              "archive-format": "t_{search_tags}_{postid}_{dataid}"
         }
    },

@oxi7589
Copy link
Author

oxi7589 commented May 4, 2021

"archive-format": "t_{search_tags}_{postid}_{dataid}"

Yes, this solves the problem. Maybe this setting should be the default for this site, since the current default is not exactly functional.

@mikf mikf closed this as completed in #1529 May 4, 2021
mikf added a commit that referenced this issue May 4, 2021
… so they actually work for posts with more than 1 file.

(fixes #1523)
@mikf mikf added the bug label May 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants