Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extractor.deviantart.folders causes unexpected error (KeyError - 'C7A2E3EA-9E34-DA1C-3831-35BA7697A8B5') when set to true #1302

Closed
32624647 opened this issue Feb 8, 2021 · 0 comments
Labels

Comments

@32624647
Copy link

32624647 commented Feb 8, 2021

output:

C:\Users\Augusto Vasconcellos>py -m gallery_dl -s -v "https://www.deviantart.com/sephzero/gallery"
[gallery-dl][debug] Version 1.16.4
[gallery-dl][debug] Python 3.9.1 - Windows-10-10.0.19041-SP0
[gallery-dl][debug] requests 2.25.1 - urllib3 1.26.3
[gallery-dl][debug] Starting SimulationJob for 'https://www.deviantart.com/sephzero/gallery'
[deviantart][debug] Using DeviantartGalleryExtractor for 'https://www.deviantart.com/sephzero/gallery'
[deviantart][debug] Using custom API credentials (client-id 14684)
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.deviantart.com:443
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/user/profile/sephzero HTTP/1.1" 200 1862
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/all?username=Sephzero&offset=0&limit=24&mature_content=true HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/metadata?deviationids%5B0%5D=7B922081-2384-0BAE-8BE9-2A7FB16E6765&deviationids%5B1%5D=95D4A109-AB9F-2FF3-A3F8-3D9C44B87591&deviationids%5B2%5D=7034587D-106E-8712-E893-AC1CC2911FA1&deviationids%5B3%5D=93E1CF26-E7AE-D014-4683-8DAB843668B5&deviationids%5B4%5D=2D47B766-FC4E-7A3C-7781-DB7CC3A59D99&deviationids%5B5%5D=4E6D6050-9773-A7F9-2FC1-392B940FCCB5&deviationids%5B6%5D=2166D4C6-A7A0-B3AF-6861-DDEEA6319D9A&deviationids%5B7%5D=6636DC61-5AB5-88FF-F0BA-BA3013AD75F3&deviationids%5B8%5D=E8A498E9-5547-39AA-116C-40E0435374A2&deviationids%5B9%5D=A0F45AAB-12C4-2046-4620-C6EBF8ABC23A&deviationids%5B10%5D=B3809D78-F504-4191-3274-34E789AC085F&deviationids%5B11%5D=50087343-EB1B-0B5D-5990-CAA3B933BEFB&deviationids%5B12%5D=81157293-1C26-ED65-4FB0-78DD1DD21F65&deviationids%5B13%5D=2BD5E9F0-5B65-7E99-3505-7ECD47BFD156&deviationids%5B14%5D=E156D60F-984E-7FC6-6EA6-4C2EE8C94C4E&deviationids%5B15%5D=D76C072D-2BF4-28FB-945B-6FA76DDB5F02&deviationids%5B16%5D=35838954-7716-18E1-83C7-A39BE6382085&deviationids%5B17%5D=06929C1C-F382-5A5D-C28F-4BAD7DA44D30&deviationids%5B18%5D=2D8258B1-F82F-A996-5BA8-394B7D22D688&deviationids%5B19%5D=9D68DF32-7337-3105-658F-BA3BF6FE2EA5&deviationids%5B20%5D=720158A0-F216-242C-2E18-561BEDB876F4&deviationids%5B21%5D=7E28943C-6A02-AB01-D494-55B871D6A655&deviationids%5B22%5D=45F5C716-0E30-39AF-891D-B7CFCC72924C&deviationids%5B23%5D=94305ECD-908F-ED79-0E4A-BAD41AE6D580&mature_content=true HTTP/1.1" 200 None
[deviantart][info] Collecting folder information for 'Sephzero'
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/folders?username=Sephzero&offset=0&limit=50&mature_content=true HTTP/1.1" 200 2006
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/folders?username=Sephzero&offset=50&limit=50&mature_content=true HTTP/1.1" 200 1912
[deviantart][error] An unexpected error occurred: KeyError - 'C7A2E3EA-9E34-DA1C-3831-35BA7697A8B5'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[deviantart][debug]
Traceback (most recent call last):
  File "C:\Users\Augusto Vasconcellos\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\cache.py", line 32, in __call__
    value = self.cache[key]
KeyError: 'Sephzero'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Augusto Vasconcellos\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\job.py", line 68, in run
    for msg in self.extractor:
  File "C:\Users\Augusto Vasconcellos\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\extractor\deviantart.py", line 82, in items
    for deviation in self.deviations():
  File "C:\Users\Augusto Vasconcellos\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\extractor\deviantart.py", line 1099, in _pagination
    self._folders(data["results"])
  File "C:\Users\Augusto Vasconcellos\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\extractor\deviantart.py", line 1121, in _folders
    deviation["folders"] = self._folders_map(
  File "C:\Users\Augusto Vasconcellos\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\cache.py", line 34, in __call__
    value = self.cache[key] = self.func(*args, **kwargs)
  File "C:\Users\Augusto Vasconcellos\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\extractor\deviantart.py", line 1135, in _folders_map
    folder["name"] = fmap[folder["parent"]] + "/" + folder["name"]
KeyError: 'C7A2E3EA-9E34-DA1C-3831-35BA7697A8B5'

my configuration file:

{
    "extractor":
    {
        "cookies": "C:\\Users\\Augusto Vasconcellos\\Downloads\\deviantart.com_cookies.txt",

        "deviantart":
        {
            "client-id": xxx,
            "client-secret": "xxx",
            "extra": true,
            "flat": true,
            "folders": true,
            "include": "gallery,scraps,journal",
            "journals": "html",
            "mature": true,
            "metadata": true,
            "original": true,
            "quality": 100,
            "wait-min": 0,
            "postprocessors": [{
                "name": "metadata",
                "mode": "custom",
                "directory"       : "Descriptions",
                "content-format"  : "{folders}\n{tags}\n{description}\n",
                "extension-format": "descr.txt"
            }]
        }
    }
}

Oddly enough, this error only appears when trying to rip this one "Sephzero" guy's gallery (warning: NSFW) with the "folders" option set to true. If the option is set to false or I use the "--ignore-config" option, everything works out fine.

That said, I haven't used the program to rip any more than a handful of galleries, so for all I know this error may pop up with some other DA galleries that I haven't tried to rip yet.

@mikf mikf added the bug label Feb 8, 2021
@mikf mikf closed this as completed in 23be484 Feb 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants