[deviantart] KeyError: 'deviation'

[ermnick]

When trying to download from scraps, the extended_fetch endpoint responds with:

{
    "error": "invalid_request",
    "errorDescription": "Invalid or expired form submission",
    "errorDetails": {
        "csrf": "missing"
    },
    "status": "error"
}

[afterdelight]

did you log in when downloading?

[ermnick]

I did. Were you not able to reproduce the issue?

[afterdelight]

please provide the example link

[ermnick]

It's not user specific. It happens with all deviantart user that has the scraps folder.
Anyways, here's a random user I found from the popular section on the front page: https://www.deviantart.com/sandara/gallery/scraps

[afterdelight]

on my end, i got this:

F:\downloader\deviant art>gallery-dl https://www.deviantart.com/sandara/gallery/scraps --verbose
[gallery-dl][debug] Version 1.23.2-dev
[gallery-dl][debug] Python 3.10.4 - Windows-10-10.0.19043-SP0
[gallery-dl][debug] requests 2.27.1 - urllib3 1.26.9
[gallery-dl][debug] Starting DownloadJob for 'https://www.deviantart.com/sandara/gallery/scraps'
[deviantart][debug] Using DeviantartScrapsExtractor for 'https://www.deviantart.com/sandara/gallery/scraps'
[deviantart][debug] Using custom API credentials (client-id 99999)
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.deviantart.com:443
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/user/profile/sandara HTTP/1.1" 200 1473
[deviantart][debug] Sleeping for 1.0 seconds
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /_napi/da-user-profile/api/gallery/contents?username=sandara&offset=0&limit=24&scraps_folder=true HTTP/1.1" 200 None
[deviantart][debug] Sleeping 2.00 seconds
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /_napi/da-browse/shared_api/deviation/extended_fetch?deviationid=858029765&username=sandara&type=art&include_session=false HTTP/1.1" 200 134
[deviantart][error] An unexpected error occurred: KeyError - 'deviation'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[deviantart][debug]
Traceback (most recent call last):
  File "C:\Python\Python310\lib\site-packages\gallery_dl\job.py", line 82, in run
    for msg in extractor:
  File "C:\Python\Python310\lib\site-packages\gallery_dl\extractor\deviantart.py", line 84, in items
    for deviation in self.deviations():
  File "C:\Python\Python310\lib\site-packages\gallery_dl\extractor\deviantart.py", line 968, in deviations
    deviation_uuid = eclipse_api.deviation_extended_fetch(
KeyError: 'deviation'

[ermnick]

Ok, so that confirms it's not just something wrong on my end.
That's the same KeyError: 'deviation' error I'm getting.

[afterdelight]

Right, waiting for mikf to fix the issue.

[mikf]

Fixed in commit 6c76b5f.

DeviantArt now requires a CSRF token for every request to their Eclipse API, which affects individual images, scraps, and followed users.

Right, waiting for mikf to fix the issue.

You could have done that from the very beginning.

[ermnick]

Can confirm the issue is fixed now 👍