Error when downloading DeviantArt Journal and Status Updates

[AtomicTEM]

[deviantart][error] API responded with 404 Not Found
[deviantart][error] Unexpected API response: {'error': 'invalid_request', 'error_description': 'Api endpoint not found.', 'status': 'error'}

Getting error above when trying to download posts/journals, also cannot seem to download status updates whatsoever yet I don't get any error.

[hpeter1994]

I would like to add that the same error comes up if i want to download anything from my watchlist. It started 2-3 days ago.

[gallery-dl][debug] Version 1.27.1 - Executable (stable/linux)
[gallery-dl][debug] Python 3.10.12 - Linux-6.9.9-arch1-1-x86_64-with-glibc2.39
[gallery-dl][debug] requests 2.31.0 - urllib3 2.2.2
[gallery-dl][debug] Configuration Files ['gallery-dl.conf', 'gallery-dl.conf']
[gallery-dl][debug] Starting DownloadJob for 'https://www.deviantart.com/notifications/watch'
[deviantart][debug] Using DeviantartWatchExtractor for 'https://www.deviantart.com/notifications/watch'
[deviantart][debug] Using custom API credentials (client-id 14xx6)
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.deviantart.com:443
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/browse/deviantsyouwatch?limit=50&offset=0&mature_content=true HTTP/11" 404 96
[deviantart][debug] {"error":"invalid_request","error_description":"Api endpoint not found.","status":"error"}
[deviantart][error] API responded with 404 Not Found
[deviantart][error] Unexpected API response: {'error': 'invalid_request', 'error_description': 'Api endpoint not found.', 'status': 'error'}
[deviantart][info] No results for https://www.deviantart.com/notifications/watch

[mikf]

The latest API update (July 1st 2024) removed to ton of /browse/… endpoints, some of which were used by gallery-dl (#):

• /browse/categorytree: Fetch category information for browsing
• /browse/deviantsyouwatch (#): Get deviations of deviants you watch
• /browse/hot: Browse whats hot deviations
• /browse/morelikethis: Fetch MoreLikeThis result for a seed deviation
• /browse/newest (#): Browse newest deviations
• /browse/popular (#): Browse popular deviations
• /browse/posts/deviantsyouwatch (#)
• /browse/recommended: Browse recommended deviations
• /browse/undiscovered: Browse undiscovered deviations
• /browse/user/journals (#): Browse journals of a user

/user/statuses/ is also gone.

There is an alternative for journals and statuses, but I can't find one for deviantsyouwatch or popular.

[hpeter1994]

By the looks of it deviantart silently restored the /browse/deviantsyouwatch API endpoint, my old script works again.

[a-washing-machine]

Not sure if I should put this here or make a new post for it, but journals seem to be broken again?

It's just that they are broken in a new and creative way I wouldn't have noticed if I didn't do a comparison download on a few test-galleries whenever I switch to a newer gallery-dl version, and noticed a lot of missing images which otherwise would've been linked in journals.

I've tried both gallery-dl 1.27.3 and 1.27.4 just today.

The downloaded files I get are missing the entire text of the journal, they just have some of the formatting and a journal title. Not sure how that happened. On August 20th it was fine using 1.27.3, now it isn't, so it's a site change?

Or is this something on my end? Can somebody else confirm the error?

[mikf]

@a-washing-machine
That's a new bug(?) in DA's API. The HTML text content of journals is returned as an empty string:

{
  "html": ""
}

[a-washing-machine]

Well, let's hope deviantArt fixes that themselves, then... -_-

<mindless ramble that can be ignored>

Ooof, I could imagine some people running gallery-dl during the next few [unit of time interval until fix] and not noticing the error... then have all those empty journals in their download directory they only discover months (or years!) later, mixed in with the rest.

After all, the html files being downloaded still have the same file names as the actual journal would, so it wouldn't re-download those particular journals correctly even after gallery-dl was fixed, and of course there would be no easy way to sort out broken journals after the fact. Maybe filtered by file size, I guess.

Lucky for me that I caught this upfront, not so lucky for anybody else who doesn't. I sure know that would cause me some frustration. :/
</mindless ramble that can be ignored>