Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What url to use when downloading profiles of bluesky #5246

Closed
Nicosow2 opened this issue Feb 26, 2024 · 12 comments
Closed

What url to use when downloading profiles of bluesky #5246

Nicosow2 opened this issue Feb 26, 2024 · 12 comments

Comments

@Nicosow2
Copy link

i tried but it did the error of unsupported url

@mikf
Copy link
Owner

mikf commented Feb 26, 2024

https://bsky.app/profile/HANdLE
for example https://bsky.app/profile/bsky.app

Make sure you are using the latest release (v1.26.8)
Run gallery-dl --version to check.

@Technotron21
Copy link

On something of a related note: how exactly would I be able to batch download every user on the following page like I can on twitter? Every attempt I've made doesn't even even prompt any sort of response, unlike every other error I've seen.

Also, is it possible to download reposts from an account? Given the lack of any presence for the site on the default config currently, I wouldn't be surprised if it's too new to implement yet.

@mikf
Copy link
Owner

mikf commented Feb 26, 2024

On something of a related note: how exactly would I be able to batch download every user on the following page like I can on twitter? Every attempt I've made doesn't even even prompt any sort of response, unlike every other error I've seen.

That's a bug. Or rather the result of me forgetting to add a line of code.
As a workaround, you can set blacklist to an empty string (-o blacklist=) or you write all its URLs to a file and then use that as input.

gallery-dl -o blacklist= https://bsky.app/profile/bsky.app/follows
gallery-dl -g https://bsky.app/profile/bsky.app/follows > urls
gallery-dl -i urls

Also, is it possible to download reposts from an account? Given the lack of any presence for the site on the default config currently, I wouldn't be surprised if it's too new to implement yet.

Use the /posts feed/timeline or adjust bluesky.include

The Posts, Replies, Media, etc tabs on bluesky profiles don't have real URLs, but you can select them in gallery-dl by adding the lowercase name after a profile URL (https://bsky.app/profile/bsky.app/posts).

Just https://bsky.app/profile/HANDLE alone uses the Media feed by default.

gallery-dl https://bsky.app/profile/bsky.app/posts
gallery-dl -o include=posts https://bsky.app/profile/bsky.app

@Technotron21
Copy link

Those definitely helped, but I am noticing the reposts are being separated by the users that did the original posts, so I wonder if it's possible to have them all under the same folder like with Twitter.

@mikf
Copy link
Owner

mikf commented Feb 27, 2024

Enable user metadata and use the name from that for directories.

edit: 311a21b

@Technotron21
Copy link

How do I go about doing that? I'm only really used to the more basic commands.

@mikf
Copy link
Owner

mikf commented Feb 27, 2024

Add this to your config file

        "bluesky": {
            "metadata": "user",
            "directory": ["{category}", "{user[handle]}"]
        }

If you don't have one, run gallery-dl --config-create and copy-paste this into the extractor block.


If you want reposts in their own subdirectory, you could do something like

            "directory": {
                "author['did'] != user['did']": ["{category}", "{user[handle]}", "Reposts", "{author[handle]}"],
                ""                            : ["{category}", "{user[handle]}"]
            }

edit:
Reposts are disabled by default in the next release, by the way (495c9ee)
Add "reposts": true to your bluesky config to keep them enabled.

@literallysomeone
Copy link

literallysomeone commented Feb 28, 2024

Hello. I'm wondering how is it possible to download likes from a bluesky account using gallery-dl. It seems to be a supported feature since it's stated in the Supported Sites page, but I can't figure out how to properly tell the application that's what I want to do.

The Posts, Replies, Media, etc tabs on bluesky profiles don't have real URLs, but you can select them in gallery-dl by adding the lowercase name after a profile URL (https://bsky.app/profile/bsky.app/posts).

I tried putting in something along the lines of https://bsky.app/profile/bsky.app/likes barring me using my own account, but I allows get back a [bluesky][error] API request failed (400 Bad Request) spit out at me. Heck, even putting in that example URL also spits out that error. I don't think this is necessarily a problem with Bluesky since like viewing websites still work from what I've seen (in terms of both my personal account and the example URL showing likes).

Am I being dumb and I'm just putting in the wrong URL or is this potentially an actual issue with the app?

@mikf
Copy link
Owner

mikf commented Feb 28, 2024

These URLs are the right ones, it's just that likes are kind of broken at the moment. With the current method, you can only access your own liked posts, and only when logged in. Furthermore, the feed has no proper end, meaning gallery-dl will request more liked posts indefinitely until you kill the process or Ctrl+C it.

@literallysomeone
Copy link

With the current method, you can only access your own liked posts, and only when logged in.

That's strange since I am indeed signed into the account I'm attempting to download likes from where it's pulling cookies from (Firefox in my case), so I'm not too sure why it's still erroring for me in that case. I also just tried exporting my cookies to a text file and setting them to be used instead of the cookies from my browser, but I still get [bluesky][error] API request failed (400 Bad Request). I'm not sure if you mean it technically works barring the issues of needing to be signed in and having to kill the process once done, or it's just bugged and not working at all, but on my end I can't get it to function.

I suppose I'll just wait for a proper fix for this. My main usage of this software is just scraping the likes and shares from across my online social medias, so I'm hopeful this will become given how more and more artists are moving to Bluesky these days.

@mikf
Copy link
Owner

mikf commented Feb 29, 2024

cookies

#4438 (comment)

edit: cookies don't work on bluesky. The site itself doesn't use cookies. You need to provide username and password to login, but you can remove password after the first login.

@literallysomeone
Copy link

edit: cookies don't work on bluesky. The site itself doesn't use cookies. You need to provide username and password to login, but you can remove password after the first login.

Oh, my mistake. Well now I got it all working now! Apologies for all of my comments here about this today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants