Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Twitter, not all original tweets (non retweets) are downloaded. #186

Closed
indrakaw opened this issue Mar 10, 2019 · 6 comments
Closed

Twitter, not all original tweets (non retweets) are downloaded. #186

indrakaw opened this issue Mar 10, 2019 · 6 comments

Comments

@indrakaw
Copy link

indrakaw commented Mar 10, 2019

The retweets config is turned to false and videos is true. This is snip of gallery-dl.conf:

        "twitter": {
            "retweets": false,
            "videos": true
        }

Source: https://twitter.com/tanosiie
OS: Windows
gallery-dl is installed via pip3.

This is the list with retweets off:

C:\Users\Acer\projects
λ ls -1 gallery-dl\twitter\tanosiie\
1101496621529935873_1.jpg
1101507236575117312_1.jpg
1101772722177114113_1.jpg
1101868990538768385_1.jpg
1101877309760393216_1.jpg
1102945454113734657_1.mp4
1103155766624641027_1.jpg
1103670923096293376_1.jpg
1103679590000582656_1.jpg
1103683556759359489_1.jpg
1103687565616414721_1.jpg
1103691423717576704_1.jpg
1103696907111550977_1.jpg
1103794700547317760_1.jpg
1104039294388199425_1.jpg
1104047893042749440_1.mp4

There are images that originally from that twitter account and not downloaded. I had retried 3 times and the same result. Here the some of them:
https://twitter.com/tanosiie/status/959671765189369857
https://twitter.com/tanosiie/status/1032677274527531009

Addendum:
These two tweets are downloaded before I set retweets to false.

@mikf
Copy link
Owner

mikf commented Mar 10, 2019

Use https://twitter.com/tanosiie/media as URL.

There is a difference between the regular timeline, which includes normal tweets and retweets, and the media timeline, which only has tweets with images/videos. Additionally there is the problem that Twitter only allows you to get the last 2000(?) tweets or so. There is really no good way to get a list of older tweets.

@indrakaw
Copy link
Author

indrakaw commented Mar 10, 2019

Twitter only allows you to get the last 2000(?) tweets or so.

Wild opinion: tweets pagination.

@mikf
Copy link
Owner

mikf commented Mar 10, 2019

That's what's already happening, but the pagination results stop after a certain point, even if there are technically more tweets available. Even the official API has such a limitation:

https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-user_timeline.html

This method can only return up to 3,200 of a user's most recent Tweets.

It might be possible to use advanced search and specify a time interval to get older tweets like that, but I don't know how doable this would be.

@indrakaw
Copy link
Author

This kind of function is basically what we wanted for years. I'm expecting more on this. 🙇

@indrakaw
Copy link
Author

I tried. It downloaded up to 1000 images.

@mikf
Copy link
Owner

mikf commented Dec 3, 2022

This is fixed with 3346f58 and all the changes/improvements following that.

@mikf mikf closed this as completed Dec 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants