CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)

[linglin506]

This error will be reported from 0:00 on September 18, 2020. It is found through debug that the https://twitter.com/i/search/timeline page does not exist. Can you fix it?By changing the IP, it can be determined that Twitter is not banning the IP

CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 1.0 secs

[sukioral]

I experienced the same problem today as well. Yesterday was still fine.

[zhixideyu]

今天用就有事

[hcanalesmx]

I am having the same issue

[bigslick87]

Glad to see I am not the only one. I just discovered this today and was excited to use it. I hope they can fix it ASAP

[hj5992]

getting the same error!

[gab-santi]

Getting the same error as well. Hope it gets fixed ASAP

[tmantynen]

I guess Twitter has deleted https://twitter.com/i/search/timeline

Twint is not the only Twitter scraper having the same problem. See: jonbakerfish/TweetScraper#92

[Buaasinong]

bad news

[zhixideyu]

我想Twitter已经删除了https://twitter.com/i/search/timeline

Twint并不是唯一遇到相同问题的Twitter抓取工具。请参阅：jonbakerfish / TweetScraper＃92

Does that mean the project is going to fail

[yemen2016]

I am also having the same issue. The system was working fine yesterday!
I hope the problem will be solved soon.

[Sebastokratos42]

I guess Twitter has deleted https://twitter.com/i/search/timeline

Twint is not the only Twitter scraper having the same problem. See: jonbakerfish/TweetScraper#92

Problem also appears using GetOldTweets3.

[shelu16]

https://twitter.com/search?f=live&q=bug%20min_retweets%3A100%20until%3A2020-09-18%20since%3A2020-09-11&src=typed_query but it can work can we do this search here

[vishal7892]

Even I am facing the same issue. Any luck?

[TryAndErr0r]

We need to find a new endpoint. I loaded the site and saw this link
New Api link?.
It is important to note that if you copy the request and past it to notepad you can see that the request is provided with headers and you can see that it provides a x-csrf-token and x-guest-token which are required. You can find these tokens in the cookie fields "gt" and "ct0".

This is somewhat the official twitter api and you need a bearer token to access this. The bearertoken is provided in the https://abs.twimg.com/responsive-web/client-web/main.6b1bdbc5.js file. Just search for "AAAAAA" and you can find it.

Now you can craft a GET request in postman with the x-csrf-token, the x-guest-token and the bearer token for authentification.

[yunusemrecatalcam]

Booom! I found a new endpoint, implemented the new parser for it. Check this out! #917
@pielco11

[estebanpdl]

Having the same issue as well. Just tried yesterday (2020-09-17) and everything worked fine. I tried Twint today (2020-09-18) and I got the same loop.

CRITICAL:root:twint.get:User:
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 15 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 15 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 27.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 64.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 125.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 216.0 secs

[boshez]

I thought I was the only person who was facing this. Hope they fix this.

[hasobi]

i'm having the same problem here.

[guilherme-sal]

Same problem here. Yesterday was working fine...

[jeremy1392]

Everything is working except to fetch tweets. Yesterday was working fine

[ivanlewin]

Hi I just want to add that I also had no problems yesterday but today I can't get any data.

Also, I was trying other tool to see if it worked and they seem to be having similar issues

[aradiayu]

I think I'm the only one facing this 'twint' problem. I hope it is fixed soon

[Dbroug01]

I am also facing this issue

[tmantynen]

Booom! I found a new endpoint, implemented the new parser for it. Check this out! #917
@pielco11

Big thanks to @yunusemrecatalcam

I'm not using Twint but instead my own similar Twitter scraper program writen in PHP. So I can't comment directly on @yunusemrecatalcam 's changes to Twint.

Anyway. Couple of days ago my own scraper stopped working just like all Twitter scrapers using i/search/timeline endpoint did. Following @yunusemrecatalcam 's example, I implemented similar changes to my program that @yunusemrecatalcam did to Twint in PR #917.

I haven't had the time to finished all changes, but preliminary impressions are mixed. The new mobile endpoint appears to function similarly to the old one, but Twitter appears to return a lot fewer tweets as a result to queries than previously. Queries that resulted hundreds of tweets earlier now result just tens of tweets. I can't yet comment on speed or rate limits etc. And again, these are just preliminary impressions.

(Edit: I incorrectly tagged @pielco11 when I should have tagged @yunusemrecatalcam. My apologies.)

[adrival13]

Same problem
CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 15 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 15 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 27.0 secs
CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)
sleeping for 64.0 secs

[rayhannr]

got the same problem though fetching followers and followings are just fine. anyway, is there any good alternative to twint?

[shizia]

今天用就有事

以前是都能用吗？才刚发现这个就发现好像出问题了

[lukaspistelak]

they changed end point url to 👍

https://twitter.com/explore/

[debjyoti003]

after installing the new one, I am getting

~/.local/lib/python3.7/site-packages/twint/format.py in Tweet(config, t)
21 output = output.replace("{hashtags}", ",".join(t.hashtags))
22 output = output.replace("{cashtags}", ",".join(t.cashtags))
---> 23 output = output.replace("{replies}", t.replies_count)
24 output = output.replace("{retweets}", t.retweets_count)
25 output = output.replace("{likes}", t.likes_count)

TypeError: replace() argument 2 must be str, not int

[debjyoti003]

Is there any specification for installing it on ubuntu?

[wildangunawan]

Is there any specification for installing it on ubuntu?

I have no problem installing it on Ubuntu 16.04 under Conda environtment. I used this command:

pip install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

[vesuvius13]

I got the same issue and the previous solution doesn't work now

create a new virtual environment in anaconda and install python 3.6 in there.
conda create -n yourenvname python=3.6 anaconda
Then, follow these steps:-

git clone https://github.com/twintproject/twint.git
cd twint
pip3 install . -r requirements.txt
pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint

Then try to run the script:-

!pip install nest_asyncio
import nest_asyncio
nest_asyncio.apply()

import twint
config = twint.Config()
config.Search = "Dogecoin"
config.Limit = 10
config.Store_csv = True
config.Output = "doge.csv"
twint.run.Search(config)

[romellfudi]

Hi everyone, when I run the following command:

pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

Does it work on MAC OS?
I have been trying to use it since 2019

[lemiemie]

Hi everyone, when I run the following command:

pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

Does it work on MAC OS?
I have been trying to use it since 2019

@romellfudi Yes it works on my macOS. It gave me error message at the beginning but after I deleted the "--user" part it works

[zachlagden]

@eabanoz @jiwoochoi-kor @hnodaro
Uninstall the previous installation of twint using the following command

pip3 uninstall twint

And then reinstall twint using the following command

pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

This should work.

thank you so much!!! it worked

worked for me

[hglite69]

Me Too ^-^

[jpblopez]

@eabanoz @jiwoochoi-kor @hnodaro
Uninstall the previous installation of twint using the following command

pip3 uninstall twint

And then reinstall twint using the following command

pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

This should work.

thank you so much!!! it worked

Also worked for me, as of September 2021

[asdrubalivan]

^ This worked for me too!

[tsaaii]

upgrading twint works.

[yehfanju]

pip3 uninstall twint

@tsaaii can you normally scrape all the tweets without encountering limit/date constraints? Thanks!

[tsaaii]

it limits tweets on time line as mentioned in documentation. I haven't seen any date constraints so far.

[yehfanju]

@tsaaii Thanks for your reply! Did you clone the git repo as well or just pips upgrade? I can only get a few tweets after uninstall and reinstall.

[tsaaii]

clone the repo first, upgrade it next. It works for me. Make sure python version is 3.6.

[yehfanju]

@tsaaii Thanks! the 3.6 version and upgrade works for me but only for the latest tweets. Twint does not return anything when specifying since and until. Just want to know if you also cannot retrieve historical tweets or it's just me!

[olivermj3]

This issue fixed after running this command:
pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

However, following this upgrade, it resumed collect briefly (16 tweets recovered then the following print out before the error:

[!] No more data! Scraping will stop now.
found 0 deleted tweets in this search.

RefreshTokenException Traceback (most recent call last)
/var/folders/7z/866jzymn52zcbnymflj7xhbw0000gn/T/ipykernel_6392/906502173.py in
7 filename = str(str(target)+".csv")
8 c.Output = filename
----> 9 twint.run.Search(c)

~/.local/lib/python3.8/site-packages/twint/run.py in Search(config, callback)
408 config.Followers = False
409 config.Profile = False
--> 410 run(config, callback)
411 if config.Pandas_au:
412 storage.panda._autoget("tweet")

~/.local/lib/python3.8/site-packages/twint/run.py in run(config, callback)
327 raise
328
--> 329 get_event_loop().run_until_complete(Twint(config).main(callback))
330
331

~/.local/lib/python3.8/site-packages/twint/run.py in init(self, config)
34 # USAGE : to get a new guest token simply do self.token.refresh()
35 self.token = token.Token(config)
---> 36 self.token.refresh()
37 self.conn = db.Conn(config.Database)
38 self.d = datelock.Set(self.config.Until, self.config.Since)

~/.local/lib/python3.8/site-packages/twint/token.py in refresh(self)
67 else:
68 self.config.Guest_token = None
---> 69 raise RefreshTokenException('Could not find the Guest token in HTML')

RefreshTokenException: Could not find the Guest token in HTML

[fwgood]

已收到您的邮件，会尽快回复

[rangersmyth74]

!pip install twint
!pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
import twint
import nest_asyncio
c = twint.Config()
c.Username = "github"
c.Search = "code"
nest_asyncio.apply()
twint.run.Search(c)

Hey all... I am having this problem now in Dec 2021. I have tried all the fixes, but none worked for me. I used Google Cloud Shell and I was able to get twint to work, but then I was getting the same errors.

I was wondering how to use the above date? Do I run it on the command line? Just wondering.

Thanks.

[machlovi]

I am facing the same issue ..I have tried all the solutions. Can anyone help me

[fwgood]

已收到您的邮件，会尽快回复

[ArshdeepSandhu]

@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command

pip3 uninstall twint

And then reinstall twint using the following command

pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

This should work.

Thank you very much Himanshu! Was really struggling with this.

[ArshdeepSandhu]

I am facing the same issue ..I have tried all the solutions. Can anyone help me

Have you tried what Himanshu suggested?

[skswldndi]

@eabanoz @jiwoochoi-kor @hnodaro Uninstall the previous installation of twint using the following command

pip3 uninstall twint

And then reinstall twint using the following command

pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

This should work.

Thank you very much Himanshu! Was really struggling with this.

This worked for me! Thanks.

[fwgood]

已收到您的邮件，会尽快回复

[palavaryunusemre]

@eabanoz @jiwoochoi-kor @hnodaro Aşağıdaki komutu kullanarak önceki twint kurulumunu kaldırın

pip3 uninstall twint

Ardından aşağıdaki komutu kullanarak twint'i yeniden yükleyin

pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint

Bu çalışmalı.

Himanshu'ya çok teşekkür ederim! Bu konuda gerçekten zorlanıyordu.

this didn't work for me

[fwgood]

已收到您的邮件，会尽快回复

[palavaryunusemre]

已收到您的邮件，会尽快回复

thanks