Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix RefreshTokenException('Could not find the Guest token in HTML') #2

Merged
merged 1 commit into from
Jan 26, 2022

Conversation

jarle
Copy link
Owner

@jarle jarle commented Jan 26, 2022

ABOUT THE PROBLEM

This problem has recently begun to occur on some environments.
This doesn't happen every time, so if you are lucky, you don't get the
error.

The cause is literally literally that twint could not find the Guest
token in HTML.
Actually, sometimes token isn't included in HTML recently.

#!/usr/bin/env python3
# This program is WTFPL.
import requests

res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])
twint require the result of running the above code is })();</script><script nonce="VALUE">document.cookie = decodeURIComponent("gt=VALUE; Max-Age=VALUE; Domain=.twitter.com; Path=/; Secure");</script>.
However, sometimes the result is only })();</script> and missing the
Guest token.

ABOUT THE SOLUTION

In this patch, twint get the Guest token from
https://api.twitter.com/1.1/guest/activate.json if could not find the
one.
The author referred to the code of gallery-dl:
https://github.com/mikf/gallery-dl/blob/47eae4c393f09937a5dbcc2cb978702fb173e747/gallery_dl/extractor/twitter.py#L780-L783

Author's note:

I don't understand session of requests, so the code may be not good.
I hope someone rewrite the patch better and create a pull request.

This commit was adopted from:
twintproject#1320 (comment)

Closes twintproject#1320.

ABOUT THE PROBLEM

This problem has recently begun to occur on some environments.
This doesn't happen every time, so if you are lucky, you don't get the
error.

The cause is literally literally that twint could not find the Guest
token in HTML.
Actually, sometimes token isn't included in HTML recently.

    #!/usr/bin/env python3
    # This program is WTFPL.
    import requests

res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])
twint require the result of running the above code is })();</script><script nonce="VALUE">document.cookie = decodeURIComponent("gt=VALUE; Max-Age=VALUE; Domain=.twitter.com; Path=/; Secure");</script>.
However, sometimes the result is only })();</script> and missing the
Guest token.

ABOUT THE SOLUTION

In this patch, twint get the Guest token from
https://api.twitter.com/1.1/guest/activate.json if could not find the
one.
The author referred to the code of gallery-dl:
https://github.com/mikf/gallery-dl/blob/47eae4c393f09937a5dbcc2cb978702fb173e747/gallery_dl/extractor/twitter.py#L780-L783

Author's note:

> I don't understand session of requests, so the code may be not good.
> I hope someone rewrite the patch better and create a pull request.

This commit was adopted from:
twintproject#1320 (comment)

Closes twintproject#1320.
@jarle jarle merged commit 09ae37d into jarle:master Jan 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment