Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Guzzle with SimplePIe #1052

Merged
merged 2 commits into from
Oct 6, 2018
Merged

Use Guzzle with SimplePIe #1052

merged 2 commits into from
Oct 6, 2018

Conversation

jtojnar
Copy link
Member

@jtojnar jtojnar commented Sep 20, 2018

SimplePie uses custom code for fetching resources from web but this is inconsistent and error-prone. To fix this, we will be overloading SimplePie’s File class with our own subclass using Guzzle.

Additionally, we will be no longer adding “Simplepie/X.Y.Z” string to the User-Agent header. Given that this is just an implementation detail, and that many websites actively block SimplePie.

@jtojnar jtojnar added this to the 2.19 milestone Sep 20, 2018
SimplePie uses custom code for fetching resources from web but this is
inconsistent and error-prone. To fix this, we will overload SimplePie’s
File class with our own subclass using Guzzle.
For RSS Feeds, we have been adding “Simplepie/X.Y.Z” string to the User-Agent
header. Given that this is just an implementation detail, and that many websites
actively block SimplePie, we will no longer continue to do so.
@jtojnar jtojnar merged commit c817b20 into fossar:master Oct 6, 2018
@jtojnar jtojnar deleted the simplepie-guzzle branch October 6, 2018 14:13
@akash07k
Copy link
Contributor

Can't we just expose the user agent as: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.75 Safari/537.36
Because it is the standard user agent for chrome and none of the websites will block it. and, I'm wondering, that why the autodiscovery of the feeds is broken due to this?

jtojnar added a commit that referenced this pull request Sep 16, 2019
In #1052, we introduced a class that allowed us to make SimplePie fetch feeds using Guzzle and then pass headers and content back to it. Unfortunately, SimplePie expects the headers to be lower-case and strings, whereas Guzzle returns original case and string arrays as mandated by PSR-7. This in turn broke autodiscovery.

Closes: #1130
@jtojnar
Copy link
Member Author

jtojnar commented Sep 25, 2019

Just for completeness, the user agent change did not cause this, the other change did.

And as for the user agent change, we already are somewhat Chromey. If we should pretend to be Chrome completely is a separate issue.

@akash07k
Copy link
Contributor

Just for completeness, the user agent change did not cause this, the other change did.

And as for the user agent change, we already are somewhat Chromey. If we should pretend to be Chrome completely is a separate issue.

Oh, ok, :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants