Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use GitHub Actions for testing #330

Merged
merged 18 commits into from
Feb 13, 2022
Merged

Conversation

marc1706
Copy link
Contributor

@marc1706 marc1706 commented May 30, 2021

With this PR, tests will be run on GitHub Actions instead of Travis CI. Previously disabled tests will work again.

On a side note, the pure PHP implementation without GMP does not work and causes the library to not work on PHP 7.2 and earlier if GMP is missing. In general, I think it would make sense to remove these and the gmp requirement and fully rely on PHP's integrated openssl functions. That would however have to be done in a separate issue / pull request.

Should also resolve #274

@marc1706
Copy link
Contributor Author

Copy link
Member

@Minishlink Minishlink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, thank you so much for taking the time to investigate this! When this is fixed, the library will be able to evolve much quicker.

@@ -42,7 +42,7 @@ public static function setUpBeforeClass(): void
*/
protected function setUp(): void
{
if (!(getenv('TRAVIS') || getenv('CI'))) {
if (getenv('TRAVIS') || getenv('CI')) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove that altogether? The description below is wrong, this should be run on CI (these are the tests that really matter actually)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description is wrong, yes. But the test itself has some issues running, e.g.
https://github.com/marc1706/web-push-php/runs/2808589708?check_suite_focus=true

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's why the tests are "broken" right now, and this is what needs to be fixed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, the tests are correct, this is a problem with web-push-testing-service

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

web-push-testing-service seems to be no longer maintained and has been archived: https://github.com/GoogleChromeLabs/web-push-testing-service
I've also tested around with it a bit more and even after updating it to newer dependencies it's inherently unstable and fails very often. Unless there is another way of having some form of functional testing I would actually recommend to remove these.

@marc1706
Copy link
Contributor Author

marc1706 commented Jan 6, 2022

Well, I spent my Christmas trying to get the web-push-testing-service to run. Then I tried directly interacting with browsers via chromedriver & geckodriver by using php-webdriver. That also failed, partially due to bad support for the push API in those drivers and also because even tests that work locally often times fail on the CI due to them being very resource intensive.
Push testing with the geckodriver is for example currently completely broken: mozilla/geckodriver#1687

After spending countless hours on this, I finally decided to do what we often times during testing: Implement a testing replacement for what we want to test.
I have therefore created this tool to test subscription, sending notifications, and retrieving their content:
https://github.com/marc1706/web-push-testing
It allows subscribing in the same way one would subscribe in a browser but does also support sending notifications to an endpoint and then retrieving them to check if all of this worked.
One major upside being that it's super quick and has none of the issues I have encountered with web-push-testing-service or using php-webdriver.
In addition to that, I also spent a bit of time on implementing a good test coverage to ensure that it's doing what it's supposed to be doing.
Is it the same as a browser? Of course not. Does it need to be a browser in CI? In my opinion that's a clear no. Test the API, not the overhead of trying to use an API in a complete browser.

Anyhow, enough of a rant. Builds currently pass on the branch with the same commits plus one extra commit to ensure that it's being built: https://github.com/marc1706/web-push-php/runs/4732073431?check_suite_focus=true

@marc1706
Copy link
Contributor Author

@Minishlink I was wondering if you could maybe have another look at this PR. To summarize my post above, I have basically replaced the testing against a browser with testing against a representation of the API used by browsers. Looking forward to your feedback. 👍

Copy link
Member

@Minishlink Minishlink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, very nice work on the web-push-testing stub, congrats and thank you!
Just a few nitpicks and we're good to go :)

src/Encryption.php Outdated Show resolved Hide resolved
src/Encryption.php Outdated Show resolved Hide resolved
src/Encryption.php Outdated Show resolved Hide resolved
@marc1706
Copy link
Contributor Author

marc1706 commented Feb 5, 2022

@Minishlink Addressed your comments. Updated "build" branch (this PR + commit to run tests) tests have ran through:
https://github.com/marc1706/web-push-php/actions/runs/1798861831

Copy link
Member

@Minishlink Minishlink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot!

@Minishlink Minishlink merged commit 199c83e into web-push-libs:master Feb 13, 2022
@marc1706 marc1706 deleted the add/ga-tests branch September 25, 2022 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Travis tests are failing
2 participants