Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add github action for tests #623

Merged
merged 9 commits into from
Sep 27, 2022
Merged

Conversation

salilmishra23
Copy link
Contributor

@salilmishra23 salilmishra23 commented Sep 12, 2022

This PR adds github actions for tests, resolves #614.

It currently fails for tests/remote/test_firefox.py::test_tinderbox_scraper test as for some arguments the request timesout (Given enough time it returns HTTP 504).
It is also happening locally.
Logs - https://github.com/mozilla/mozdownload/actions/runs/3041139681/jobs/4897959420#step:5:366

{'branch': 'mozilla-central', 'platform': 'linux64'},
{'branch': 'mozilla-central', 'platform': 'win32'},
{'branch': 'mozilla-central', 'platform': 'win64'},

Let me know if you have suggestions.

Copy link
Contributor

@whimboo whimboo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's great to see that we can make progress here! Some suggestions regarding to your changes:

  1. Please see my inline comment so we can have a broader coverage across differen Python releases.
  2. Also remove the configurations for Travis CI and AppVeyour which are not necessary anymore given that GitHub actions now handle all platforms.
  3. If we can extend the timeout for the remote tinderbox tests then I'm happy with a change there as well. Otherwise feel free to mark them as skipped for now so that we have a passing test suite. We can check for improvements in a follow-up issue.

Thanks!

.github/workflows/test.yml Outdated Show resolved Hide resolved
.github/workflows/test.yml Show resolved Hide resolved
@salilmishra23 salilmishra23 force-pushed the github-actions branch 2 times, most recently from 4629c17 to 996ac23 Compare September 14, 2022 00:50
@salilmishra23
Copy link
Contributor Author

Thanks for the suggestion. I have added more python version for ubuntu-latest, let me know if you want the tests to run on all the versions for the all the OSs.

The tests are failing on Python 3.10 due to wptserve v3.0 being pinned on h2 v3.0.1 which is not compatible with Python 3.10. There hasn't been a new release from them(wptserve) since 2020. [1]

[1] https://pypi.org/project/wptserve/

@salilmishra23
Copy link
Contributor Author

Other possible improvements:

  1. You can also add dependabot in the repository settings, and we can remove the .pyup.yaml.
  2. Getting coveralls to work as done by mozregression.
    • Adding Github token to the github action job and coveralls --service=github to the tox.ini - Link
    • Adding .coveralls.yml file as done by them - Link

@whimboo
Copy link
Contributor

whimboo commented Sep 20, 2022

That's great! Thanks a lot for the update!

Thanks for the suggestion. I have added more python version for ubuntu-latest, let me know if you want the tests to run on all the versions for the all the OSs.

It would probably be enough for now to only use a single Python version for MacOS and Windows. Maybe we use the last official release for it?

The tests are failing on Python 3.10 due to wptserve v3.0 being pinned on h2 v3.0.1 which is not compatible with Python 3.10. There hasn't been a new release from them(wptserve) since 2020. [1]

Would you mind filing an issue for that at https://github.com/web-platform-tests/wpt/issues and referencing here and the yml file? We might have to ignore Python 3.10 for now until it got fixed.

Other possible improvements:
1. You can also add dependabot in the repository settings, and we can remove the .pyup.yaml.

Lets do that. But as I can read in the GitHub documentation it would require a dependabot.yml to be added.

2. Getting `coveralls` to work as done by `mozregression`.     
   * Adding Github token to the github action job and `coveralls --service=github` to the `tox.ini` - [Link](https://github.com/mozilla/mozregression/blob/4f002a750e457fac26fbc0c24adbc85189288acd/.github/workflows/build.yml#L65&L94)
   * Adding `.coveralls.yml` file as done by them - [Link](https://github.com/mozilla/mozregression/blob/4f002a750e457fac26fbc0c24adbc85189288acd/.coveralls.yml)

Please add whatever you can to the appropriate files. Once the PR is merged I can update the repository settings.

@salilmishra23
Copy link
Contributor Author

Hey @whimboo, the PR is ready for review.

Copy link
Contributor

@whimboo whimboo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. Just two remaining questions. Once these are solved I would land the code and setup the repository to handle the dependabot and coveralls services.

Please also update the README.md file and exchange the pyup badge with dependabot as well.

requirements/tests.txt Outdated Show resolved Hide resolved
tests/remote/test_firefox.py Outdated Show resolved Hide resolved
@salilmishra23
Copy link
Contributor Author

Dependabot doesn't seem to have a badge yet dependabot/dependabot-core#1912, so I will remove the pyup badge.

Copy link
Contributor

@whimboo whimboo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the last update! There is just one small thing and then we seem to be ready to get this PR merged.

tests/remote/test_firefox.py Outdated Show resolved Hide resolved
@salilmishra23
Copy link
Contributor Author

Created the issue #625, once the PR is merged, I will update the description with the skip reference.

@whimboo
Copy link
Contributor

whimboo commented Sep 27, 2022

Thanks for filing the issue. I've updated the skip reason with a link to the newly filed issue. If tests are all passing I'm going ahead and merge this PR.

Copy link
Contributor

@whimboo whimboo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are all passing. As such lets get this landed and if something is still missing we can follow-up on. Again, thanks a lot for working on this PR and bringing the project back to life!

@whimboo whimboo merged commit 104a536 into mozilla:master Sep 27, 2022
@salilmishra23 salilmishra23 deleted the github-actions branch September 27, 2022 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Travis CI free usage ends Dec 3; mozilla repos should switch to other CI platforms
2 participants