ci(win): setup GH actions for windows CI #4402

outsideris · 2020-08-09T12:29:36Z

Description of the Change

Replace AppVeyor with GitHub Actions for testing on Windows.
Because windows builds were failed on AppVeyor, I believe it is the platform issue, not ours, and AppVeyor introduced new Node versions sometimes, I set GitHub Actions for Windows CI.

You can see build results on my repository.

I tested actions/cache, but it spent more time than without cache since downloading cache files.

Benefits

We can have robust windows build results than before.

Possible Drawbacks

I set it up on Windows Server 2019. Do we need to set it up on Windows Server 2016 as well?
I didn't install Growl for Windows because all test already passed.
I added smoke tests and linting on Windows. I thought we need to do it even we run it on Travis. If you think it doesn't need on Windows, let me know.
GitHub provides 2,000 build minutes. Our Travis used 2,200 build mins last month and Travis build time is about 2 times than GitHub Actions Windows build. So, I think we can use GitHub Actions freely, but we will pay for Github Actions in the future.

Applicable issues

#4117
Close #4390

I want to hear feedback from @mochajs/core .

Signed-off-by: Outsider <[email protected]>

coveralls · 2020-08-09T12:42:29Z

Coverage increased (+0.06%) to 93.885% when pulling 3d9b919 on outsideris:windows-actions into c1db2a6 on mochajs:master.

Signed-off-by: Outsider <[email protected]>

craigtaub

I added smoke tests and linting on Windows

Whats the value in adding these on Windows? Just thinking that it adds time and expense. Wondering if we have Windows users who get passing CI builds only to fail on release.

Any idea if there is a performance difference against AppVeyor?

That's great that we should get it free for now, what does AppVeyor cost currently (as that's a saving)?

craigtaub · 2020-08-10T20:50:14Z

appveyor.yml

-
-## Build configuration
-platform:
-  - x64


64-bit, so I'd assume windows server 2019 is enough by itself, but not sure.

craigtaub · 2020-08-10T20:56:35Z

appveyor.yml

-## General configuration
-version: '{build}'
-skip_commits:
-  message: /\[ci\s+skip\]/


Can the action skip too?

I will check it.

outsideris · 2020-08-11T16:50:20Z

Whats the value in adding these on Windows? Just thinking that it adds time and expense. Wondering if we have Windows users who get passing CI builds only to fail on release.

The purpose of smoke tests is to confirm that mocha will work when we release it. So, I thought it is better if we run smoke tests on Windows as well.

Linting is less useful than smoke tests. I'm not sure that failure to lint on windows while lint passed on Linux is possible. Just I want to check it for our windows contributors.

I can remove them from it if we agree it is useless.

Any idea if there is a performance difference against AppVeyor?

AppVeyor took about 22 mins and GitHub Actions took about 12 mins even though GitHub Actions have more tasks.
IMO, AppVeyor is free for us.

craigtaub · 2020-08-11T17:08:12Z

AppVeyor took about 22 mins and GitHub Actions took about 12 mins even though GitHub Actions have more tasks.

🤩 🚀

So, I thought it is better if we run smoke tests on Windows as well.

I suppose with perf high and cost low its worth it. Linting I agree probably not necessary.

boneskull · 2020-08-12T04:26:08Z

My only requirement is that it runs on Windows 10 and PowerShell. It should not be WSL or "Git Bash" or whatever.

If that's the case then I am 100000000% interested in merging this. 🥳

outsideris · 2020-08-16T09:19:11Z

I'm trying to add a feature that skip CI with [ci skip]. Skipping CI based on branches is easy, but based on PRs is complex because GitHub doesn't provide the latest commit message on their context when triggered by PRs.

I will back after some test for that.

outsideris · 2020-08-16T09:23:27Z

@boneskull I'm not familiar with Windows as development environment.
GitHub Actions only supports Windows Server 2016 and Windows Server 2019 and Windos Server 2019 looks similar with Windows 10.
And Powershell is the default shell for Windows in GitHub Actions.

powershell: This is the default shell used on Windows. The Desktop PowerShell.

Signed-off-by: Outsider <[email protected]>

outsideris · 2020-08-17T09:35:44Z

Done.

Now, it will be skipped with [ci skip] commit message. You can check it in my repository for push and pull_request.

Retrive head commit message job always runs to get a head commit message when action triggered by pull request.

And I removed lint job from Windows CI.

Munter · 2020-08-17T11:20:08Z

I really like this addition. I just double checked out settings. Looks like we have 3000 minutes per month, apparently from a free open source team plan which must have been set up at some point.

We can also limit spending to $0 if we are afraid of running up a bill at some point. That would require someone to put down a credit card though, and I assume we don't have a cards associated with our opencollective profile, so it's probably an individual that needs to do this

boneskull · 2020-08-17T23:06:35Z

I can put my feelers out and see if we can get more minutes if we start running into the limit.

boneskull · 2020-08-17T23:08:12Z

sounds good @outsideris. travis has windows builds but they aren't powershell envs afaik. would be interested in moving everything to GHA eventually

Signed-off-by: Outsider <[email protected]>

outsideris · 2020-09-05T09:52:47Z

.github/workflows/windows-ci.yml

+          restore-keys: |
+            ${{ runner.os }}-growl-installer
+      - name: Download Growl Installer
+        if: steps.cache-growl.outputs.cache-hit != 'true'


dowlnload Growl Installer only when the cache is not hit,

outsideris · 2020-09-05T09:54:10Z

.github/workflows/windows-ci.yml

+          echo "Verifying Growl responding"
+          Start-Sleep -s 5
+          growlnotify test
+          echo "Staring test"


Start and verify growl before running the test because GH actions don't keep growl process previous steps.

Sometimes, there are DllNotFoundException, not always. But we can verify Growl responding regardless the exception.

outsideris · 2020-09-05T10:06:45Z

I added Growl installer and cache it which mentioned #4391

outsideris · 2020-09-05T10:52:33Z

It is ready to merge.

outsideris · 2020-09-12T11:10:50Z

I will merge this in a few days.

Munter · 2020-09-20T08:07:56Z

Great work on this! Do good to have both speed increases and better access for everyone to see the results

outsideris added 2 commits August 9, 2020 20:42

ci(win): setup GH actions for windows CI

9c72663

Signed-off-by: Outsider <[email protected]>

ci(win): retire AppVeyor

c491e3c

Signed-off-by: Outsider <[email protected]>

outsideris added the qa label Aug 9, 2020

outsideris requested review from boneskull, craigtaub, juergba, Munter and nicojs August 9, 2020 12:30

ci(win): install production deps for smoke test

c7a7d5f

Signed-off-by: Outsider <[email protected]>

craigtaub reviewed Aug 10, 2020

View reviewed changes

outsideris force-pushed the windows-actions branch 2 times, most recently from 2bdfe59 to c7a7d5f Compare August 11, 2020 19:21

outsideris force-pushed the windows-actions branch from 2faa859 to c7a7d5f Compare August 16, 2020 09:05

boneskull mentioned this pull request Aug 16, 2020

Update dependencies & dev dependencies #4396

Merged

outsideris added 2 commits August 17, 2020 18:17

ci(win): remove lint from Windows CI

89b2e47

Signed-off-by: Outsider <[email protected]>

ci(win): skip ci base on commit msg

fcd3272

Signed-off-by: Outsider <[email protected]>

boneskull mentioned this pull request Aug 18, 2020

cache Growl installer on AppVeyor #4391

Closed

install growl on Windows CI

428f577

Signed-off-by: Outsider <[email protected]>

outsideris force-pushed the windows-actions branch from 396216d to 074e849 Compare September 5, 2020 09:38

cache growl on Windows CI

3d9b919

Signed-off-by: Outsider <[email protected]>

outsideris force-pushed the windows-actions branch from 074e849 to 3d9b919 Compare September 5, 2020 09:51

outsideris commented Sep 5, 2020

View reviewed changes

outsideris merged commit 28f970e into mochajs:master Sep 19, 2020

outsideris mentioned this pull request Sep 19, 2020

Proposal for using Azure DevOps CI for Windows test #4013

Closed

boneskull mentioned this pull request Oct 2, 2020

run all node.js tests on GHA #4459

Merged

outsideris mentioned this pull request Jan 10, 2021

Remove caching growl step #4546

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci(win): setup GH actions for windows CI #4402

ci(win): setup GH actions for windows CI #4402

outsideris commented Aug 9, 2020 •

edited

Loading

coveralls commented Aug 9, 2020 •

edited

Loading

craigtaub left a comment •

edited

Loading

craigtaub Aug 10, 2020

craigtaub Aug 10, 2020

outsideris Aug 11, 2020

outsideris commented Aug 11, 2020

craigtaub commented Aug 11, 2020

boneskull commented Aug 12, 2020

outsideris commented Aug 16, 2020

outsideris commented Aug 16, 2020

outsideris commented Aug 17, 2020

Munter commented Aug 17, 2020

boneskull commented Aug 17, 2020

boneskull commented Aug 17, 2020

outsideris Sep 5, 2020 •

edited

Loading

outsideris Sep 5, 2020

outsideris Sep 5, 2020

outsideris commented Sep 5, 2020

outsideris commented Sep 5, 2020

outsideris commented Sep 12, 2020

Munter commented Sep 20, 2020

ci(win): setup GH actions for windows CI #4402

ci(win): setup GH actions for windows CI #4402

Conversation

outsideris commented Aug 9, 2020 • edited Loading

Description of the Change

Benefits

Possible Drawbacks

Applicable issues

coveralls commented Aug 9, 2020 • edited Loading

craigtaub left a comment • edited Loading

Choose a reason for hiding this comment

craigtaub Aug 10, 2020

Choose a reason for hiding this comment

craigtaub Aug 10, 2020

Choose a reason for hiding this comment

outsideris Aug 11, 2020

Choose a reason for hiding this comment

outsideris commented Aug 11, 2020

craigtaub commented Aug 11, 2020

boneskull commented Aug 12, 2020

outsideris commented Aug 16, 2020

outsideris commented Aug 16, 2020

outsideris commented Aug 17, 2020

Munter commented Aug 17, 2020

boneskull commented Aug 17, 2020

boneskull commented Aug 17, 2020

outsideris Sep 5, 2020 • edited Loading

Choose a reason for hiding this comment

outsideris Sep 5, 2020

Choose a reason for hiding this comment

outsideris Sep 5, 2020

Choose a reason for hiding this comment

outsideris commented Sep 5, 2020

outsideris commented Sep 5, 2020

outsideris commented Sep 12, 2020

Munter commented Sep 20, 2020

outsideris commented Aug 9, 2020 •

edited

Loading

coveralls commented Aug 9, 2020 •

edited

Loading

craigtaub left a comment •

edited

Loading

outsideris Sep 5, 2020 •

edited

Loading