wait-on issues with Node.js 18

[MikeMcC399]

Problem description

With the GitHub migration of runners to use Node.js 18 as default on Feb 18, 2023, some workflows using the built-in wait-on parameter of cypress-io/github-action are now were failing: for example, #760 with Error: connect ECONNREFUSED 127.0.0.1:4200. Edit: no longer failing due to workaround implementation.

This affects development webservers which are not listening on both IPv4 and IPv6. Some, like vite, can be set to start and listen on 0.0.0.0 which solves the issue. Others don't seem to be so flexible.

It also affects the alternate method of using start-server-and-test as an argument to the built-in wait-on parameter. Due to the fact that start-server-and-test depends on the separate npm module wait-on and this external wait-on also trips up on Node.js 18 in certain configurations, it also becomes a problem for start-server-and-test.

Workarounds can be found, such as using the IPv6 loopback address ::1 instead of the more generic localhost hostname. The workarounds differ depending on the dev webserver and the environment it is being run under.

Questions

1. Is it necessary for cypress-io/github-action to adapt to dev webservers which are not listening on both IPv4 and IPv6, or is the onus on the dev webserver providers to make changes and on users to find suitable workarounds?
2. If changes are made to cypress-io/github-action, is the older got v11 still the right choice for src/ping.js?
3. Should cypress-io/github-action migrate to the current got v12 v13 version? This is a breaking change due to the migration from CommonJS to native ESM.
4. Or should another technology be used to check if a webserver is running?

[MikeMcC399]

Workarounds

This is a suggestion for workarounds in users' workflows when using the built-in wait-on parameter of cypress-io/github-action@v5 in Node.js 18 (and later version) environments on GitHub:

1. Remove the wait-on parameter completely as a test and possible permanent solution. Some workflows do not need this step at all.
2. Find out from documentation or community interaction if your development server can be started so that it listens to both IPv4 and IPv6 network traffic. Implement changed start-up if available.
3. Instead of waiting for localhost, wait for the IPv4 loopback address 127.0.0.1 or the IPv6 loopback address ::1. It will be the opposite to the one mentioned in the connect error message.
4. Use an alternate dev webserver which listens on both IPv4 and IPv6.

Below are specific examples:

Angular

Angular 15 built with ng new angular-test (see Angular: Setting up the local environment and workspace).

      - name: Test
        uses: cypress-io/github-action@v5
        with:
          start: npm start
          wait-on: 'http://localhost:4200'

fails on GitHub ubuntu-22.04, windows-2022 and macos-12 with

Error: connect ECONNREFUSED 127.0.0.1:4200

Workaround: use the IPv6 loopback address ::1 as in wait-on: 'http://[::1]:4200' instead of using localhost.

At this time the literal IPv6 address ::1 cannot be used in config: baseUrl=http://[::1]:4200 (see issue cypress-io/cypress#25950), so baseUrl has to stay unchanged using localhost in cypress config.

      - name: Test
        uses: cypress-io/github-action@v5
        with:
          start: npm start
          wait-on: 'http://[::1]:4200'

Vite

Vite v3 & v4 built with npm create vite with options Vanilla and JavaScript (see Vite: Getting Started).

    - name: Test
      uses: cypress-io/github-action@v5
      with:
        wait-on: http://localhost:5173
        start: npm run dev

fails on GitHub ubuntu-22.04, windows-2022 and macos-12 with

Error: connect ECONNREFUSED 127.0.0.1:5173

Workaround: start server with npx vite --host (server then listens on 0.0.0.0):

    - name: Test
      uses: cypress-io/github-action@v5
      with:
        wait-on: http://localhost:5173
        start: npx vite --host

[MikeMcC399]

There are now generic workaround instructions posted to wait-on with Node.js 18+ concerning this issue.

[AtofStryker]

I think considering how limited we use got, I am not sure it is worth the effort to convert to ESM just for the reason of upgrading got. However, there might be other reasons we want to upgrade to ESM.

I am going to bring this up to the team. I think whatever we decide to do, we need to have the same alignment across our tools repos, for example the netlify-plugin-cypress uses a very similar ping structure with got v11

[MikeMcC399]

@AtofStryker

I don't believe that upgrading got from v11 to v12 will solve this issue. I posted a question to sindresorhus/got#2235 to see if I could get some feedback, but there has been no comment so far.

[MikeMcC399]

I also tried to revive an old issue in facebook/create-react-app#11302 as an example server which is not listening on both stacks, but there hasn't been any response there either.

[emilyrohrbough]

@MikeMcC399

Is it necessary for cypress-io/github-action to adapt to dev webservers which are not listening on both IPv4 and IPv6, or is the onus on the dev webserver providers to make changes and on users to find suitable workarounds?

The typical Cypress approach here would be to try & find a solution that would handle both IPv4 and IPv6. This action is meant to be a convenience and provide easy CI setup and that would be inclusive of a local dev-setup.

As for vite/angular workarounds - thank you for posting! Hopefully these user are able to use the CT testing successfully regarldess of wait-on if they are purely testing the components & component composition.

I agree with @AtofStryker - Not worth updating the entire project to be ESM for one dependency with limited use. It be worth considering a different dependency to try & resolve this issue. Another dependency I was recently looking at swapped got for node-fetch. That may be an option.

[MikeMcC399]

@emilyrohrbough

I agree with @AtofStryker - Not worth updating the entire project to be ESM for one dependency with limited use. It be worth considering a different dependency to try & resolve this issue. Another dependency I was recently looking at swapped got for node-fetch. That may be an option.

The discussion about ESM is a distraction, since updating got would not fix the issue as far as I can see. But anyway, to update to got 12.x it would not need the whole project updating, only the utility src/ping.js. Migrating to node-fetch would have a similar impact to updating got. "node-fetch from v3 is an ESM-only module" and the API is different, so, it would also require a re-write of src/ping.js to migrate to node-fetch@latest.

There hasn't been much user-feedback on this issue lately, so perhaps the workarounds I posted are sufficient?

[MikeMcC399]

got releases have now reached 13.0. This remains an ESM-only module, so updating requires migration work from CommonJS to ESM.

[MikeMcC399]

The wait-on issues described here are not due to a bug in cypress-io/github-action. They are due to changes in Node.js and the issues affect servers which do not respond on both IPv4 and IPv6 stacks, so that attempts to correctly connect via hostname localhost fail on Node.js 18.

Workarounds were published to README > wait-on with Node.js 18+ in March 2023.

In the meantime, tests using Node.js 20, which is planned to become the LTS version in October 2023, show that the issue does not occur using the original tests for Angular, Gatsby and Vite.

Closing this issue as no special action is necessary for cypress-io/github-action. The situation should however still be monitored.

[courtneycb]

I'm experiencing this issue using Node 20.x on GitHub ubuntu-22.04.

Error: connect ECONNREFUSED 127.0.0.1:5173

The workaround posted above of starting the server with npx vite --host instead of npm run dev works for me. Using Vite v4.

[MikeMcC399]

Updating from the now deprecated cypress-io/github-action@v5 (or earlier versions) to cypress-io/github-action@v6 resolves the wait-on issue for Angular. Gatsby and Vite.

This seems to be related to the fact that cypress-io/github-action@v6 runs under node20 instead of previous versions which ran under node16.