feat: add playwright CI testing

[weboko]

Problem

js-waku is a library dedicated for browser environment and sometimes it happens that something breaks at the point of integrations of js-waku and some web application (consumer).

Solution

To solve this problems I considered different options:

• basic karma testing;
  discarded due to limitations of what can be tested (most common use is unit tests execution in browser context), poor integration with web frameworks;

• jest with add-ons;
  the same goes here since, would require karma anyway and it is still not the answer to our problem. One thing that is good is to test React components - but we are not interested in it;

• puppeteer;
  poor support of anything but Chrome;

• playwright;
  this is what I choose to move forward. This tool allows easily run tests in different browsers and allows to treat web app under testing as a web site, which is the end goal of our testing - to make sure latest version builds and works in browser environment

How playwright will work:

• right now it fetches web-chat example by using @waku/create-app and it injects links to repository's version of packages;
• it runs the example as a production build web application on localhost:8080;
• it runs checks over it in different browsers and gives report;

Current design allows us to do following (will document into Epic):

• add some more tests and cover more functionality of js-waku;
• add support for other examples, maybe apps crated by the community (bounties);
• add more complex testing - spawning nwaku nodes, connecting from some example, doing send - receive test;
• add tests for examples like logs testing to check if no errors are there - stability of examples;

Notes

• Resolves Do a bundler run as part of the CI #1418
• Related to Cover Several Environments As Part of Testing #52

[github-actions]

size-limit report 📦

Path	Size	Loading time (3g)	Running time (snapdragon)	Total time
Waku core	27.69 KB (0%)	554 ms (0%)	2.1 s (+49.36% 🔺)	2.6 s
Waku Simple Light Node	262.6 KB (+5.6% 🔺)	5.3 s (+5.6% 🔺)	6 s (+24.08% 🔺)	11.2 s
ECIES encryption	28.7 KB (0%)	574 ms (0%)	1.7 s (+68.79% 🔺)	2.3 s
Symmetric encryption	28.7 KB (0%)	575 ms (0%)	1.7 s (-5% 🔽)	2.3 s
DNS discovery	110.62 KB (0%)	2.3 s (0%)	2.6 s (-2.05% 🔽)	4.8 s
Privacy preserving protocols	117.38 KB (+0.32% 🔺)	2.4 s (+0.32% 🔺)	2.7 s (-23.12% 🔽)	5.1 s
Light protocols	25.83 KB (0%)	517 ms (0%)	1.5 s (-16.36% 🔽)	2 s
History retrieval protocols	24.9 KB (0%)	499 ms (0%)	1.5 s (-13.82% 🔽)	2 s
Deterministic Message Hashing	5.65 KB (0%)	113 ms (0%)	594 ms (+10.87% 🔺)	707 ms

[fryorcraken]

I wonder what is the best practice here as I assume this will fail if we do any breaking change in the API.

[weboko]

Once breaking change is introduced to an example under testing it won't affect CI right away since examples are shipped with a particular version of @waku/create-app.

Or what API do you mean?

[fryorcraken]

Once breaking change is introduced to an example under testing it won't affect CI right away since examples are shipped with a particular version of @waku/create-app.

From the PR's description:

it injects links to repository's version of packages;

what repository are we referring to? My understanding is:

1. Pull latest web-chat code from js-waku-examples repo
2. Does some symlink to replace web-chat's dependency with code from the current js-waku git branch
3. build web-chat, runs some test

if a breaking change on any of the js-waku package happens, then the web-chat code pull cannot use the new changed js-waku API, and hence the CI would fail.

What part am I not understanding?

[weboko]

you are right, to solve this one should update an example to correspond to the breaking changes in js-waku
To do this there are two ways:

• locally bind to the latest version of js-waku and push changes to an example - dirty and doesn't really cover everything;
• some intermediate publications of js-waku so this issue should solve it - update release process to make releases for commits  #1543

[weboko]

that means the flow for release will be something like:

• check if test against example fails;
• if yes - go and upgrade example to next package version;
• check again - if ok - release;

[danisharora099]

any specific reasons we're doing .js files and not .ts?

[danisharora099]

should .env be pushed to GH?

[weboko]

this is ok to push .env file, to make it explicit - renamed it to .env.local to imply it is defaults for local running

[weboko]

any specific reasons we're doing .js files and not .ts?

@danisharora099 yes, because they are simple scripts that are run over node.js as it is and there is no reason to complicate it