Speed up e2e execution in CI

[bafonins]

Summary

Currently, e2e tests already take ~16minutes to execute (~21mins with several end device e2e tests, see #3065). Taking into account that most of the console/oauth app is still lacking any tests. Later with more e2e tests CI pipeline will take even more time.

Why do we need this?

Make sure we do not end up with long CI execution because of e2e tests.

What is already there? What do you see now?

A bunch of e2e tests and e2e workflow that executes all tests on almost any PR.

What is missing? What do you want to see?

Reduced CI time for e2e tests execution.

How do you propose to implement this?

• execute e2e tests in parallel
  • consider Cypress Dashboard
• split e2e tests into groups and run them in separate jobs (e.g. run console, oauth and smoke tests separately)
• Only run smoke tests on pull requests and execute all e2e tests on releases

2. ✅ Cache or/and speed up js:build target Speed up frontend builds #2661
3. ✅ Optimise cy.loginConsole to avoid going through the whole oauth flow to get the auth token. Fix flaky e2e tests #3978

How do you propose to test this?

CI

Can you do this yourself and submit a Pull Request?

yes

[bafonins]

Consider using an admin API key to perform api calls. See #3746

[kschiffer]

I was able to bring the end-to-end tests execution time down to between <5mins (best case) and <10mins (worst case) by activating parallelization (to 4 machines) and by adding more caching to the workflow runs. An example run can be seen in my fork here:
https://github.com/kschiffer/lorawan-stack/actions/runs/906365351

In order to use parallelization in cypress, we need a paid account. However, it seems like there are cheaper plans for open source projects.

@johanstokking @htdvisser please share your thought about whether you think this is worth pursuing.

[johanstokking]

The paid account has now been setup.