Bring back tests to CI/CD pipeline

[Splines]

#562 removed the testing from the CI/CD pipeline. We should bring it back in an easy to use fashion. This includes

• Easy to use test integration into VS Code. We have to run the commands in docker, see this commit
• Preconfigured commands in a justfile to run tests in the terminal (if other IDE is used or when no IDE access is available, just the shell).
• Automatic and quick (under 3 minutes for unit tests) test runs on pushes on a PR. Test output should be concise and summarize what went wrong.
• Integration to Codecov? If so: the comments should be smaller and only contain the most relevant information with a link to go to the full detailed report.

[Splines]

With regards to the DatabaseCleaner, the following links might be useful:

• database cleaner blog post

[Splines]

Cypress vs. Capybara

There is a suggestion whether we should switch to capybara for front-end integration tests instead of using cypress. I gathered some (admittedly anectodical) posts, some of which try to provide a fair comparison, others endorsing cypress in a direct comparison. (I haven't been able to find a post that directly compares cypress & capybara and favors capybara over cypress, only the other way around.)

These arguments might be biased towards favoring Cypress.

• First of, there's the Cypress website itself, which of course might be biased. In comparison to capybara, it outlines its GUI-centric approach where one can actually see screen recordings of what is happening in the tests. I think this is a huge benefit as one can easily identify problems in failing tests with this approach.
• Moving from Capybara to Cypress blog post. Also emphasizes the aspect of the screen recording. Talks about frustrations in Capybara setup.
• Stackshare direct comparison. One point worth considering here is the "Synchronisation" part:

Capybara provides implicit waiting, which means it waits for a certain amount of time before performing an action. This can lead to slower test execution and makes it harder to catch asynchronous issues. Cypress, on the other hand, uses automatic waiting and retries, which eliminates the need for explicit waits and makes the tests faster and more reliable by automatically synchronizing with the application.

The asynchronous approach of Capybara is described in its documentation here. However, Cypress apparently also follows this timeout-and-try strategy as seen here. But with Cypress, for DOM events, it will automatically wait:

DOM based commands will automatically retry and wait for their corresponding elements to exist before failing.

as seen here

---

We should also take into account, that we already have quite a few tests written in Cypress for MaMpf, so switching to Capybara would imply the additional effort of having to rewrite those tests in another language. Based on my very limited research on both tools, I prefer Cypress so far, as:

• we already have tests written with Cypress
• Cypress offers screen recordings for tests, such that debugging is easier when a test fails (instead of having to work through a browser stack trace, we get to see what happened on the page in the browser directly)
• Cypress is JS-based and everything in the frontend is done in JS (or Coffeescript), so the syntax is easier to adopt. Having seen only few examples, I prefer the JS-centric way of dealing with the DOM. It just feels more natural as I'm used to deal with the DOM in JS and not in Rails.
• With Cypress, we really get an "end-to-end testing" in the sense that Cypress only sees what a user would see, it does not know anything about Rails in the backend.

We have to also consider that Cypress might be more difficult to setup, especially with regards to our docker setup. But I'm sure, we can manage to get a workable solution with no flaky tests and easy to use setup locally (which might require some initial effort, but that's probably also the case with capybara).

---

After having talked in persona (@fosterfarrell9 and @Splines), we found that staying with Cypress would be the best option for us, especially since we already have test cases written with Cypress. However, we want to focus more on the integration test aspect, i.e. not try to manipulate the API or the backend via app factory methods. Instead, testing with Cypress will allow us to test MaMpf just like a manual tester would, without any knowledge of the backend and without even knowing that it is written in Ruby on Rails.

Therefore we will probably not mock any data and instead preseed/initialize the database with our already-existing mock data before the Cypress test runs. This is contrary to the philosophy of unit tests where every test should only have a very narrow scope and where we mock our data with FactoryBot.

Nevertheless, we still have to find a good solution such that Cypress tests don't affect each other when manipulating the database. This could be achieved by using transactions and automatically rolling them back after each test run; we'll see what a good strategy for that is. I will try to get the setup to work locally and in the CI/CD pipeline again in the near future. Before that, unit test support is brought back via #581.

[Splines]

Getting Cypress up and running

• Rails, Cypress : testing the whole stack is definitely easier -> but this one uses a different gem
• Cypress best practices
• Cypress on Rails Gem we're using
• Introduction to Cypress on Rails
• 7min walkthrough on how to setup

[Splines]

So finally, in #652, we also brought back Cypress. I think that this issue is therefore done.