Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make FE E2E tests more reliable #16759

Merged
merged 10 commits into from
Sep 17, 2022
Merged

make FE E2E tests more reliable #16759

merged 10 commits into from
Sep 17, 2022

Conversation

lmossman
Copy link
Contributor

@lmossman lmossman commented Sep 14, 2022

What

Our frontend E2E connection tests have been failing more often than normal. They were temporarily disabled in #16752 in order to not block other teams.

The goal of this PR is to make the tests more reliable so that they do not fail so often.

How

This PR aims to make them more reliable by making a few changes:

  1. Increase the command timeout to 10s (from the default of 4s). It seemed like some operations were right on the cusp of ~4s, so increasing this gives the tests bit more breathing room.
  2. Suppress uncaught exceptions that occur in the webapp. Without this change, any uncaught exception causes the cypress test to fail, even if the exceptions don't actually result in the webapp performing incorrectly. With this change, these irrelevant errors will be suppressed, and tests will still fail if the webapp actually behaves incorrectly.
  3. Make sure that names of sources and dests are unique between tests runs, so that multiple test runs do not affect each other.
  4. Uncomment the connection tests

@github-actions github-actions bot added area/platform issues related to the platform area/frontend Related to the Airbyte webapp labels Sep 16, 2022
);

return React.createElement(as, {
...remainingProps,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was needed so that the data-id prop passed in here is actually set on the resulting element, so that our cypress tests still work

@lmossman lmossman marked this pull request as ready for review September 16, 2022 01:06
@lmossman lmossman requested a review from a team as a code owner September 16, 2022 01:06
@lmossman lmossman merged commit 0ff7234 into master Sep 17, 2022
@lmossman lmossman deleted the lmossman/fix-fe-e2e branch September 17, 2022 18:21
robbinhan pushed a commit to robbinhan/airbyte that referenced this pull request Sep 29, 2022
* make tests more reliable

* switch back to localhost

* run test 100 times

* pass props through to Text element

* ensure names are unique from one test run to the next

* fix test result check
jhammarstedt pushed a commit to jhammarstedt/airbyte that referenced this pull request Oct 31, 2022
* make tests more reliable

* switch back to localhost

* run test 100 times

* pass props through to Text element

* ensure names are unique from one test run to the next

* fix test result check
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/frontend Related to the Airbyte webapp area/platform issues related to the platform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants