[ON-WEEK][POC] Playwright (elastic#190803)

## UPDATE
It has been removed the execution of the playwright tests on buildkite,
the execution will be re-enabled as soon as we are ready and as
described below in the PR, there are still steps pending to be done.

## Motivation

**Cypress is not performing well lately.**
* We have been facing significant performance issues with Cypress. For
instance, it takes a long time to open the visual interface and start
executing tests.

**Teams are finding it increasingly challenging to write new tests and
debug existing ones.**
* The time and effort required to create new tests or troubleshoot
existing ones have become burdensome.

**Concern about the impact this could have on our testing practices.**
* Lose motivation to write tests or, worse, skip writing crucial tests.

## Why Playwright?

* Compared to Cypress, Playwright seems to be known for its faster
execution times and lower resource consumption. What could have a
positive impact by having faster feedback during development and
execution of new tests as well as more efficient use of CI resources.

* Provides powerful debugging tools which can make easier to write,
debug and execute tests.

* Seems to provide the same capabilities we currently use in our Cypress
tests.

* Given Playwright's active development and backing by Microsoft, it is
likely to continue evolving rapidly, making it a safe long-term choice.

Considering all the above, Playwright seems to be a strong candidate to
replace Cypress and address all the issues we are facing lately
regarding UI test automation.

## Objective of this POC

To write in Playwright a couple of tests we currently have on Cypress to
check the performance of the tool as well as the development experience.

The tests selected have been:
-
[enable_risk_score_redirect.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/entity_analytics/dashboards/enable_risk_score_redirect.cy.ts)
- Owned by Entity Analytics team and selected by its simplicity since it
does not need any special setup to be executed and is short.

-
[manual_rule_run.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine/rule_gaps/manual_rule_run.cy.ts)
- Owned by Detection Engine team and selected because is short and adds
a bit more of complexity due to it needs of clean-up and setting up
initial data through the API.
  
## How to execute the tests

### Visual mode
- Navigate to: `x-pack/test/security_solution_playwright`
- Execute: `yarn open:ess` for ESS environment or `yarn open:serverless`
for serverless environment.

### Headless mode
- Navigate to: `x-pack/test/security_solution_playwright`
- Execute: `yarn run:ess` for ESS environment or `yarn run:serverless`
for serverless environment.

### From VScode
- Install `Playwright Test for VScode` extension by Microsoft
- Navigate to: `x-pack/test/security_solution_playwright`
- Execute: `yarn open:ess` for ESS environment or `yarn open:serverless`
for serverless environment.
- Open your IDE
- Click on the `Testing` icon
- On the `Test Explorer` click on the three dots to select the profile
you are going to execute `ess` or `serverless`
- Click on the test you want to execute or navigate to the spec file of
the test and execute it from the same spec.

## My experience
- Tests are way easier to implement than with Cypress.
- Playwright does not rely on chainable commands. Chainable commands on
Cypress can lead to confusing code.
- Without chainable commands, the flow of the tests is more explicit and
easier to understand.
- You can notice that the tool has been designed with Typescript in
mind.
- Is super easy to implement the Page Object Model pattern (POM).
- With POM the test code is clean and focused on "what" rather than
"how".
- Love the fact that you can execute the tests from the same IDE without
having to switch windows during test development.
- The visual mode execution gives you lots of information out of the
box.

## The scope of this PR
- Sets the initial infrastructure to write and execute tests with
Playwright.
- Has examples and set a basis about how to write tests using the POM.
- Allows the execution of the tests in ESS and serverless (just
stateless environment).
- Integrates the execution of the tests with buildkite.

## Pending to be done/investigate
- Proper readme
- How to split tests and PO between the different teams
- Good reports on CI
- Upload screenshots on CI
- Flaky test suite runner 
- Complete the labeling
- Execution of the tests on MKI environments

## FAQ
**Can I start adding tests to playwright?**
Currently, you can explore and experiment with Playwright, but there is
still work pending to be done to make the tool officially usable.

**Why security engineering productivity is the owner of the playwright
folder?**
This is something temporary to make sure that good practices are
followed.

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: dkirchan <[email protected]>
Co-authored-by: Aleh Zasypkin <[email protected]>
Co-authored-by: Jon <[email protected]>

.buildkite/ftr_security_serverless_configs.yml

@@ -12,6 +12,9 @@ disabled:
   - x-pack/test_serverless/functional/test_suites/security/cypress/security_config.ts
   - x-pack/test/security_solution_cypress/serverless_config.ts
 
+  # Playwright
+  - x-pack/test/security_solution_playwright/serverless_config.ts
+
   # Serverless base config files
   - x-pack/test_serverless/api_integration/config.base.ts
   - x-pack/test_serverless/functional/config.base.ts

.buildkite/ftr_security_stateful_configs.yml

@@ -24,6 +24,9 @@ disabled:
   - x-pack/test/threat_intelligence_cypress/cli_config_parallel.ts
   - x-pack/test/threat_intelligence_cypress/config.ts
 
+  # Playwright
+  - x-pack/test/security_solution_playwright/playwright.config.ts
+
 defaultQueue: 'n2-4-spot'
 enabled:
   - x-pack/test/security_solution_api_integration/test_suites/detections_response/detection_engine/actions/trial_license_complete_tier/configs/ess.config.ts

.buildkite/pipelines/pull_request/security_solution/playwright.yml

@@ -0,0 +1,30 @@
+steps:
+  - command: .buildkite/scripts/steps/functional/security_serverless_playwright.sh
+    label: 'Serverless Playwright - Security Solution Tests'
+    agents:
+      machineType: n2-standard-4
+      preemptible: true
+    depends_on:
+      - build
+      - quick_checks
+    timeout_in_minutes: 60
+    parallelism: 1
+    retry:
+      automatic:
+        - exit_status: '-1'
+          limit: 1
+
+  - command: .buildkite/scripts/steps/functional/security_solution_playwright.sh
+    label: 'Playwright - Security Solution Tests'
+    agents:
+      machineType: n2-standard-4
+      preemptible: true
+    depends_on:
+      - build
+      - quick_checks
+    timeout_in_minutes: 60
+    parallelism: 1
+    retry:
+      automatic:
+        - exit_status: '-1'
+          limit: 1

.buildkite/scripts/steps/functional/security_serverless_playwright.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+source .buildkite/scripts/steps/functional/common.sh
+
+export JOB=kibana-security-solution-playwright
+export KIBANA_INSTALL_DIR=${KIBANA_BUILD_LOCATION}
+
+echo "--- SERVERLESS - Security Solution Playwright Tests"
+
+cd x-pack/test/security_solution_playwright
+
+yarn run:serverless; exit_code=$?; exit $exit_code

.buildkite/scripts/steps/functional/security_solution_playwright.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+source .buildkite/scripts/steps/functional/common.sh
+
+export JOB=kibana-security-solution-playwright
+export KIBANA_INSTALL_DIR=${KIBANA_BUILD_LOCATION}
+
+echo "--- ESS - Security Solution Playwright Tests"
+
+cd x-pack/test/security_solution_playwright
+
+yarn run:ess; exit_code=$?; exit $exit_code

.github/CODEOWNERS

@@ -1481,6 +1481,7 @@ x-pack/test/security_solution_api_integration/test_suites/sources @elastic/secur
 /x-pack/test/security_solution_cypress/cypress/* @elastic/security-engineering-productivity
 /x-pack/test/security_solution_cypress/cypress/tasks/login.ts @elastic/security-engineering-productivity
 /x-pack/test/security_solution_cypress/es_archives @elastic/security-engineering-productivity
+/x-pack/test/security_solution_playwright @elastic/security-engineering-productivity
 /x-pack/plugins/security_solution/scripts/run_cypress @MadameSheema @patrykkopycinski @maximpn @banderror
 
 ## Security Solution sub teams - Threat Hunting Investigations

.gitignore

@@ -150,3 +150,11 @@ oas_docs/output/kibana.serverless.tmp*.yaml
 oas_docs/output/kibana.tmp*.yaml
 oas_docs/output/kibana.new.yaml
 oas_docs/output/kibana.serverless.new.yaml
+
+# Security Solution Playwright
+x-pack/test/security_solution_playwright/test-results/
+x-pack/test/security_solution_playwright/playwright-report/
+x-pack/test/security_solution_playwright/blob-report/
+x-pack/test/security_solution_playwright/playwright/.cache/
+x-pack/test/security_solution_playwright/.auth/
+x-pack/test/security_solution_playwright/.env

kbn_pm/src/lib/bazel.mjs

@@ -152,6 +152,10 @@ export async function installYarnDeps(log, opts = undefined) {
   });
 
   log.success('yarn deps installed');
+
+  await run('yarn', ['playwright', 'install']);
+
+  log.success('Playwright browsers installed');
 }
 
 /**

package.json

@@ -1072,6 +1072,7 @@
     "deepmerge": "^4.2.2",
     "del": "^6.1.0",
     "diff": "^5.1.0",
+    "dotenv": "^16.4.5",
     "elastic-apm-node": "^4.7.3",
     "email-addresses": "^5.0.0",
     "eventsource-parser": "^1.1.1",
@@ -1462,6 +1463,7 @@
     "@mapbox/vector-tile": "1.3.1",
     "@octokit/rest": "^17.11.2",
     "@parcel/watcher": "^2.1.0",
+    "@playwright/test": "=1.46.0",
     "@redocly/cli": "^1.21.0",
     "@statoscope/webpack-plugin": "^5.28.2",
     "@storybook/addon-a11y": "^6.5.16",
@@ -1761,7 +1763,8 @@
     "pirates": "^4.0.1",
     "piscina": "^3.2.0",
     "pixelmatch": "^5.3.0",
-    "playwright": "=1.38.0",
+    "playwright": "=1.46.0",
+    "playwright-chromium": "=1.46.0",
     "pngjs": "^3.4.0",
     "postcss": "^8.4.31",
     "postcss-loader": "^4.2.0",

renovate.json

@@ -485,6 +485,15 @@
       "matchBaseBranches": ["main"],
       "labels": ["release_note:skip", "team:obs-entities"],
       "enabled": true
+    },
+    {
+      "groupName": "Security Engineering Productivity",
+      "matchDepNames": ["dotenv", "playwright-chromium", "@playwright/test"],
+      "reviewers": ["team:security-engineering-productivity"],
+      "matchBaseBranches": ["main"],
+      "labels": ["Team: Sec Eng Productivity", "release_note:skip", "backport:all-open"],
+      "minimumReleaseAge": "7 days",
+      "enabled": true
     }
   ],
   "customManagers": [