Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ON-WEEK][POC] Playwright #190803

Merged
merged 79 commits into from
Sep 6, 2024
Merged
Show file tree
Hide file tree
Changes from 53 commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
d3f7ac5
first iteration
MadameSheema Aug 19, 2024
c7a3d6a
extracts loging
MadameSheema Aug 20, 2024
77b67fa
adds page objects
MadameSheema Aug 20, 2024
9224f2b
clean-up
MadameSheema Aug 20, 2024
d07e114
preparing for serverless
MadameSheema Aug 20, 2024
95389df
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Aug 20, 2024
31d5086
Merge branch 'main' into on-week/playwright
MadameSheema Aug 21, 2024
96cfb8c
adds support for serverless
MadameSheema Aug 21, 2024
5d42b62
adjustments
MadameSheema Aug 21, 2024
3c54a46
fixes
MadameSheema Aug 21, 2024
7c21558
fixes check issue
MadameSheema Aug 21, 2024
5787149
adds script to execute serverless in headless mode
MadameSheema Aug 21, 2024
913cfcf
buildkite execution
MadameSheema Aug 21, 2024
13a6d73
adding dependencies
MadameSheema Aug 21, 2024
e566904
fixes naming
MadameSheema Aug 21, 2024
ca4759a
fixes
MadameSheema Aug 21, 2024
7595282
adjustments
MadameSheema Aug 21, 2024
3510cd3
trying to fix buildkite
MadameSheema Aug 22, 2024
edeca9d
adding more tests
MadameSheema Aug 22, 2024
8065dfa
Merge branch 'main' into on-week/playwright
MadameSheema Aug 22, 2024
50aa417
Merge branch 'main' into on-week/playwright
MadameSheema Aug 23, 2024
bffa81c
refactor
MadameSheema Aug 23, 2024
da0b2ea
experiment
MadameSheema Aug 23, 2024
f06fd77
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Aug 23, 2024
9bde417
uses apikey for API calls when the server is serverless
MadameSheema Aug 23, 2024
3d93299
merge main
MadameSheema Aug 23, 2024
648ba44
ess fixes
MadameSheema Aug 23, 2024
3706b37
improvements
MadameSheema Aug 23, 2024
3cd3ae2
package changes
MadameSheema Aug 23, 2024
e1e2b3f
adjustments
MadameSheema Aug 23, 2024
21817bb
ftr config
MadameSheema Aug 23, 2024
957270b
preparing to be able to execute tests on the ide
MadameSheema Aug 24, 2024
38fe12c
Merge branch 'main' into on-week/playwright
MadameSheema Aug 24, 2024
52ef2b1
adjustments
MadameSheema Aug 25, 2024
1368c3b
quick check fix
MadameSheema Aug 25, 2024
ca011de
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Aug 25, 2024
30c38c7
Merge branch 'main' into on-week/playwright
MadameSheema Aug 25, 2024
c72f324
completing script
MadameSheema Aug 25, 2024
bdb983d
tsconfig modif
MadameSheema Aug 25, 2024
95579cb
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Aug 25, 2024
6316f24
tsconfig
MadameSheema Aug 25, 2024
cdb8450
fixes type check buildkite issues
MadameSheema Aug 25, 2024
66458a0
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Aug 25, 2024
7b1f9bf
undo changes on common api
MadameSheema Aug 25, 2024
616d1fd
fixes buildkite errors
MadameSheema Aug 25, 2024
6bbb612
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Aug 25, 2024
22f5f27
adds soft_fail to the execution
MadameSheema Aug 28, 2024
31c9f18
updates codeowners
MadameSheema Aug 28, 2024
7d8c87a
changes on the gitignore
MadameSheema Aug 28, 2024
4d046a9
Merge branch 'main' into on-week/playwright
MadameSheema Aug 28, 2024
1012289
Merge branch 'main' into on-week/playwright
MadameSheema Aug 28, 2024
d69d4d2
Merge branch 'main' into on-week/playwright
MadameSheema Aug 28, 2024
dc1fd35
Merge branch 'main' into on-week/playwright
MadameSheema Aug 28, 2024
18adcda
system browsers
MadameSheema Aug 29, 2024
a09a61b
Merge branch 'main' into on-week/playwright
MadameSheema Aug 30, 2024
aa81ff5
adjustments
MadameSheema Aug 30, 2024
199c1b5
removes soft fail
MadameSheema Aug 30, 2024
7c5e063
copy-paste mistake fix
MadameSheema Aug 30, 2024
315a2e3
refactor
MadameSheema Aug 30, 2024
5bf48dc
New line to fix the exit code issue
dkirchan Aug 30, 2024
a9893a5
Fixed exit code to be reflecting actual errors
dkirchan Aug 30, 2024
f126e55
Merge branch 'main' into on-week/playwright
MadameSheema Sep 3, 2024
5163fdc
fixes on buildkite script
MadameSheema Sep 3, 2024
d679772
fixes on playwright script
MadameSheema Sep 3, 2024
acb090e
merge main
MadameSheema Sep 3, 2024
484b4e0
[CI] Auto-commit changed files from 'node scripts/yarn_deduplicate'
kibanamachine Sep 3, 2024
beb50f1
adding new dependencies to renovate
MadameSheema Sep 3, 2024
9cf36d2
fix
MadameSheema Sep 3, 2024
ee4c3b2
moves the group
MadameSheema Sep 3, 2024
a4065b0
Update renovate.json
MadameSheema Sep 3, 2024
eb359e5
adjusting scripts
MadameSheema Sep 4, 2024
d3d232e
Update .buildkite/scripts/steps/functional/security_serverless_playwr…
MadameSheema Sep 4, 2024
3064584
Update .buildkite/scripts/steps/functional/security_solution_playwrig…
MadameSheema Sep 4, 2024
dcf5894
removes log
MadameSheema Sep 5, 2024
28dbdab
modifies script
MadameSheema Sep 5, 2024
f3f00db
refactor on gitignore
MadameSheema Sep 5, 2024
0dd92ff
removes buildkite execution
MadameSheema Sep 5, 2024
384b470
Merge branch 'main' into on-week/playwright
MadameSheema Sep 5, 2024
7344b41
Merge branch 'main' into on-week/playwright
MadameSheema Sep 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .buildkite/ftr_security_serverless_configs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ disabled:
- x-pack/test_serverless/functional/test_suites/security/cypress/security_config.ts
- x-pack/test/security_solution_cypress/serverless_config.ts

# Playwright
- x-pack/test/security_solution_cypress/serverless_config.ts
MadameSheema marked this conversation as resolved.
Show resolved Hide resolved

# Serverless base config files
- x-pack/test_serverless/api_integration/config.base.ts
- x-pack/test_serverless/functional/config.base.ts
Expand Down
3 changes: 3 additions & 0 deletions .buildkite/ftr_security_stateful_configs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ disabled:
- x-pack/test/threat_intelligence_cypress/cli_config_parallel.ts
- x-pack/test/threat_intelligence_cypress/config.ts

# Playwright
- x-pack/test/security_solution_playwright/playwright.config.ts
jbudz marked this conversation as resolved.
Show resolved Hide resolved

defaultQueue: 'n2-4-spot'
enabled:
- x-pack/test/security_solution_api_integration/test_suites/detections_response/detection_engine/actions/trial_license_complete_tier/configs/ess.config.ts
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
steps:
- command: .buildkite/scripts/steps/functional/security_serverless_playwright.sh
label: 'Serverless Playwright - Security Solution Tests'
agents:
machineType: n2-standard-4
preemptible: true
depends_on:
- build
- quick_checks
timeout_in_minutes: 60
parallelism: 1
retry:
automatic:
- exit_status: '-1'
limit: 1
soft_fail: true
MadameSheema marked this conversation as resolved.
Show resolved Hide resolved

- command: .buildkite/scripts/steps/functional/security_solution_playwright.sh
label: 'Playwright - Security Solution Tests'
agents:
machineType: n2-standard-4
preemptible: true
depends_on:
- build
- quick_checks
timeout_in_minutes: 60
parallelism: 1
retry:
automatic:
- exit_status: '-1'
limit: 1
soft_fail: true
6 changes: 6 additions & 0 deletions .buildkite/scripts/pipelines/pull_request/pipeline.ts
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,9 @@ const getPipeline = (filename: string, removeSteps = true) => {
pipeline.push(
getPipeline('.buildkite/pipelines/pull_request/security_solution/rule_management.yml')
);
pipeline.push(
getPipeline('.buildkite/pipelines/pull_request/security_solution/playwright.yml')
);
}

if (
Expand Down Expand Up @@ -268,6 +271,9 @@ const getPipeline = (filename: string, removeSteps = true) => {
pipeline.push(
getPipeline('.buildkite/pipelines/pull_request/security_solution/investigations.yml')
);
pipeline.push(
getPipeline('.buildkite/pipelines/pull_request/security_solution/playwright.yml')
);
}

if (
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/usr/bin/env bash

set -euo pipefail

source .buildkite/scripts/steps/functional/common.sh

export JOB=kibana-security-solution-playwright
export KIBANA_INSTALL_DIR=${KIBANA_BUILD_LOCATION}

echo "---SERVERLESS - Security Solution Playwright Tests"
MadameSheema marked this conversation as resolved.
Show resolved Hide resolved

yarn playwright install
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add this to yarn kbn bootstrap so it can be cached on CI, or use the already installed system browsers.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbudz can you please point me out where/how to do so? I tried to do it myself in this commit but the test is failing, so I'm assuming I did it in the wrong way.

Copy link
Member Author

@MadameSheema MadameSheema Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UPDATE: I found the issue that was making the test fail, was not related with those changes.

@jbudz would you mind giving another look at the PR? If I'm not mistaken I already addressed all your comments. Thanks! :)

MadameSheema marked this conversation as resolved.
Show resolved Hide resolved

cd x-pack/test/security_solution_playwright

set +e
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of set +e here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbudz I was replicating what we have in all our scripts.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The set +e allows the script to continue to the end of the run if errors are encountered. Normally a bash script will terminate immediately on the first error encountered.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it needed here? The only command run is yarn run:serverless

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing it now.


yarn run:serverless;

exit_code=$?
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/usr/bin/env bash

set -euo pipefail

source .buildkite/scripts/steps/functional/common.sh

export JOB=kibana-security-solution-playwright
export KIBANA_INSTALL_DIR=${KIBANA_BUILD_LOCATION}

echo "---ESS - Security Solution Playwright Tests"
MadameSheema marked this conversation as resolved.
Show resolved Hide resolved

yarn playwright install

cd x-pack/test/security_solution_playwright

set +e

yarn run:ess;

exit_code=$?
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -1480,6 +1480,7 @@ x-pack/test/security_solution_api_integration/test_suites/sources @elastic/secur
/x-pack/test/security_solution_cypress/cypress/* @elastic/security-engineering-productivity
/x-pack/test/security_solution_cypress/cypress/tasks/login.ts @elastic/security-engineering-productivity
/x-pack/test/security_solution_cypress/es_archives @elastic/security-engineering-productivity
/x-pack/test/security_solution_playwright @elastic/security-engineering-productivity
/x-pack/plugins/security_solution/scripts/run_cypress @MadameSheema @patrykkopycinski @maximpn @banderror

## Security Solution sub teams - Threat Hunting Investigations
Expand Down
5 changes: 4 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -1071,6 +1071,7 @@
"deepmerge": "^4.2.2",
"del": "^6.1.0",
"diff": "^5.1.0",
"dotenv": "^16.4.5",
MadameSheema marked this conversation as resolved.
Show resolved Hide resolved
"elastic-apm-node": "^4.7.3",
"email-addresses": "^5.0.0",
"eventsource-parser": "^1.1.1",
Expand Down Expand Up @@ -1459,6 +1460,7 @@
"@mapbox/vector-tile": "1.3.1",
"@octokit/rest": "^17.11.2",
"@parcel/watcher": "^2.1.0",
"@playwright/test": "=1.46.0",
"@redocly/cli": "^1.19.0",
"@statoscope/webpack-plugin": "^5.28.2",
"@storybook/addon-a11y": "^6.5.16",
Expand Down Expand Up @@ -1758,7 +1760,8 @@
"pirates": "^4.0.1",
"piscina": "^3.2.0",
"pixelmatch": "^5.3.0",
"playwright": "=1.38.0",
"playwright": "=1.46.0",
"playwright-chromium": "=1.46.0",
"pngjs": "^3.4.0",
"postcss": "^8.4.31",
"postcss-loader": "^4.2.0",
Expand Down
Loading