playwright: Update to latest versions and add new page objects

- Update `playwright` to the latest version
  - Update test configuration to adapt to latest version
  - Remove obsolete multiple config files and scripts
- Add page objects for OutputView, OutputChannel, Toolbar, MonacoEditor
- Extend tests by testing the new page objects
- Extend ExplorerView tests with compact folders
- Extend QuickCommand tests
- Update GETTING_STARTED.md and DEVELOPING.md
- Update build script to use ensure clean and lint are called
  - Ensure all necessary playwright dependencies are installed on initial build

Signed-off-by: Nina Doschek <[email protected]>

.vscode/launch.json

@@ -176,21 +176,6 @@
       "outFiles": [
         "${workspaceFolder}/../.js"
       ]
-    },
-    {
-      "name": "Run Playwright Test",
-      "type": "node",
-      "request": "launch",
-      "program": "${workspaceFolder}/examples/playwright/node_modules/.bin/playwright",
-      "args": [
-        "test",
-        "--debug",
-        "--config=./configs/playwright.debug.config.ts",
-        "${fileBasenameNoExtension}"
-      ],
-      "cwd": "${workspaceFolder}/examples/playwright",
-      "console": "integratedTerminal",
-      "internalConsoleOptions": "neverOpen"
     }
   ],
   "compounds": [

examples/playwright/.eslintrc.js

@@ -1,12 +1,24 @@
 /** @type {import('eslint').Linter.Config} */
 module.exports = {
     extends: [
-        '../../configs/build.eslintrc.json',
-        './configs/ui-tests.eslintrc.json',
-        './configs/ui-tests.playwright.eslintrc.json'
+        '../../configs/build.eslintrc.json'
     ],
+    ignorePatterns: ['playwright.config.ts'],
     parserOptions: {
         tsconfigRootDir: __dirname,
-        project: 'tsconfig.json'
-    }
+        project: 'tsconfig.json',
+        // suppress warning from @typescript-eslint/typescript-estree plugin
+        warnOnUnsupportedTypeScriptVersion: false
+    },
+    overrides: [
+        {
+            files: ['*.ts'],
+            rules: {
+                // override existing rules for playwright test package
+                "no-null/no-null": "off",
+                "no-undef": "off", // disabled due to 'browser', '$', '$$'
+                "no-unused-expressions": "off"
+            }
+        }
+    ]
 };

examples/playwright/docs/DEVELOPING.md

@@ -2,26 +2,56 @@
 
 ## Building
 
-Run `yarn` in the root directory of the repository.
-In order to build Playwright and install dependencies (ex: chromium) run `yarn --cwd examples/playwright` at the root of the repository.
+Run `yarn` in the root directory of the repository to build the Theia application.
+
+In order to build Playwright library, the tests and install all dependencies (ex: chromium) run the build script:
+
+```bash
+cd examples/playwright
+yarn build
+```
 
 ## Executing the tests
 
 ### Prerequisites
 
-To work with the tests the Theia Application under test needs to be running.
+Before running your tests, the Theia application under test needs to be running.
 
-Run `yarn browser start` to start the browser-app located in this repository.
+The Playwright configuration however is aware of that and starts the backend (`yarn theia:start`) on port 3000 if not already running.
+This is valid for executing tests with the VS Code Playwright extension or from your command line.
 
 You may also use the `Launch Browser Backend` launch configuration in VS Code.
 
-### Running the tests headless
+### Running the tests in VS Code via the Playwright extension
+
+For quick and easy execution of tests in VS Code, we recommend to use the [VS Code Playwright extension (`ms-playwright.playwright`)](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright).
+
+Once you have installed the VS Code Playwright test extension, open the *Test* view and click the `Run Tests` button on the top toolbar or the `Run Test` button for a particular test.
+It uses the default configuration with chromium as test profile by default.
+
+To run the tests headful, simply enable the checkbox `Show browser` in the Playwright section of the *Test* view.
+
+### Running the tests headless via CLI
 
-To start the tests run `yarn ui-tests` in the root of this package. This will start the tests located in `src/tests` in a headless mode.
+To start the tests run `yarn ui-tests` in the folder `playwright`.
+This will start the tests in a headless state.
 
 To only run a single test file, the path of a test file can be set with `yarn ui-tests <path-to-file>` or `yarn ui-tests -g "<partial test file name>"`.
 See the [Playwright Test command line documentation](https://playwright.dev/docs/intro#command-line).
 
+### Running the tests headful via CLI
+
+If you want to observe the execution of the tests in a browser, use `yarn ui-tests-headful` for all tests or `yarn ui-tests-headful <path-to-file>` to only run a specific test.
+
+### Watch the tests
+
+Run `yarn watch` in the root of this package to rebuild the test code after each change.
+This ensures, that the executed tests are up to date also when running them with the [Playwright VS Code Extension](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright).
+
 ### Debugging the tests
 
-Please refer to the section [debugging tests](./GETTING_STARTED.md#debugging-the-tests).
+Please refer to the section [Debugging the tests via the VS Code Playwright extension](./GETTING_STARTED.md#debugging-the-tests-via-the-vs-code-playwright-extension).
+
+### UI Mode - Watch and Trace Mode
+
+Please refer to the section [UI Mode - Watch and Trace Mode](./GETTING_STARTED.md#ui-mode---watch-and-trace-mode).

examples/playwright/docs/GETTING_STARTED.md

@@ -12,7 +12,7 @@ The most important files in the theia-playwright-template are:
 
 * Example test in `tests/theia-app.test.ts`
 * Example page object in `test/page-objects/theia-app.ts`
-* Configuration files in `configs`, including the base playwright configuration at `configs/playwright.config.ts`
+* The base Playwright configuration file at `playwright.config.ts`
 * `package.json` with all required dependencies and scripts for running and debugging the tests
 
 Now, let's run the tests:
@@ -90,7 +90,7 @@ test("should undo and redo text changes and correctly update the dirty state", a
 });
 ```
 
-Below you can see this example test in action by stepping through the code with the VSCode debug tools.
+Below you can see this example test in action by stepping through the code with the VS Code debug tools.
 
 <div style='margin:0 auto;width:100%;'>
 
@@ -120,38 +120,62 @@ This keeps your tests independent of the underlying browser automation framework
 
 ## Executing tests
 
+## Building
+
+Run `yarn` in the root directory of the repository to build the Theia application.
+
+In order to build Playwright library, the tests and install all dependencies (ex: chromium) run the build script:
+
+```bash
+cd examples/playwright
+yarn build
+```
+
+### Starting the Theia Application under test
+
 Before running your tests, the Theia application under test needs to be running.
 This repository already provides an example Theia application, however, you might want to test your custom Theia-based application instead.
 
-To run the application provided in this repository, run `yarn browser start` in the root of this repository after building everything with `yarn`.
-You may also use the `Launch Browser Backend` launch configuration in VS Code.
+The Playwright configuration however is aware of that and starts the backend (`yarn theia:start`) on port 3000 if not already running.
+This is valid for executing tests with the VS Code Playwright extension or from your command line.
+
+### Running the tests in VS Code via the Playwright extension
 
-### Running the tests headless
+For quick and easy execution of tests in VS Code, we recommend to use the [VS Code Playwright extension (`ms-playwright.playwright`)](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright).
+
+Once you have installed the VS Code Playwright test extension, open the *Test* view and click the `Run Tests` button on the top toolbar or the `Run Test` button for a particular test.
+It uses the default configuration with chromium as test profile by default.
+
+To run the tests headful, simply enable the checkbox `Show browser` in the Playwright section of the *Test* view.
+
+### Running the tests headless via CLI
 
 To start the tests run `yarn ui-tests` in the folder `playwright`.
 This will start the tests in a headless state.
 
 To only run a single test file, the path of a test file can be set with `yarn ui-tests <path-to-file>` or `yarn ui-tests -g "<partial test file name>"`.
 See the [Playwright Test command line documentation](https://playwright.dev/docs/intro#command-line).
 
-### Running the tests headfull
+### Running the tests headful via CLI
 
 If you want to observe the execution of the tests in a browser, use `yarn ui-tests-headful` for all tests or `yarn ui-tests-headful <path-to-file>` to only run a specific test.
 
-### Debugging the tests
+### Debugging the tests via the VS Code Playwright extension
+
+To debug Playwright tests, open the *Test* view in VS Code and click the `Debug Tests` button on the top toolbar or the `Debug Test` for a particular test.
+It uses the default configuration with chromium as test profile by default.
 
-To debug a test, open the test file in the code editor and run the `Run Playwright Test` configuration inside VS Code.
-This will start the Playwright inspector and debug the currently opened test file.
-Using this approach, you will be able to observe the tests in the browser and set breakpoints in VSCode.
+For more information on debugging, please refer to the [Playwright documentation](https://playwright.dev/docs/debug).
 
-_Note that the tests need to be started in the playwright inspector again, as playwright pauses once the test reaches the `page.goto()` call._
+### UI Mode - Watch and Trace Mode
 
-The Playwright inspector contains an editor that shows the currently executed code and offers debugging capabilities, which can be used instead of attaching the VS code debugger.
+For an advanced test development experience, Playwright provides the so-called *UI Mode*. To enable this, simply add the flag `--ui` to the CLI command.
 
-The launched browser instance contains some additional features that are useful to debugging playwright tests. The browsers console contains a playwright object, which can be used to test the playwright API.
-For example the result of a given selector can be inspected. Additionally, this browser instance offers some quality of life improvements, such as preventing to resize the web application when the developer tools are opened, etc.
+```bash
+yarn ui-tests --ui
+```
 
-For more information on debugging, please refer to the [Playwright documentation](https://playwright.dev/docs/debug).
+For more information on the UI mode,  please refer to the [Playwright announcement of the UI mode](https://playwright.dev/docs/release-notes#introducing-ui-mode-preview).
 
 ## Advanced Topics
 

examples/playwright/package.json

@@ -12,14 +12,15 @@
   },
   "homepage": "https://github.com/eclipse-theia/theia",
   "scripts": {
-    "clean": "rimraf lib *.tsbuildinfo",
-    "build": "tsc --incremental && npx playwright install chromium",
+    "clean": "rimraf lib *.tsbuildinfo .eslintcache",
+    "build": "yarn && yarn clean && tsc --incremental && yarn lint && yarn playwright:install",
+    "watch": "tsc -w --incremental",
     "theia:start": "rimraf .tmp.cfg && THEIA_CONFIG_DIR=$PWD/.tmp.cfg yarn --cwd ../browser start",
     "lint": "eslint -c ./.eslintrc.js --ext .ts ./src",
     "lint:fix": "eslint -c ./.eslintrc.js --ext .ts ./src --fix",
-    "ui-tests": "yarn && playwright test --config=./configs/playwright.config.ts",
-    "ui-tests-ci": "yarn && playwright test --config=./configs/playwright.ci.config.ts",
-    "ui-tests-headful": "yarn && playwright test --config=./configs/playwright.headful.config.ts",
+    "playwright:install": "playwright install chromium",
+    "ui-tests": "yarn build && playwright test",
+    "ui-tests-headful": "yarn build && playwright test --headed",
     "ui-tests-report-generate": "allure generate ./allure-results --clean -o allure-results/allure-report",
     "ui-tests-report": "yarn ui-tests-report-generate && allure open allure-results/allure-report"
   },
@@ -28,13 +29,15 @@
     "src"
   ],
   "dependencies": {
-    "@playwright/test": "^1.30.0",
+    "@playwright/test": "^1.32.1",
     "fs-extra": "^9.0.8"
   },
   "devDependencies": {
     "@types/fs-extra": "^9.0.8",
-    "allure-commandline": "^2.13.8",
-    "allure-playwright": "^2.0.0-beta.14"
+    "allure-commandline": "^2.21.0",
+    "allure-playwright": "^2.1.0",
+    "rimraf": "^2.6.1",
+    "typescript": "~4.5.5"
   },
   "publishConfig": {
     "access": "public"