Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jest + Playwright Testing #1276

Merged
merged 131 commits into from
Oct 5, 2020
Merged
Show file tree
Hide file tree
Changes from 123 commits
Commits
Show all changes
131 commits
Select commit Hold shift + click to select a range
6eb7c90
Initial Jest+Playwright setup
jhildenbiddle Jul 7, 2020
f299c26
Fix eslint warnings
jhildenbiddle Jul 7, 2020
7e6a7b0
Add placeholder for common options
jhildenbiddle Jul 7, 2020
c35aea5
Fix load order of scriptURLs
jhildenbiddle Jul 7, 2020
08e1bbf
Add docsifyURL and waitForSelector options
jhildenbiddle Jul 7, 2020
a8c5cb7
Add executeScript scenarios
jhildenbiddle Jul 7, 2020
5229727
Enable firefox and webkit tests by default
jhildenbiddle Jul 7, 2020
045e91e
Prevent prettier from reformatting browsers list
jhildenbiddle Jul 8, 2020
83fd5fd
Update options and add JSDoc comments
jhildenbiddle Jul 8, 2020
f704f0b
Initial commit
jhildenbiddle Jul 8, 2020
25747d6
Complete initial example tests
jhildenbiddle Jul 8, 2020
e0dffb6
Minor tweaks
jhildenbiddle Jul 8, 2020
efab952
Change suite name
jhildenbiddle Jul 8, 2020
2c7e66d
Rename file
jhildenbiddle Jul 8, 2020
36c5c33
Add npm script to run jest+playwright example test
jhildenbiddle Jul 8, 2020
e8da21d
Clean up server properties
jhildenbiddle Jul 12, 2020
aed09be
Isolate e2e, integration, and unit test environments
jhildenbiddle Jul 13, 2020
3ce73f1
Update docsifyInit helper
jhildenbiddle Jul 13, 2020
2b7c54f
Add jestPlaywright.debug() to e2e examples
jhildenbiddle Jul 13, 2020
1ab3de2
Removed need to pass page as arg to docsifyInit()
jhildenbiddle Jul 14, 2020
d167031
Add VSCode debug configurations
jhildenbiddle Jul 14, 2020
251119c
Change test server port from 8080 to 3001
jhildenbiddle Jul 18, 2020
6dab0e2
Clean up test names and expect() order
jhildenbiddle Jul 20, 2020
359ead4
Update npm scripts to ignore example tests
jhildenbiddle Jul 20, 2020
1adceac
Add CLI commands and resources section
jhildenbiddle Jul 20, 2020
ff1a9f6
Add Jest unit test snapshot example
jhildenbiddle Jul 21, 2020
b73f863
Added Jest unit and e2e tests to Github workflows
jhildenbiddle Jul 21, 2020
38b0135
Add npm script to run Jest examples
jhildenbiddle Jul 21, 2020
5cf9e83
Remove node 10.x and add 14.x from tests
jhildenbiddle Jul 21, 2020
734a257
Temporarily disable testPathIgnorePatterns for ci
jhildenbiddle Jul 22, 2020
47dc27a
Temporarily remove node 14 from matrix
jhildenbiddle Jul 22, 2020
adb9862
Alternate workflow setup for new test env
jhildenbiddle Jul 22, 2020
e0e7153
Update workflow platforms and node versions
jhildenbiddle Jul 22, 2020
fe9f7c6
Restoring original workflow names
jhildenbiddle Jul 22, 2020
181f6e7
Remove Cypress tests
jhildenbiddle Jul 22, 2020
154540c
Remove/Reset GitHub actions
jhildenbiddle Jul 22, 2020
7e12c5a
Restore/reset workflows
jhildenbiddle Jul 22, 2020
b2fe355
Bump actions/checkout@v1 to v2
jhildenbiddle Jul 22, 2020
aafca2a
Use playright GH action
jhildenbiddle Jul 22, 2020
a62098b
Update playwright GH action configuration
jhildenbiddle Jul 22, 2020
c1aa72b
Remove unnecessary steps and update checkout version
jhildenbiddle Jul 22, 2020
3cb7dbb
Add install step
jhildenbiddle Jul 22, 2020
c570757
Add --ignore-scripts flag
jhildenbiddle Jul 22, 2020
e05aa3e
Swicth from ci to i for install
jhildenbiddle Jul 22, 2020
a26aae9
Add new Build & Test workflow
jhildenbiddle Jul 22, 2020
f5aed8a
Add OS tests
jhildenbiddle Jul 22, 2020
34c6c04
Wait for network idle for more reliable ready state
jhildenbiddle Jul 23, 2020
b50385c
Configure image snapshot testing and add example
jhildenbiddle Jul 23, 2020
b407a1c
Remove example fixture file
jhildenbiddle Jul 23, 2020
ba80d56
Merge branch 'develop' into jest-playwright
jhildenbiddle Jul 23, 2020
dfa5b54
Adjust image snapshot threshold for local & CI tests
jhildenbiddle Jul 23, 2020
456ed6d
Upload diff-output artifact
jhildenbiddle Jul 23, 2020
2b521aa
Add failure() check to upload-artifact
jhildenbiddle Jul 23, 2020
a91634e
Bump image snapshot threshold for local & CI tests
jhildenbiddle Jul 23, 2020
5b5c895
Set diffDirection
jhildenbiddle Jul 23, 2020
4659b4b
Fix XHR caching in playwright tests
jhildenbiddle Jul 23, 2020
9be0ffc
Update image snapshot example and theshold
jhildenbiddle Jul 23, 2020
efead40
Bump image snapshot threshold for local & CI tests
jhildenbiddle Jul 23, 2020
17628a6
Remove old workflows
jhildenbiddle Jul 23, 2020
8b2a40f
Bump image snapshot threshold for local & CI tests
jhildenbiddle Jul 23, 2020
45937cd
Bump image snapshot failureThreshold for local & CI tests
jhildenbiddle Jul 23, 2020
cc64d07
Set failureThresholdType to percent
jhildenbiddle Jul 23, 2020
9754cf9
Change image snapshot comparisonMethod to ssim
jhildenbiddle Jul 23, 2020
cbc5697
Remove pixelMatch options (incompatible with ssim)
jhildenbiddle Jul 23, 2020
7bd9156
Bump image snapshot failureThreshold
jhildenbiddle Jul 23, 2020
07ef151
Bump image snapshot failureThreshold
jhildenbiddle Jul 23, 2020
ef4e739
Disable fast-fail to allow all checks to complete
jhildenbiddle Jul 23, 2020
8903cba
Rename workflow
jhildenbiddle Jul 23, 2020
1c75d81
Disable fast-fail to allow all checks to complete
jhildenbiddle Jul 23, 2020
8f21d5e
Store pixel and ssim comparison configs
jhildenbiddle Jul 23, 2020
8c9aef7
Add link to image snapshot test
jhildenbiddle Jul 23, 2020
ee87b23
Fix CI errors on Windows due to image size mismatch
jhildenbiddle Jul 23, 2020
62b3ab9
Attempt to fix macos e2e exit code 134 issue
jhildenbiddle Jul 23, 2020
9e27b99
Update test.yml
jhildenbiddle Jul 23, 2020
e2e8c12
Update test.yml
jhildenbiddle Jul 23, 2020
7068b19
Update test.yml
jhildenbiddle Jul 23, 2020
a894851
Fix --ci flag
jhildenbiddle Jul 23, 2020
f054322
Set Jets to use --runInBand for macOS e2e
jhildenbiddle Jul 23, 2020
65375c3
Remove unnecessary macOS check
jhildenbiddle Jul 23, 2020
77e3c9d
Set image snapshot to runInProcess (macOS CI fix)
jhildenbiddle Jul 23, 2020
18183ad
Update test.yml
jhildenbiddle Jul 23, 2020
cf3f612
Temporary failureThreshold change for debugging
jhildenbiddle Jul 24, 2020
7b6b6e9
Upload os+node-specific diff artifacts
jhildenbiddle Jul 24, 2020
9cfcc43
Remove node version from diff artifact
jhildenbiddle Jul 24, 2020
ad6c189
Revert "Remove node version from diff artifact"
jhildenbiddle Jul 24, 2020
5aaa259
Revert "Revert "Remove node version from diff artifact""
jhildenbiddle Jul 24, 2020
862bcc9
Restoring failureThreshold after debugging
jhildenbiddle Jul 24, 2020
667ed6c
Remove runInProcess option
jhildenbiddle Jul 24, 2020
80a3873
Revert "Remove runInProcess option"
jhildenbiddle Jul 24, 2020
0d402d6
Add node-specific artifact uploads
jhildenbiddle Jul 24, 2020
cf45c2b
Set ssim mode to fast and restore runInProcess
jhildenbiddle Jul 24, 2020
315226f
Set failureThreshold to 0.01
jhildenbiddle Jul 24, 2020
d871d46
Updating to playwright@next for webkit fix
jhildenbiddle Jul 24, 2020
7b320b0
Restore optimal ssim configuration
jhildenbiddle Jul 24, 2020
79d141b
Testing pixel vs. ssim image comparison
jhildenbiddle Jul 24, 2020
90edd84
Move shared test setup to unit/int/e2e setups
jhildenbiddle Jul 26, 2020
71576f9
Refactor test server setup
jhildenbiddle Jul 26, 2020
4cfbe04
Replace live-server with browser-sync
jhildenbiddle Jul 27, 2020
275fa61
Update script for running local docs site
jhildenbiddle Jul 27, 2020
70d1451
Allow launching test server preview with --start
jhildenbiddle Jul 27, 2020
dedca07
Add integration tests and refactor unit + e2e
jhildenbiddle Aug 3, 2020
209ce4f
Merge branch 'develop' into jest-playwright
jhildenbiddle Aug 3, 2020
512762d
Unify docsifyInit() and cleanup
jhildenbiddle Sep 24, 2020
c5e2335
Fixed webkit routes by specifying ContentType
jhildenbiddle Sep 24, 2020
b64eb8b
Update snapshot
jhildenbiddle Sep 24, 2020
a099160
Update dependencies
jhildenbiddle Sep 24, 2020
a6fb809
Update tests to use unified docsifyInit()
jhildenbiddle Sep 24, 2020
2da83e3
Remove Cypress (old e2e test environment)
jhildenbiddle Sep 24, 2020
7b71b65
Update tests to run (unit/integration/e2e)
jhildenbiddle Sep 24, 2020
64e2989
Remove Cypress-related dependency
jhildenbiddle Sep 24, 2020
4be2845
Remove mocha+chai+jsdom (old unit/int test env)
jhildenbiddle Sep 24, 2020
f3f357c
Rename testing directories and scripts
jhildenbiddle Sep 24, 2020
0826148
- Configure test server for availability with all tests (previously e…
jhildenbiddle Sep 25, 2020
4fec434
Reorganize test files
jhildenbiddle Sep 25, 2020
da36452
Fix basePath option
jhildenbiddle Sep 26, 2020
97399e4
Replace do-mock-ajax with xhr-mock
jhildenbiddle Sep 26, 2020
e4887ae
Added new waitFor helpers
jhildenbiddle Sep 26, 2020
10e0e80
Clean up globals
jhildenbiddle Sep 30, 2020
4cbfc0a
Add try/catch for waitForFunction callback
jhildenbiddle Sep 30, 2020
79aeaa9
Move playwright config into jest.config.js
jhildenbiddle Sep 30, 2020
473bb5a
Add runInBand option to tests for reliability
jhildenbiddle Sep 30, 2020
2440a0c
Remove unnecessary XHR Mock teardown
jhildenbiddle Sep 30, 2020
9f5e562
Add —runInBand to test script
jhildenbiddle Sep 30, 2020
f6c34f8
Merge develop
jhildenbiddle Oct 3, 2020
0983d53
Cleanup
jhildenbiddle Oct 3, 2020
5ccea9d
Removed fixed host value from docs
jhildenbiddle Oct 5, 2020
af71553
Updated test after merge (docsify version change)
jhildenbiddle Oct 5, 2020
1c365fb
Added startPath for manual instance
jhildenbiddle Oct 5, 2020
4623677
Merge branch 'develop' into jest-playwright
jhildenbiddle Oct 5, 2020
befa408
Remove Node 10 from CI tests
jhildenbiddle Oct 5, 2020
d58b3ad
Remove Node 10 from CI lint tests
jhildenbiddle Oct 5, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,8 @@ packages/docsify-server-renderer/build.js
node_modules
build
server.js
cypress
lib
themes
build
docs/
**/*.md
**/*.md
1 change: 0 additions & 1 deletion .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ module.exports = {
ecmaVersion: 2019,
},
env: {
jest: true,
browser: true,
node: true,
es6: true,
Expand Down
31 changes: 0 additions & 31 deletions .github/workflows/e2e.yml

This file was deleted.

41 changes: 20 additions & 21 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -1,31 +1,30 @@
name: Linting Checks
name: Lint

on:
on:
push:
branches:
- master
- develop
branches:
- master
- develop
pull_request:
branches:
- master
- develop
branches:
- master
- develop

jobs:
build:
runs-on: ubuntu-16.04
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
node-version: [10.x, 12.x, 13.x]
node-version: [10.x, 12.x, 14.x]

steps:
- uses: actions/checkout@v1
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- name: bootstrap
run: npm run bootstrap
- name: Build
run: npm run build
- name: Linting
run: npm run lint
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- name: Install
run: npm i --ignore-scripts
- name: Lint
run: npm run lint
42 changes: 42 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Build & Test

on:
push:
branches:
- master
- develop
pull_request:
branches:
- master
- develop

jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
node-version: [10.x, 12.x, 14.x]
os: ['macos-latest', 'ubuntu-latest', 'windows-latest']

steps:
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- name: bootstrap
run: npm run bootstrap
- name: unit tests
run: npm run test:unit -- -ci --runInBand
- name: integration tests
run: npm run test:integration -- -ci --runInBand
- uses: microsoft/[email protected]
- name: e2e tests
run: npm run test:e2e -- --ci --runInBand
- name: Upload artifacts (diff output)
uses: actions/upload-artifact@v2
if: failure()
with:
name: ${{ matrix.os }}-${{ matrix.node-version }}-diff-output
path: ${{ github.workspace }}/test/**/__diff_output__/*
31 changes: 0 additions & 31 deletions .github/workflows/unit.yml

This file was deleted.

7 changes: 3 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
*.log
.DS_Store
.idea
__diff_output__
lib/
node_modules
/themes/
/lib/
/cypress/integration/examples
/cypress/fixtures/docs
themes/

# exceptions
!.gitkeep
73 changes: 73 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
Comment on lines +2 to +4
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to keep this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's default text created by VSCode when a new launch.json file is created (I didn't write it).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I don't use vscode, I use JetBrains's

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should not commit this file. The .idea folder is currently ignored, which serves the same purpose as .vscode but for JetBrains IntelliJ users.

This file may vary from user to user.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@trusktr -- Standardizing debug configurations for IDEs is more akin to standardizing NPM and CI scripts than user preferences. Developers don't want to type ./node_modules/.bin/someBinary -a arg1 -b arg2 -c var3 for frequently run tasks, so we simplify by using npm run something. Similarly, I don't want to type npm run tests:jest -- -i /path/to/file.test.js -t "name of describe of test block" just to run a single test, so running a debug script from the context of a file (so I don't have to type the path) and/or with a test name selected (so I don't have to type the test name) means I can just press a button to run a single test, making testing easier and more efficient. Win win.

There is no reason to omit these kinds of project-specific files from the repo because they should not vary between users. That's the whole purpose behind storing their configuration as files that can be managed by source control instead of storing them as user/app preferences. They should be managed like any other change to the project: users can add/modify them, submit changes as PRs, then they are reviewed and (hopefully) merged.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Someone may not want to run in integratedTerminal for example. Maybe VS Code shouldn't mix personal configs with actually-useful/convenient project configs. Not sure why they do that. F.e. why should the software project dictate if the test runs in integratedTerminal or not?

But I do see the convenience (choose a file, highlight text with cursor, hit the button and it runs only that test). No harm in keeping it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can ensure the project's required steps (f.e. npm test) are not in an editor-specific config (which is currently the case), and reserve the editor configs for conveniences only.

Copy link
Member Author

@jhildenbiddle jhildenbiddle Jul 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@trusktr --

Agreed on not putting anything critical in editor-specific scripts/configs. These are convenience only. If folks using webstorm for example want to check in the .idea folder, that's fine with me as long as I won't be required to use webstorm to perform some critical function (and you don't do something silly like force an editor theme on everyone else).

As for integrated vs external terminals, this is where I think individuals are free to modify files in their local dev environment and tell git to ignore their own changes to those files. I went with the integrated terminal because it's the default selection and it seems the least intrusive, but folks can change it locally (or if everyone wants to use an external terminal, just file PR and we'll discuss).

"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Run tests",
"runtimeExecutable": "npm",
"runtimeArgs": ["run-script", "test"],
"console": "integratedTerminal"
},
{
"type": "node",
"request": "launch",
"name": "Run current test file",
"runtimeExecutable": "npm",
"runtimeArgs": ["run-script", "test"],
"args": ["--", "-i", "${relativeFile}", "--testPathIgnorePatterns"],
"console": "integratedTerminal"
},
{
"type": "node",
"request": "launch",
"name": "Run selected test name",
"runtimeExecutable": "npm",
"runtimeArgs": ["run-script", "test"],
"args": [
"--",
"-i",
"${relativeFile}",
"-t",
"${selectedText}",
"--testPathIgnorePatterns"
],
"console": "integratedTerminal"
},
{
"type": "node",
"request": "launch",
"name": "Update current test file snapshot(s)",
"runtimeExecutable": "npm",
"runtimeArgs": ["run-script", "test"],
"args": [
"--",
"-i",
"${relativeFile}",
"--updateSnapshot",
"--testPathIgnorePatterns"
],
"console": "integratedTerminal"
},
{
"type": "node",
"request": "launch",
"name": "Update selected test name snapshot(s)",
"runtimeExecutable": "npm",
"runtimeArgs": ["run-script", "test"],
"args": [
"--",
"-i",
"${relativeFile}",
"-t",
"${selectedText}",
"--updateSnapshot",
"--testPathIgnorePatterns"
],
"console": "integratedTerminal"
}
]
}
12 changes: 12 additions & 0 deletions babel.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
module.exports = {
presets: [
[
'@babel/preset-env',
{
targets: {
node: 'current',
},
},
],
],
};
111 changes: 0 additions & 111 deletions cypress/fixtures/tpl/docs.index.html

This file was deleted.

Loading