-
-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Jest + Playwright Testing #1276
Merged
Merged
Changes from 123 commits
Commits
Show all changes
131 commits
Select commit
Hold shift + click to select a range
6eb7c90
Initial Jest+Playwright setup
jhildenbiddle f299c26
Fix eslint warnings
jhildenbiddle 7e6a7b0
Add placeholder for common options
jhildenbiddle c35aea5
Fix load order of scriptURLs
jhildenbiddle 08e1bbf
Add docsifyURL and waitForSelector options
jhildenbiddle a8c5cb7
Add executeScript scenarios
jhildenbiddle 5229727
Enable firefox and webkit tests by default
jhildenbiddle 045e91e
Prevent prettier from reformatting browsers list
jhildenbiddle 83fd5fd
Update options and add JSDoc comments
jhildenbiddle f704f0b
Initial commit
jhildenbiddle 25747d6
Complete initial example tests
jhildenbiddle e0dffb6
Minor tweaks
jhildenbiddle efab952
Change suite name
jhildenbiddle 2c7e66d
Rename file
jhildenbiddle 36c5c33
Add npm script to run jest+playwright example test
jhildenbiddle e8da21d
Clean up server properties
jhildenbiddle aed09be
Isolate e2e, integration, and unit test environments
jhildenbiddle 3ce73f1
Update docsifyInit helper
jhildenbiddle 2b7c54f
Add jestPlaywright.debug() to e2e examples
jhildenbiddle 1ab3de2
Removed need to pass page as arg to docsifyInit()
jhildenbiddle d167031
Add VSCode debug configurations
jhildenbiddle 251119c
Change test server port from 8080 to 3001
jhildenbiddle 6dab0e2
Clean up test names and expect() order
jhildenbiddle 359ead4
Update npm scripts to ignore example tests
jhildenbiddle 1adceac
Add CLI commands and resources section
jhildenbiddle ff1a9f6
Add Jest unit test snapshot example
jhildenbiddle b73f863
Added Jest unit and e2e tests to Github workflows
jhildenbiddle 38b0135
Add npm script to run Jest examples
jhildenbiddle 5cf9e83
Remove node 10.x and add 14.x from tests
jhildenbiddle 734a257
Temporarily disable testPathIgnorePatterns for ci
jhildenbiddle 47dc27a
Temporarily remove node 14 from matrix
jhildenbiddle adb9862
Alternate workflow setup for new test env
jhildenbiddle e0e7153
Update workflow platforms and node versions
jhildenbiddle fe9f7c6
Restoring original workflow names
jhildenbiddle 181f6e7
Remove Cypress tests
jhildenbiddle 154540c
Remove/Reset GitHub actions
jhildenbiddle 7e12c5a
Restore/reset workflows
jhildenbiddle b2fe355
Bump actions/checkout@v1 to v2
jhildenbiddle aafca2a
Use playright GH action
jhildenbiddle a62098b
Update playwright GH action configuration
jhildenbiddle c1aa72b
Remove unnecessary steps and update checkout version
jhildenbiddle 3cb7dbb
Add install step
jhildenbiddle c570757
Add --ignore-scripts flag
jhildenbiddle e05aa3e
Swicth from ci to i for install
jhildenbiddle a26aae9
Add new Build & Test workflow
jhildenbiddle f5aed8a
Add OS tests
jhildenbiddle 34c6c04
Wait for network idle for more reliable ready state
jhildenbiddle b50385c
Configure image snapshot testing and add example
jhildenbiddle b407a1c
Remove example fixture file
jhildenbiddle ba80d56
Merge branch 'develop' into jest-playwright
jhildenbiddle dfa5b54
Adjust image snapshot threshold for local & CI tests
jhildenbiddle 456ed6d
Upload diff-output artifact
jhildenbiddle 2b521aa
Add failure() check to upload-artifact
jhildenbiddle a91634e
Bump image snapshot threshold for local & CI tests
jhildenbiddle 5b5c895
Set diffDirection
jhildenbiddle 4659b4b
Fix XHR caching in playwright tests
jhildenbiddle 9be0ffc
Update image snapshot example and theshold
jhildenbiddle efead40
Bump image snapshot threshold for local & CI tests
jhildenbiddle 17628a6
Remove old workflows
jhildenbiddle 8b2a40f
Bump image snapshot threshold for local & CI tests
jhildenbiddle 45937cd
Bump image snapshot failureThreshold for local & CI tests
jhildenbiddle cc64d07
Set failureThresholdType to percent
jhildenbiddle 9754cf9
Change image snapshot comparisonMethod to ssim
jhildenbiddle cbc5697
Remove pixelMatch options (incompatible with ssim)
jhildenbiddle 7bd9156
Bump image snapshot failureThreshold
jhildenbiddle 07ef151
Bump image snapshot failureThreshold
jhildenbiddle ef4e739
Disable fast-fail to allow all checks to complete
jhildenbiddle 8903cba
Rename workflow
jhildenbiddle 1c75d81
Disable fast-fail to allow all checks to complete
jhildenbiddle 8f21d5e
Store pixel and ssim comparison configs
jhildenbiddle 8c9aef7
Add link to image snapshot test
jhildenbiddle ee87b23
Fix CI errors on Windows due to image size mismatch
jhildenbiddle 62b3ab9
Attempt to fix macos e2e exit code 134 issue
jhildenbiddle 9e27b99
Update test.yml
jhildenbiddle e2e8c12
Update test.yml
jhildenbiddle 7068b19
Update test.yml
jhildenbiddle a894851
Fix --ci flag
jhildenbiddle f054322
Set Jets to use --runInBand for macOS e2e
jhildenbiddle 65375c3
Remove unnecessary macOS check
jhildenbiddle 77e3c9d
Set image snapshot to runInProcess (macOS CI fix)
jhildenbiddle 18183ad
Update test.yml
jhildenbiddle cf3f612
Temporary failureThreshold change for debugging
jhildenbiddle 7b6b6e9
Upload os+node-specific diff artifacts
jhildenbiddle 9cfcc43
Remove node version from diff artifact
jhildenbiddle ad6c189
Revert "Remove node version from diff artifact"
jhildenbiddle 5aaa259
Revert "Revert "Remove node version from diff artifact""
jhildenbiddle 862bcc9
Restoring failureThreshold after debugging
jhildenbiddle 667ed6c
Remove runInProcess option
jhildenbiddle 80a3873
Revert "Remove runInProcess option"
jhildenbiddle 0d402d6
Add node-specific artifact uploads
jhildenbiddle cf45c2b
Set ssim mode to fast and restore runInProcess
jhildenbiddle 315226f
Set failureThreshold to 0.01
jhildenbiddle d871d46
Updating to playwright@next for webkit fix
jhildenbiddle 7b320b0
Restore optimal ssim configuration
jhildenbiddle 79d141b
Testing pixel vs. ssim image comparison
jhildenbiddle 90edd84
Move shared test setup to unit/int/e2e setups
jhildenbiddle 71576f9
Refactor test server setup
jhildenbiddle 4cfbe04
Replace live-server with browser-sync
jhildenbiddle 275fa61
Update script for running local docs site
jhildenbiddle 70d1451
Allow launching test server preview with --start
jhildenbiddle dedca07
Add integration tests and refactor unit + e2e
jhildenbiddle 209ce4f
Merge branch 'develop' into jest-playwright
jhildenbiddle 512762d
Unify docsifyInit() and cleanup
jhildenbiddle c5e2335
Fixed webkit routes by specifying ContentType
jhildenbiddle b64eb8b
Update snapshot
jhildenbiddle a099160
Update dependencies
jhildenbiddle a6fb809
Update tests to use unified docsifyInit()
jhildenbiddle 2da83e3
Remove Cypress (old e2e test environment)
jhildenbiddle 7b71b65
Update tests to run (unit/integration/e2e)
jhildenbiddle 64e2989
Remove Cypress-related dependency
jhildenbiddle 4be2845
Remove mocha+chai+jsdom (old unit/int test env)
jhildenbiddle f3f357c
Rename testing directories and scripts
jhildenbiddle 0826148
- Configure test server for availability with all tests (previously e…
jhildenbiddle 4fec434
Reorganize test files
jhildenbiddle da36452
Fix basePath option
jhildenbiddle 97399e4
Replace do-mock-ajax with xhr-mock
jhildenbiddle e4887ae
Added new waitFor helpers
jhildenbiddle 10e0e80
Clean up globals
jhildenbiddle 4cbfc0a
Add try/catch for waitForFunction callback
jhildenbiddle 79aeaa9
Move playwright config into jest.config.js
jhildenbiddle 473bb5a
Add runInBand option to tests for reliability
jhildenbiddle 2440a0c
Remove unnecessary XHR Mock teardown
jhildenbiddle 9f5e562
Add —runInBand to test script
jhildenbiddle f6c34f8
Merge develop
jhildenbiddle 0983d53
Cleanup
jhildenbiddle 5ccea9d
Removed fixed host value from docs
jhildenbiddle af71553
Updated test after merge (docsify version change)
jhildenbiddle 1c365fb
Added startPath for manual instance
jhildenbiddle 4623677
Merge branch 'develop' into jest-playwright
jhildenbiddle befa408
Remove Node 10 from CI tests
jhildenbiddle d58b3ad
Remove Node 10 from CI lint tests
jhildenbiddle File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -6,7 +6,6 @@ module.exports = { | |
ecmaVersion: 2019, | ||
}, | ||
env: { | ||
jest: true, | ||
browser: true, | ||
node: true, | ||
es6: true, | ||
|
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,31 +1,30 @@ | ||
name: Linting Checks | ||
name: Lint | ||
|
||
on: | ||
on: | ||
push: | ||
branches: | ||
- master | ||
- develop | ||
branches: | ||
- master | ||
- develop | ||
pull_request: | ||
branches: | ||
- master | ||
- develop | ||
branches: | ||
- master | ||
- develop | ||
|
||
jobs: | ||
build: | ||
runs-on: ubuntu-16.04 | ||
runs-on: ubuntu-latest | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
node-version: [10.x, 12.x, 13.x] | ||
node-version: [10.x, 12.x, 14.x] | ||
|
||
steps: | ||
- uses: actions/checkout@v1 | ||
- name: Use Node.js ${{ matrix.node-version }} | ||
uses: actions/setup-node@v1 | ||
with: | ||
node-version: ${{ matrix.node-version }} | ||
- name: bootstrap | ||
run: npm run bootstrap | ||
- name: Build | ||
run: npm run build | ||
- name: Linting | ||
run: npm run lint | ||
- uses: actions/checkout@v2 | ||
- name: Use Node.js ${{ matrix.node-version }} | ||
uses: actions/setup-node@v1 | ||
with: | ||
node-version: ${{ matrix.node-version }} | ||
- name: Install | ||
run: npm i --ignore-scripts | ||
- name: Lint | ||
run: npm run lint |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
name: Build & Test | ||
|
||
on: | ||
push: | ||
branches: | ||
- master | ||
- develop | ||
pull_request: | ||
branches: | ||
- master | ||
- develop | ||
|
||
jobs: | ||
build: | ||
runs-on: ${{ matrix.os }} | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
node-version: [10.x, 12.x, 14.x] | ||
os: ['macos-latest', 'ubuntu-latest', 'windows-latest'] | ||
|
||
steps: | ||
- uses: actions/checkout@v2 | ||
- name: Use Node.js ${{ matrix.node-version }} | ||
uses: actions/setup-node@v1 | ||
with: | ||
node-version: ${{ matrix.node-version }} | ||
- name: bootstrap | ||
run: npm run bootstrap | ||
- name: unit tests | ||
run: npm run test:unit -- -ci --runInBand | ||
- name: integration tests | ||
run: npm run test:integration -- -ci --runInBand | ||
- uses: microsoft/[email protected] | ||
- name: e2e tests | ||
run: npm run test:e2e -- --ci --runInBand | ||
- name: Upload artifacts (diff output) | ||
uses: actions/upload-artifact@v2 | ||
if: failure() | ||
with: | ||
name: ${{ matrix.os }}-${{ matrix.node-version }}-diff-output | ||
path: ${{ github.workspace }}/test/**/__diff_output__/* |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,10 @@ | ||
*.log | ||
.DS_Store | ||
.idea | ||
__diff_output__ | ||
lib/ | ||
node_modules | ||
/themes/ | ||
/lib/ | ||
/cypress/integration/examples | ||
/cypress/fixtures/docs | ||
themes/ | ||
|
||
# exceptions | ||
!.gitkeep |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
{ | ||
// Use IntelliSense to learn about possible attributes. | ||
// Hover to view descriptions of existing attributes. | ||
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 | ||
"version": "0.2.0", | ||
"configurations": [ | ||
{ | ||
"type": "node", | ||
"request": "launch", | ||
"name": "Run tests", | ||
"runtimeExecutable": "npm", | ||
"runtimeArgs": ["run-script", "test"], | ||
"console": "integratedTerminal" | ||
}, | ||
{ | ||
"type": "node", | ||
"request": "launch", | ||
"name": "Run current test file", | ||
"runtimeExecutable": "npm", | ||
"runtimeArgs": ["run-script", "test"], | ||
"args": ["--", "-i", "${relativeFile}", "--testPathIgnorePatterns"], | ||
"console": "integratedTerminal" | ||
}, | ||
{ | ||
"type": "node", | ||
"request": "launch", | ||
"name": "Run selected test name", | ||
"runtimeExecutable": "npm", | ||
"runtimeArgs": ["run-script", "test"], | ||
"args": [ | ||
"--", | ||
"-i", | ||
"${relativeFile}", | ||
"-t", | ||
"${selectedText}", | ||
"--testPathIgnorePatterns" | ||
], | ||
"console": "integratedTerminal" | ||
}, | ||
{ | ||
"type": "node", | ||
"request": "launch", | ||
"name": "Update current test file snapshot(s)", | ||
"runtimeExecutable": "npm", | ||
"runtimeArgs": ["run-script", "test"], | ||
"args": [ | ||
"--", | ||
"-i", | ||
"${relativeFile}", | ||
"--updateSnapshot", | ||
"--testPathIgnorePatterns" | ||
], | ||
"console": "integratedTerminal" | ||
}, | ||
{ | ||
"type": "node", | ||
"request": "launch", | ||
"name": "Update selected test name snapshot(s)", | ||
"runtimeExecutable": "npm", | ||
"runtimeArgs": ["run-script", "test"], | ||
"args": [ | ||
"--", | ||
"-i", | ||
"${relativeFile}", | ||
"-t", | ||
"${selectedText}", | ||
"--updateSnapshot", | ||
"--testPathIgnorePatterns" | ||
], | ||
"console": "integratedTerminal" | ||
} | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
module.exports = { | ||
presets: [ | ||
[ | ||
'@babel/preset-env', | ||
{ | ||
targets: { | ||
node: 'current', | ||
}, | ||
}, | ||
], | ||
], | ||
}; |
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it necessary to keep this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's default text created by VSCode when a new launch.json file is created (I didn't write it).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, I don't use vscode, I use JetBrains's
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should not commit this file. The
.idea
folder is currently ignored, which serves the same purpose as.vscode
but for JetBrains IntelliJ users.This file may vary from user to user.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@trusktr -- Standardizing debug configurations for IDEs is more akin to standardizing NPM and CI scripts than user preferences. Developers don't want to type
./node_modules/.bin/someBinary -a arg1 -b arg2 -c var3
for frequently run tasks, so we simplify by usingnpm run something
. Similarly, I don't want to typenpm run tests:jest -- -i /path/to/file.test.js -t "name of describe of test block"
just to run a single test, so running a debug script from the context of a file (so I don't have to type the path) and/or with a test name selected (so I don't have to type the test name) means I can just press a button to run a single test, making testing easier and more efficient. Win win.There is no reason to omit these kinds of project-specific files from the repo because they should not vary between users. That's the whole purpose behind storing their configuration as files that can be managed by source control instead of storing them as user/app preferences. They should be managed like any other change to the project: users can add/modify them, submit changes as PRs, then they are reviewed and (hopefully) merged.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Someone may not want to run in
integratedTerminal
for example. Maybe VS Code shouldn't mix personal configs with actually-useful/convenient project configs. Not sure why they do that. F.e. why should the software project dictate if the test runs inintegratedTerminal
or not?But I do see the convenience (choose a file, highlight text with cursor, hit the button and it runs only that test). No harm in keeping it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can ensure the project's required steps (f.e.
npm test
) are not in an editor-specific config (which is currently the case), and reserve the editor configs for conveniences only.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@trusktr --
Agreed on not putting anything critical in editor-specific scripts/configs. These are convenience only. If folks using webstorm for example want to check in the
.idea
folder, that's fine with me as long as I won't be required to use webstorm to perform some critical function (and you don't do something silly like force an editor theme on everyone else).As for integrated vs external terminals, this is where I think individuals are free to modify files in their local dev environment and tell git to ignore their own changes to those files. I went with the integrated terminal because it's the default selection and it seems the least intrusive, but folks can change it locally (or if everyone wants to use an external terminal, just file PR and we'll discuss).