Jest + Playwright Testing #1276

jhildenbiddle · 2020-07-08T02:30:49Z

Summary

Added Jest test environment for unit and integration tests
Added Playwright test environment for e2e tests
Removed previous unit and integration test environments (mocha + chai + jsdom)
Removed previous e2e test environment (Cypress.io)
Ported existing unit, integration, and e2e tests of value to new environments

Details

An implementation of Jest+Playwright. This setup offers a number of advantages over the existing test structures (mocha+chai+jsdom & Cypress):

Ability to perform unit, integration, and e2e tests with one test configuration.
Separate test environments for unit + integration (Jest+jsdom) and e2e (Jest + Playwright) tests.
Testing in real browsers (Chromium, Firefox, and Webkit) for e2e tests.
Significantly faster and easier to create complex tests that require custom docsify configurations and site content thanks to a new docsifyInit() helper. This abstraction allows new docsify instances to be dynamically generated declaratively with a fraction of the code that would otherwise be required. It also removes the need to manage a large collection of fixture files (although you can use them if you prefer).
Documentation content is mapped to /docs/ on the local testing server, making it easy to use our documentation as fixtures without having to duplicate the content.
The local ./lib/ directory is also mapped to /lib/ on the local testing server, making it easy to use the latest docsify JS and CSS.
Example tests provided to help reduce the learning curve for new contributors

Changes have been made to the GitHub workflows to accommodate these changes:

A new "Build & Test" workflow has been added. This workflow runs all tests in a single workflow to prevent separate checkout+build operations for each test type. It updates the test matrix to build and test docsify on Linux (ubuntu-latest), macOS (macos-latest), and Windows (windows-latest) on the three latest LTS versions of Node (10, 12, 14).
The linting workflow has been updated to prevent building docsify, as there is no need to perform a build to link source code (the build output is ignored by eslint).

Part of the motivation for doing this work was to allow me to create proper tests for #1271. The other part was to try and clean up our test configuration so that we can offer a reliable and easy-to-learn environment for devs to work in. I looked at our existing test structures and realized there was no way for me do what I needed to do easily without creating complex tests with repeated boilerplate code and a large number of fixtures. Specifically, testing #1271 requires multiple docsify configurations that must be run in a browser with and without additional third-party scripts (vue.js). With this setup, every custom docisfy test is between 8 and 11 lines of code and zero test-specific file fixtures.

A few tips for reviewers:

All Jest and Playwright code is isolated in a new ./test/ directory. This means this PR can safely be merged and it will not affect existing tests.
Start by reviewing /test/README.md. This file provides an overview of the test environments, global variables, and CLI commands available.
Review ./jest.config.js and note that we are using Jest projects to simplify unique environment configurations for unit, integration, and e2e tests.
Review ./test/helpers/docsify-init.js. This helper greatly simplifies the process of creating dynamic docsify sites for individual tests.
For Jest (and jsdom) testing, review /test/unit/example.test.js and /test/integration/example.test.js.
For Jest + Playwright testing, review /test/e2e/example.test.js.
I've created real-world tests for vue.js reactivity (the reason Fix Vue Reactivity #1271 exists). These are available for review in /test/e2e/vue.test.js;
VSCode debug configurations have been added to make testings a single test or file easier. You can access these by clicking the debug icon on VSCode's sidebar (or pressing SHFT+CMD/CTRL+D). The two tests currently in place are:
- Run current test file
- Run selected test name (requires highlighting describe() or test() name)
- Update current test file snapshot(s)
- Update selected test name snapshot(s) (requires highlighting describe() or test() name)

What kind of change does this PR introduce? (check at least one)

Does this PR introduce a breaking change? (check one)

Yes
No

The PR fulfills these requirements:

When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)

Fix #1217

You have tested in the following browsers: (Providing a detailed version will be better.)

If adding a new feature, the PR's description includes:

A convincing reason for adding this feature
Related documents have been updated
Related tests have been updated

vercel · 2020-07-08T02:30:55Z

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/docsify-core/docsify-preview/fs42flso2
✅ Preview: https://docsify-preview-git-jest-playwright.docsify-core.vercel.app

codesandbox-ci · 2020-07-08T02:34:58Z

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit d58b3ad:

Sandbox	Source
docsify-template	Configuration

trusktr · 2020-07-08T04:50:25Z

Nice! I will take a look this weekend!

- Setup projects for e2e, integration, and unit tests in Jest configuration - Setup /tests/e2e/ tests for Jest+PlayWright - Setup /tests/integration/ tests for Jest - Setup /tests/unit/ tests for Jest - Setup eslint for Jest and Jest+Playwright environments - Move e2e configuration files into separate folder - Update e2e example tests - Update unit example tests - Update HTML fixtures

- Add `debug` option - Append `Markdown` suffix to markdown-related options - Reorder options alphabetically

sy-records · 2020-10-01T03:02:40Z

LGTM.

jhildenbiddle · 2020-10-01T16:19:50Z

@anikethsaha --

Few non-blocking suggestion/question

is it safe to remove the cypress now ?

Yep. The Cypress tests we had were limited (only tested sidebar links on /doc site which we can do much more easily with snapshots), slow, and brittle. There's no reason to hang on to the tests or the dependencies.

can we add the new script in the github action

GitHub Actions have already been updated in this PR. Unfortunately, the workflows aren't running because of GitHub issue that prevents workflows from running on PRs that have merge conflicts. The conflicts that need to be resolved are very minor and easily resolved. Once resolved and merged, workflows should run as expected.

not sure what is unit in the tests. I couldn't notice any jsdom imports, are those just mockups ?

Jest comes with jsdom baked in. No need for a separate jsdom dependency or import.

jhildenbiddle · 2020-10-01T18:35:02Z

@sy-records --

LGTM.

I just flipped the PR from "draft" to "ready". Can you approve the PR if everything looks good?

jhildenbiddle · 2020-10-01T18:36:36Z

@anikethsaha --

The whole setup and tests looks good to me.

I just flipped the PR from "draft" to "ready". Can you approve the PR if everything looks good?

Koooooo-7

@jhildenbiddle Awesome.

anikethsaha

Thanks

- Relocate carbon plugin script with other scripts - Update zh-cn docs URL to align with other translatins (GitHub, not jsdelivr) - Add major version locks to URLs

# Conflicts: # cypress/fixtures/tpl/docs.index.html # docs/index.html # package-lock.json # test/unit/render.test.js

* develop: (81 commits) fix: upgrade dompurify from 2.1.0 to 2.1.1 (#1402) fix: upgrade dompurify from 2.0.17 to 2.1.0 (#1397) fix: search on homepage test (#1398) fix: the sidebar links to another site. (#1336) fix: Can't search homepage content (#1391) fix: upgrade debug from 4.1.1 to 4.3.0 (#1390) fix: packages/docsify-server-renderer/package.json & packages/docsify-server-renderer/package-lock.json to reduce vulnerabilities (#1389) Fix eslint warnings (#1388) docs: add crossOriginLinks configurations details. (#1386) Remove Cypress screenshots Fix friendly message display Add Vue 3 compatibility Show dir listing & help msg for manual instance Add NODE_MODULES_URL global Jest + Playwright Testing (#1276) update doc (#1381) Fix scroll event end value fix: upgrade docsify from 4.11.4 to 4.11.6 (#1373) chore(deps): bump node-fetch in /packages/docsify-server-renderer (#1370) test: fix cannot search list content (#1367) ...

jhildenbiddle added 15 commits July 7, 2020 06:31

Initial Jest+Playwright setup

6eb7c90

Fix eslint warnings

f299c26

Add placeholder for common options

7e6a7b0

Fix load order of scriptURLs

c35aea5

Add docsifyURL and waitForSelector options

08e1bbf

Add executeScript scenarios

a8c5cb7

Enable firefox and webkit tests by default

5229727

Prevent prettier from reformatting browsers list

045e91e

Update options and add JSDoc comments

83fd5fd

Initial commit

f704f0b

Complete initial example tests

25747d6

Minor tweaks

e0dffb6

Change suite name

efab952

Rename file

2c7e66d

Add npm script to run jest+playwright example test

36c5c33

jhildenbiddle added the tests label Jul 8, 2020

jhildenbiddle requested review from anikethsaha, trusktr, sy-records, Koooooo-7 and QingWei-Li July 8, 2020 02:31

jhildenbiddle added 6 commits July 12, 2020 11:38

Clean up server properties

e8da21d

Update docsifyInit helper

3ce73f1

- Add `debug` option - Append `Markdown` suffix to markdown-related options - Reorder options alphabetically

Add jestPlaywright.debug() to e2e examples

2b7c54f

Removed need to pass page as arg to docsifyInit()

1ab3de2

Add VSCode debug configurations

d167031

vercel bot deployed to Preview September 30, 2020 22:32 View deployment

jhildenbiddle marked this pull request as ready for review October 1, 2020 16:12

sy-records previously approved these changes Oct 2, 2020

View reviewed changes

Koooooo-7 previously approved these changes Oct 2, 2020

View reviewed changes

anikethsaha previously approved these changes Oct 3, 2020

View reviewed changes

jhildenbiddle added 5 commits October 3, 2020 18:25

Merge develop

f6c34f8

Cleanup

0983d53

- Relocate carbon plugin script with other scripts - Update zh-cn docs URL to align with other translatins (GitHub, not jsdelivr) - Add major version locks to URLs

Removed fixed host value from docs

5ccea9d

Updated test after merge (docsify version change)

af71553

Added startPath for manual instance

1c365fb

jhildenbiddle dismissed stale reviews from anikethsaha, Koooooo-7, and sy-records via 1c365fb October 5, 2020 18:07

vercel bot deployed to Preview October 5, 2020 18:07 View deployment

Merge branch 'develop' into jest-playwright

4623677

# Conflicts: # cypress/fixtures/tpl/docs.index.html # docs/index.html # package-lock.json # test/unit/render.test.js

vercel bot deployed to Preview October 5, 2020 18:12 View deployment

Remove Node 10 from CI tests

befa408

vercel bot deployed to Preview October 5, 2020 18:20 View deployment

Remove Node 10 from CI lint tests

d58b3ad

vercel bot deployed to Preview October 5, 2020 18:24 View deployment

jhildenbiddle merged commit 7f0c42e into develop Oct 5, 2020

sy-records deleted the jest-playwright branch October 5, 2020 23:54

jhildenbiddle mentioned this pull request Oct 8, 2020

fix: Cannot serve off /.../index.html #1372

Merged

19 tasks

trusktr mentioned this pull request Nov 2, 2020

[WIP] Finish initial SSR (server side rendering) and add tests. #1227

Draft

21 tasks

sy-records mentioned this pull request Feb 5, 2021

[Release]: v4.12.0 #1493

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Jest + Playwright Testing #1276

Jest + Playwright Testing #1276

jhildenbiddle commented Jul 8, 2020 •

edited

Loading

vercel bot commented Jul 8, 2020 •

edited

Loading

codesandbox-ci bot commented Jul 8, 2020 •

edited

Loading

trusktr commented Jul 8, 2020

sy-records commented Oct 1, 2020

jhildenbiddle commented Oct 1, 2020

jhildenbiddle commented Oct 1, 2020

jhildenbiddle commented Oct 1, 2020

Koooooo-7 left a comment

anikethsaha left a comment

Jest + Playwright Testing #1276

Jest + Playwright Testing #1276

Conversation

jhildenbiddle commented Jul 8, 2020 • edited Loading

vercel bot commented Jul 8, 2020 • edited Loading

codesandbox-ci bot commented Jul 8, 2020 • edited Loading

trusktr commented Jul 8, 2020

sy-records commented Oct 1, 2020

jhildenbiddle commented Oct 1, 2020

jhildenbiddle commented Oct 1, 2020

jhildenbiddle commented Oct 1, 2020

Koooooo-7 left a comment

Choose a reason for hiding this comment

anikethsaha left a comment

Choose a reason for hiding this comment

jhildenbiddle commented Jul 8, 2020 •

edited

Loading

vercel bot commented Jul 8, 2020 •

edited

Loading

codesandbox-ci bot commented Jul 8, 2020 •

edited

Loading