Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Light refactor of visual tests #5585

Merged
merged 63 commits into from
Aug 12, 2023
Merged

Conversation

unlikelyzero
Copy link
Collaborator

@unlikelyzero unlikelyzero commented Aug 1, 2022

Closes

Describe your changes:

  • Adds Example Telemetry Object to appActions which returns uuid. Is titled similar to a common VIPER telemetry object
  • Adds types to clockOptions. Provided by the sinonjs__Fake_timers
  • Updates createDomainObject with uuid (@ozyx to delete)
  • Moves Visual Test Data Generation and associated localstorage file to a new framework suite called generateLocalStorageData
  • Update all references in visual tests to only control the clock when necessary (so keep controlledClock.visual.spec.js as-is)
  • Update percycss to make times transparent and put in the percy yml file
  • ensure percy tests are picked up in junit step in circleci

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Is this change backwards compatible? For example, developers won't need to change how they are calling the API or how they've extended core plugins such as Tables or Plots.

Author Checklist

  • Changes address original issue?
  • Tests included and/or updated with changes?
  • Command line build passes?
  • Has this been smoke tested?
  • Testing instructions included in associated issue OR is this a dependency/testcase change?

Reviewer Checklist

  • Changes appear to address issue?
  • Changes appear not to be breaking changes?
  • Appropriate unit tests included?
  • Code style and in-line documentation are appropriate?
  • Commit messages meet standards?
  • Has associated issue been labelled unverified? (only applicable if this PR closes the issue)
  • Has associated issue been labelled bug? (only applicable if this PR is for a bug fix)

@codecov
Copy link

codecov bot commented Aug 1, 2022

Codecov Report

Merging #5585 (98eafd4) into master (10f3e13) will increase coverage by 1.30%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #5585      +/-   ##
==========================================
+ Coverage   46.96%   48.27%   +1.30%     
==========================================
  Files         643      643              
  Lines       25627    25627              
  Branches     2496     2496              
==========================================
+ Hits        12036    12371     +335     
+ Misses      13116    12779     -337     
- Partials      475      477       +2     
Flag Coverage Δ *Carryforward flag
e2e-full 42.02% <ø> (ø) Carriedforward from 10f3e13
e2e-stable 56.99% <ø> (+3.83%) ⬆️
unit 38.74% <0.00%> (-0.04%) ⬇️

*This pull request uses carry forward flags. Click here to find out more.

Files Changed Coverage Δ
src/plugins/plot/inspector/PlotOptionsItem.vue 0.00% <0.00%> (ø)

... and 53 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 10f3e13...98eafd4. Read the comment docs.

@unlikelyzero unlikelyzero changed the base branch from master to release/2.0.8 August 11, 2022 20:02
e2e/appActions.js Outdated Show resolved Hide resolved
e2e/baseFixtures.js Outdated Show resolved Hide resolved
e2e/pluginFixtures.js Outdated Show resolved Hide resolved
e2e/pluginFixtures.js Outdated Show resolved Hide resolved
@ozyx ozyx marked this pull request as ready for review August 12, 2022 16:41
@akhenry akhenry requested a review from scottbell August 15, 2022 21:09
e2e/appActions.js Outdated Show resolved Hide resolved
Copy link
Contributor

@scottbell scottbell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Had a few minor comments.

@ozyx ozyx requested a review from scottbell August 16, 2022 18:28
e2e/baseFixtures.js Outdated Show resolved Hide resolved
@ozyx ozyx changed the base branch from release/2.0.8 to master August 24, 2022 20:16
@ozyx ozyx force-pushed the light-refactor-of-visual-tests branch from 45c50ce to 0b8ec26 Compare September 12, 2022 17:32
@akhenry
Copy link
Contributor

akhenry commented Oct 3, 2022

@johnhill2 Looks like there are some conflicts to resolve here.

@ozyx ozyx self-requested a review March 14, 2023 04:33
Copy link
Contributor

@ozyx ozyx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall! Just a few comments regarding selectors, and possibly some changes that got reverted in the merge?

e2e/appActions.js Outdated Show resolved Hide resolved
e2e/pluginFixtures.js Outdated Show resolved Hide resolved
e2e/pluginFixtures.js Outdated Show resolved Hide resolved
e2e/tests/framework/generateLocalStorageData.e2e.spec.js Outdated Show resolved Hide resolved
e2e/tests/visual/components/about.visual.spec.js Outdated Show resolved Hide resolved
e2e/tests/visual/notebook.visual.spec.js Outdated Show resolved Hide resolved
e2e/tests/visual/notebook.visual.spec.js Outdated Show resolved Hide resolved
e2e/tests/visual/notebook.visual.spec.js Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
e2e/tests/framework/generateLocalStorageData.e2e.spec.js Outdated Show resolved Hide resolved
@shefalijoshi
Copy link
Contributor

@unlikelyzero There are conflicts.

@shefalijoshi shefalijoshi self-requested a review May 8, 2023 20:41
@deploysentinel
Copy link

deploysentinel bot commented May 10, 2023

Current Playwright Test Results Summary

✅ 14 Passing

Run may still be in progress, this comment will be updated as current testing workflow or job completes...

(Last updated on 08/12/2023 12:07:10am UTC)

Run Details

Running Workflow e2e-couchdb on Github Actions

Commit: 98eafd4

Started: 08/12/2023 12:05:17am UTC

View Detailed Build Results


Current Playwright Test Results Summary

✅ 133 Passing - ⚠️ 5 Flaky

Run may still be in progress, this comment will be updated as current testing workflow or job completes...

(Last updated on 08/12/2023 12:07:10am UTC)

Run Details

Running Job e2e-stable on CircleCI

Commit: 98eafd4

Started: 08/11/2023 11:50:36pm UTC

⚠️ Flakes

📄   functional/plugins/plot/logPlot.e2e.spec.js • 1 Flake

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Log plot tests Log Plot ticks are functionally correct in regular and log mode and after refresh
Retry 1Initial Attempt
0% (0) 0 / 91 runs
failed over last 7 days
29.67% (27) 27 / 91 runs
flaked over last 7 days

📄   functional/plugins/conditionSet/conditionSet.e2e.spec.js • 1 Flake

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Basic Condition Set Use ConditionSet should output blank instead of the default value
Retry 2Retry 1Initial Attempt
11.82% (13) 13 / 110 runs
failed over last 7 days
31.82% (35) 35 / 110 runs
flaked over last 7 days

📄   functional/plugins/imagery/exampleImagery.e2e.spec.js • 2 Flakes

Top 1 Common Error Messages

null

2 Test Cases Affected

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Example Imagery in Display Layout View Large action pauses imagery when in realtime and returns to realtime
Retry 1Initial Attempt
0% (0) 0 / 90 runs
failed over last 7 days
2.22% (2) 2 / 90 runs
flaked over last 7 days
Example Imagery in Display Layout Toggle layer visibility by clicking on label
Retry 1Initial Attempt
0% (0) 0 / 89 runs
failed over last 7 days
1.12% (1) 1 / 89 run
flaked over last 7 days

📄   functional/plugins/notebook/restrictedNotebook.e2e.spec.js • 1 Flake

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Restricted Notebook with a page locked and with an embed @addinit Allows embeds to be deleted if page unlocked @addinit
Retry 1Initial Attempt
1.10% (1) 1 / 91 run
failed over last 7 days
35.16% (32) 32 / 91 runs
flaked over last 7 days

View Detailed Build Results


Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ripped out the visual suite to move here for all consumers of localStorage (i.e. mobile)

@ozyx ozyx mentioned this pull request May 12, 2023
15 tasks
@ozyx ozyx force-pushed the light-refactor-of-visual-tests branch from 241da7f to 160a592 Compare May 19, 2023 00:04
@unlikelyzero unlikelyzero requested a review from ozyx August 11, 2023 23:38
@unlikelyzero unlikelyzero added the pr:e2e:couchdb npm run test:e2e:couchdb label Aug 11, 2023
@unlikelyzero unlikelyzero enabled auto-merge (squash) August 11, 2023 23:40
@github-actions github-actions bot removed the pr:e2e:couchdb npm run test:e2e:couchdb label Aug 11, 2023
@ozyx ozyx added the pr:e2e:couchdb npm run test:e2e:couchdb label Aug 12, 2023
@github-actions github-actions bot removed the pr:e2e:couchdb npm run test:e2e:couchdb label Aug 12, 2023
@unlikelyzero unlikelyzero merged commit 50694f6 into master Aug 12, 2023
8 of 9 checks passed
@unlikelyzero unlikelyzero deleted the light-refactor-of-visual-tests branch August 12, 2023 00:18
unlikelyzero added a commit that referenced this pull request Sep 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants