-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Some autotests failing in Firefox #1075
Comments
## 🤨 Rationale Relates to #1075 I fixed 7 of the 17 tests that were failing on Firefox. We want to start running the Firefox tests on the CI (and `npm run test`), but skip the ones known to be failing. ## 👩💻 Implementation Malcolm pointed me to a package called `karma-jasmine-spec-tags` which they had used on the WebVI team. This package allows running or skipping tests based on arbitrary tags found in the test names. I've added "#SkipFirefox" to the 10 remaining tests that are still failing on Firefox. I then added `--skip-tags SkipFirefox` to the arguments passed when executing `npm run test-firefox`. I added similar skipping support for Chrome and Webkit. I documented our policy for skipping tests in the CONTRIBUTING. ## 🧪 Testing Tested that the tagged tests are indeed skipped now, and the rest are being run on the CI. ## ✅ Checklist - [x] I have updated the project documentation to reflect my changes or determined no changes are needed. --------- Co-authored-by: Milan Raj <[email protected]>
# Pull Request ## 🤨 Rationale As part of #1156 a change was made to tooltip icon tests to run in Firefox. The implemented solution added arbitrary additional waits which are turning out to be intermittent. This PR disables those tests until a more robust implementation can be found. Additional investigation is being skipped to get main stable at the moment. Fixes #1166 (fixes the intermittency on main, and #1075 covers fixing the actual disabled tests) I'm also disabling the intermittent test associated with #1172 as it was causing failues locally trying to reproduce the intermittency of this test. ## 👩💻 Implementation Revert the test changes and place them into the existing original bucket of firefox failing tests. ## 🧪 Testing Relying on CI ## ✅ Checklist - [x] I have updated the project documentation to reflect my changes or determined no changes are needed.
All of the dialog/drawer tests that deal with the focusing of a certain control are due to a bug/compatibility issue in Firefox. I have filed an issue with webcompat. Basically, the dialog element fails to auto-focus elements if they are slotted. With a plain dialog (no slotting), it focuses just fine. |
## 🤨 Rationale See #1075 Six tooltip tests listed in that bug were failing on Firefox, so we marked them to be skipped. ## 👩💻 Implementation The tests were checking for the display property of icons before the anchored region of the tooltip had been loaded. This was resulting in `getComputedStyle` for the icon to return an empty set of style info. Waiting for the `loaded` event of the anchored region results in the tests passing. ## 🧪 Testing Ran tests locally ## ✅ Checklist - [x] I have updated the project documentation to reflect my changes or determined no changes are needed.
Next action item is to take another pass at addressing the failures. Anything that's left should be broken into separate issues for each underlying cause. Those issues should be appropriately marked as blocked/upstream with bugs filed to browser vendors. |
Updated the issue description to link to specific issues for each root cause. Closing this one. |
# Pull Request ## 🤨 Rationale One of the tasks in #1747 is to run nimble-components tests in WebKit and see if there are any new failures. There are several. We also wanted to start tracking specific issues for each root cause rather than catch all issues like #1074 and #1075. ## 👩💻 Implementation Marked each failing test with one of these specific issues: - #1936 - #1938 TODO: ALSO FILE AZDO BUG - #1939 - #1940 - #1942 - #1943 Also re-enabled a couple of table header tests in Firefox which are now passing. ## 🧪 Testing Ran tests in Playwright webkit browser, Firefox, and Safari and and all the enabled tests now pass. ## ✅ Checklist <!--- Review the list and put an x in the boxes that apply or ~~strike through~~ around items that don't (along with an explanation). --> - [x] I have updated the project documentation to reflect my changes or determined no changes are needed. --------- Co-authored-by: Milan Raj <[email protected]>
🧹 Tech Debt
We currently have some test failures in Firefox. We should triage these to see if there's any user-visible impact (i.e. if they could be actual bugs in Nimble on Firefox) and fix them if possible.
Note: The drawer/dialog test failures are also present in WebKit. If we find a fix for one browser, we should test it in the other too.
To reproduce:
npm run test-firefox -w @ni/nimble-components
NO LONGER FAILING:
Expected null to equal 'descending'.
Expected null to equal 'ascending'.
TRACKED BY #1943
Expected ... to be . Tip: To check for deep equality, use .toEqual() instead of .toBe().
Expected ... to be . Tip: To check for deep equality, use .toEqual() instead of .toBe().
TRACKED BY #1937
Error: Timeout - Async function did not complete within 5000ms (set by jasmine.DEFAULT_TIMEOUT_INTERVAL)
TRACKED BY WEBCOMPAT BUG (https://webcompat.com/issues/121098) and #1936 :
Expected ... to be .... Tip: To check for deep equality, use .toEqual() instead of .toBe().
Expected ... to be .... Tip: To check for deep equality, use .toEqual() instead of .toBe().
Expected ... to be .... Tip: To check for deep equality, use .toEqual() instead of .toBe().
Expected ... to be .... Tip: To check for deep equality, use .toEqual() instead of .toBe().
FIXED:
Expected undefined to be 'true'.
Expected true to be false.
Expected true to be false.
Expected true to be false.
Expected true to be false.
Expected true to be false.
Expected true to be false.
Expected '' to be 'a very long value that should get ellipsized due to not fitting within the default cell width'.
The text was updated successfully, but these errors were encountered: