Skip to content

Commit

Permalink
chore(e2e): still more e2e report fixes (#578)
Browse files Browse the repository at this point in the history
* chore(e2e): fix e2e result deduping for [email protected]

Example broken run: https://github.com/netlify/next-runtime-minimal/actions/runs/9410953045.

* chore(e2e): fix summary counts when deduping occurs

We were updating summary tallies as side effects of building up the report, but when I introduced
deduping of test suite retries, this led to a discrepancy between the results in the report and the
summary counts.

I could have fixed this by subtracting the right counts from the running tallies when deduping, but
it seemed cleaner to just refactor this to summary counts computed purely from the final results.

As a bonus, this makes it a lot more obvious that the skipped count makes no sense. I translated the
logic as is, but I'll try to fix this in the next commit.

* chore(e2e): more explicitly output skip counts

It was really misleading and error prone.

This also removes the total, since it wasn't necessary and it was also misleading. We were
calculating it as `passed + failed + individually skipped tests` and then only using it in one
place, which subtracted the individually skipped tests back out.

This also updates the local dev fixtures.
  • Loading branch information
serhalp authored Jun 12, 2024
1 parent 3384245 commit 9727c6b
Show file tree
Hide file tree
Showing 6 changed files with 8,098 additions and 4,287 deletions.
4 changes: 2 additions & 2 deletions e2e-report/app/page.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import Hero from '../components/hero.js'
import testData from '../data/test-results.json'

export default function Home() {
const { results, passed, failed, total, passRate, skipped, testDate, nextVersion } = testData
const { results, passed, failed, passRate, testDate, nextVersion } = testData
const skippedSuites = results.filter(({ skipped }) => skipped === true)
const skippedTestCases = results.flatMap(
({ testCases }) => testCases?.filter(({ status }) => status === 'skipped') ?? [],
Expand All @@ -16,7 +16,7 @@ export default function Home() {
return (
<>
<header className="hero">
<Hero passed={passed} failed={failed} total={total} passRate={passRate} skipped={skipped} />
<Hero passed={passed} failed={failed} passRate={passRate} />
</header>
<div className="title">
<h2>E2E Test Results</h2>
Expand Down
7 changes: 3 additions & 4 deletions e2e-report/components/hero.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import Image from 'next/image.js'

export default function Hero({ passed, failed, total, passRate, skipped }) {
const totalWithoutSkipped = total - skipped
export default function Hero({ passed, failed, passRate }) {
const total = passed + failed
return (
<>
<div className="testResults">
Expand All @@ -19,8 +19,7 @@ export default function Hero({ passed, failed, total, passRate, skipped }) {
<br /> Pass rate
</h3>
<h3>
<span>{passed.toLocaleString()}</span> of{' '}
<span>{totalWithoutSkipped.toLocaleString()}</span>
<span>{passed.toLocaleString()}</span> of <span>{total.toLocaleString()}</span>
<br /> Next.js tests passing
</h3>
<h3>
Expand Down
Loading

0 comments on commit 9727c6b

Please sign in to comment.