-
Notifications
You must be signed in to change notification settings - Fork 9.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tests: fix coverage generation #7475
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
"unit": "yarn unit-core && yarn unit-cli && yarn unit-viewer", | ||
"unit:ci": "yarn unit-core:ci && yarn unit-cli:ci && yarn unit-viewer", | ||
"unit:ci": "npm run unit-core -- --runInBand --ci --coverage && npm run unit-cli -- --runInBand --ci --coverage && npm run unit-viewer -- --runInBand --ci --coverage", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is a fun line to read :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is a fun line to read :)
ha, happy for ideas on how to make it more readable :)
"unit-cli:ci": "jest --runInBand --coverage --ci \"lighthouse-cli/\"", | ||
"unit-viewer": "jest \"lighthouse-viewer/\"", | ||
"unit-cli": "jest \"lighthouse-cli/\"", | ||
"unit-viewer": "jest lighthouse-viewer/**/*-test.js", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
was this bringing in too many things the old way?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
was this bringing in too many things the old way?
yeah, when we switched the viewer puppeteer test from mocha to jest, that meant the looser jest lighthouse-viewer/
was running that test as a unit test which seems wrong and doesn't help coverage anyways because it runs against the built viewer in dist/
, not the original source files.
Maybe the puppeteer tests should live somewhere else?
added another commit
|
here's my take on fixing coverage :)
This seems to be the minimal fix for what appears to be two separate problems:
--hook-run-in-context
appears to be the key here, so I assume something changed between howjest
and/oryarn
create subprocesses and/or hownyc
hooks into themyarn coverage
andyarn unit:ci
fromyarn
tonpm run
.I also got rid of
unit-cli:ci
andunit-core:ci
, since these were only called within npm scripts, and we can simplify by havingyarn unit:ci
pass in those extra arguments instead.(my original approach was to stop using
nyc
to generate coverage and just usejest
for that directly. We could still do this, but it turns out merging multiplejest
coverage reports with existing lcov tools is a pain, so I vote leaving that for another day since we still neednyc
for smoke coverage anyways)(comment from below)
added another commit
renamed
unit:silentcoverage
tounit:cicoverage
(andsmoke:cicoverage
) as that's what it really means (making an lcov file for uploading to coveralls/codecov). Removed the--silent
which was really just for silencing the"text"
reporter...but we don't need to run the"text"
reporter, so just removed that :) Replaced with the three line"text-summary"
reporter, which seems fine in the CI outputunit:coverage
can now just callunit:cicoverage
(same withcoverage:smoke
) so we don't need thenyc
logic duplicated, and just add an html output for local devadded
--all
tonyc
for a slightly more honest code coverage number by covering all files (excluding the obvious tests, etc) inlighthouse-core/
,lighthouse-cli/
, andlighthouse-viewer/
. If a file had no tests, it wouldn't show up in the coverage and bring the number down, which seems like a bad policy :)This does bring our coverage just below 90%, but it's not too bad. Majority of the change is in
connections
(extension.js
andraw.js
have no coverage...it's possible we just want to ignore these) andlighthouse-viewer/
(which has basically no unit tests).