-
Notifications
You must be signed in to change notification settings - Fork 40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add code coverage reporting #1866
Conversation
af3a168
to
0a00e14
Compare
https://artsyproduct.atlassian.net/browse/PLATFORM-1136 Problem: One part of the product health matrix story [spike][spike] is code coverage for each service. However, there is no pattern at Artsy for measuring code coverage for Javascript applications. While there are existing tools out there for tracking and reporting code coverage, such as [istanbul.js][istanbul-js], [coveralls][coveralls], and [codecov][codecov], the complexities: 1. A multi-runner test suite (jest and mocha), and 2. A Circle CI -> Docker -> Docker test runtime have resulted in a tough problem to solution around. Solution: After spending a lot of time trying to get Coveralls instrumentation to work, we pivoted to using Codecov. At a high level, this commit does the following: 1. Updates our calls to `jest` and `mocha` to write code coverage information to a `.nyc_output` directory (default for nyc/istanbul) 2. Manipulates the `jest` coverage information to be compatible with `mocha` coverage information via a JS script. See [motivating GH issue][gh-issue]. 3. Merges the `jest` and `mocha` coverage information together using the `nyc` CLI 4. Introduces a bash executable provided by codecov to instrument the merged information to codecov Co-authored-by: Kieran Gillen <[email protected]> [spike]:https://artsyproduct.atlassian.net/browse/PLATFORM-1098 [istanbul-js]:https://github.com/istanbuljs/nyc [coveralls]:https://coveralls.io/ [codecov]:https://codecov.io/gh/artsy/positron/ [gh-issue]:jestjs/jest#2418
* Add BRANCH_NAME to .env.test so that it's whitelisted for Docker * Pass through CIRCLE_BRANCH to `hokusai test` * Extend expression to `codecov` to take a branch name
0a00e14
to
f8a9232
Compare
@dleve123 - can we add a screenshot of example output from the CLI within the top description? |
scripts/normalize-jest-coverage.js
Outdated
const path = require('path') | ||
|
||
const normalizeJestCoverage = (obj) => { | ||
const result = { ...obj }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why this line instead of renaming the obj
arg result
?
scripts/normalize-jest-coverage.js
Outdated
fs.copyFile( | ||
'./coverage/coverage-final.json', | ||
'./.nyc_output/coverage-final.json', | ||
(error) => { if (error) console.log(error) } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks super readable (not suggesting a change) but could also do
error => error && console.log(error)
Also, by the way this is formatted I can tell your editor is missing a Prettier integration :) which we use for all of our JS repos based on configuration located here.
@damassi Is Prettier compliance enforced during CI? I think style checking would be a bit more robust if we asserted upon a certain style during CI, instead of assuming that everyone use an editor that is integrated with Prettier. |
On all of our JS repos we typically run things like prettier during a pre-commit hook which is defined here -- not too sure why this didn't run? |
It's possible that I ran a |
Code Coverage Info in CodeCov
Anyone within the Artsy org should be able to sign into Codecov via Github-auth to get more granular coverage statistics: https://codecov.io/gh/artsy/positron
Code Coverage Into in Console Output
Supersedes #1854 (wanted to rename branch)
https://artsyproduct.atlassian.net/browse/PLATFORM-1136
Problem:
One part of the product health matrix story spike is code
coverage for each service. However, there is no pattern at Artsy for
measuring code coverage for Javascript applications.
While there are existing tools out there for tracking and reporting code
coverage, such as istanbul.js, coveralls,
and codecov, the complexities:
have resulted in a tough problem to solution around.
Solution:
After spending a lot of time trying to get Coveralls instrumentation to
work, we pivoted to using Codecov. At a high level, this commit does the
following:
Updates our calls to
jest
andmocha
to write code coverageinformation to a
.nyc_output
directory (default for nyc/istanbul)Manipulates the
jest
coverage information to be compatible withmocha
coverage information via a JS script. See motivating GHissue.
Merges the
jest
andmocha
coverage information together using thenyc
CLIIntroduces a bash executable provided by codecov to instrument the
merged information to codecov
Co-authored-by: Kieran Gillen [email protected]