-
Notifications
You must be signed in to change notification settings - Fork 247
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HTML reporter now no longer works with browserify-istanbul in v4.0 #157
Comments
Thanks for the detailed report, but I think something went wrong when you inserted the diff for the change you made, or I just don't understand what it's trying to say :( |
Sorry, a vanilla diff may have not been the best choice https://github.com/gotwarlost/istanbul/blob/v0.3.15/lib/report/html.js#L200 and https://github.com/gotwarlost/istanbul/blob/v0.3.15/lib/report/html.js#L231 to cause it to no longer throw
where 'undefined' in this error is structuredText[startLine] This was just a cheat to prevent istanbul from throwing an error when given an invalid start time and help isolate the issue. |
I see, so the fix is actually inside the istanbul code. To be honest I have no idea how or why this is happening, maybe @gotwarlost or @davglass have an idea how this can happen and if this an issue from Istanbul or from our side |
We pushed out |
From the diff above it looks like he is using |
TWO different projects' collaborators responding in 2 hours? Well now I feel obligated to better reproduce the issue. Yes, 0.3.15 (in both browserify-istanbul and karma-coverage).
I reproduced the issue here: So it appears that the issue stems from me using the 'hbsfy' browserify transform leaving me with absolutely no idea who's issue this really is haha. So the pieces involved so far are:
... |
Switched Istanbul to 0.3.14: https://github.com/notnarb/karma-coverage-issue-157/tree/istanbul-0.3.14
which matches what is reported by this comment #123 (comment) |
have same error with
As temp solution, I wrapped all places in istanbul/lib/report/html.js with errors
wrapped:
At least I have some html report |
I was able to work-around this issue by setting karma's |
Just tried that
BUT
no html reports get made at all (I assume it just fails silently?) |
Thanks @blazzy ! Your workaround helped. |
My project uses browserify with babelify to convert es6 to es5, karma-coverage 0.2.7 and babel-istanbul 0.2.10 for coverage reporting.
I used the @akoval 's fix, but then I get incorrect coverage report - it sais that there is uncovered code at a nonexisting line. Investigated a bit and found out that babel in order to convert the code introduces an extra function:
is converted to
I guess karma-coverage wants to mark The workaround, for my progect, was to change the name of the method function
so that babel won't generate it's weird wrapper function and add |
I think the problem is with the environment 'NODE_ENV' when i set it to 'test' it give me the text error. When i change it to 'testing' it works fine. Hope this helps in fixing the issue |
@blazzy -- +1 -- worked for me. here is my karma.conf + package.json // Karma configuration
// Generated on Thu Jul 23 2015 23:21:04 GMT-0400 (EDT)
module.exports = function (config) {
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: ['browserify', 'tap'],
// list of files / patterns to load in the browser test runner
files: [
{pattern: './test/**/*.js', included: true},
{pattern: './lib/**/*.js', included: false}
],
// list of files to exclude
exclude: [],
// preprocess matching files before serving them to the browser
// available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
// create the ES5 equivalents of these ES6 modules (these are loaded asynchronously)
'./lib/**/*.js': ['browserify', 'coverage'],
// create the ES5 versions of the tests written in ES6 (these are included in the test runner page)
'./test/**/*.js': ['browserify', 'coverage']
},
browserify: {
debug: true,
transform: ['babelify', 'browserify-istanbul']
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: ['tape', 'coverage'],
coverageReporter: {
dir: './reports/coverage',
reporters: [
// reporters not supporting the `file` property
{type: 'html', subdir: 'html'},
{type: 'lcov', subdir: '.'},
{type: 'text', subdir: '.', file: 'text.txt'},
{type: 'text-summary', subdir: '.', file: 'text-summary.txt'}
]
},
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: false,
customLaunchers: {
'ChromeES6': {
base: 'Chrome',
flags: ['--enable-javascript-harmony']
},
'ChromeCanaryES6': {
base: 'ChromeCanary',
flags: ['--enable-javascript-harmony']
}
},
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
//browsers: ['ChromeES6', 'Chrome', 'ChromeCanaryES6', 'ChromeCanary', 'Firefox', 'Safari', 'PhantomJS'],
browsers: ['Chrome'],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: true
});
}; {
"name": "",
"version": "0.0.0",
"description": "7",
"main": "lib/cli/main.js",
"dependencies": {
"babel": "^5.5.5",
"core-js": "^0.9.8",
"rsvp": "^3.0.18",
"templeton": "^2.1.1",
"xhttp": "^0.0.6",
"cookie-parser": "^1.3.5",
"body-parser": "^1.12.4",
"express": "^4.12.4",
"pem": "^1.7.2",
"request-promise": "^0.4.2"
},
"devDependencies": {
"babel-eslint": "^3.1.26",
"babel-istanbul": "^0.2.10",
"babelify": "^6.1.0",
"browserify": "^10.1.3",
"browserify-istanbul": "^0.2.1",
"eslint": "^0.21.0",
"isparta": "^3.0.3",
"karma": "^0.13.3",
"karma-browserify": "^4.2.1",
"karma-chrome-launcher": "^0.2.0",
"karma-coverage": "douglasduteil/karma-coverage#next",
"karma-firefox-launcher": "^0.1.6",
"karma-phantomjs-launcher": "^0.2.0",
"karma-safari-launcher": "^0.1.1",
"karma-sourcemap-loader": "^0.3.5",
"karma-tap": "^1.0.3",
"karma-tape-reporter": "^1.0.3",
"minifyify": "^7.0.3",
"phantomjs": "^1.9.17",
"plato": "^1.5.0",
"tape": "^4.0.1"
},
"scripts": {
"lint": "eslint -c ./.eslintrc ./lib/**/*.js",
"test": "karma start",
"start": "babel-node ./lib/server/main.js",
"metrics:compile_source": "./node_modules/.bin/babel --optional runtime lib/ --out-dir ./build/metrics/lib",
"metrics": "npm run metrics:compile_source && ./node_modules/.bin/plato -r -l .eslintrc -d ./reports/metrics ./build/metrics/lib"
},
} |
Realized that the LOG_DISABLE hack, just hides the error. The full report is never generated. What you wind up with is a report that looks complete, but if you drill down to each of the files, you will hit 404's on any of them that were not parsed due to the error. minal2709's NODE_ENV patch did not work either.
Any ideas on what sort of a fix this needs -- is it actually an issue with karma-coverage, or is it with istanbul/isparta? |
Are there any news on this? My work-around is to switch to the text-reporter, but i'd love to have the html-reporter working again. |
@blazzy's workaround simply hides the error message. The error still occurs and execution still short-circuits, resulting in an incomplete, erroneous coverage report. The issue appears to be a result of the way Istanbul handles (or doesn't) source maps. There is a fix implemented in the Istanbul repo, but it hasn't been merged into the master branch yet. I explicitly installed Istanbul directly from that GitHub branch (source-map) and it worked like a charm. For now, you can install the fixed Istanbul branch via |
@SpenceDiNicolantonio is correct. The reason the HTML reporter is failing is most probably due to the fact that it is being given the transformed version of the source and the original source file name and it can't find the lines there. The source map branch has support for handling this correctly via source maps. The problem is that the source map implementation is a proof of concept which muddies the istanbul APIs and makes them unmaintainable. And I have deadlines in my day job that I need to handle. Will get to it as soon as I can. |
Thanks @SpenceDiNicolantonio and @gotwarlost for your answers and the fix! |
+1 |
Thanks for the solution branch for this @SpenceDiNicolantonio. That helps a lot. 👍 |
@SpenceDiNicolantonio -- i am not able to get it to work, even with the modified |
Sure, we're doing this with the fix branch installed: const istanbul = require( 'browserify-istanbul' ),
isparta = require( 'isparta' );
module.exports = function( config ) {
config.set( {
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: [
'browserify',
'mocha',
'chai',
'sinon',
'phantomjs-shim'
],
// list of files / patterns to load in the browser
files: [
'spec/**/*.js'
],
// list of files to exclude
exclude: [
],
// preprocess matching files before serving them to the browser
// available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
'app/**/*.js': [ 'browserify' ],
'spec/**/*.js': [ 'browserify' ]
},
// https://github.com/Nikku/karma-browserify
browserify: {
debug: true,
bundleDelay: 1000,
extensions: [ '.js', '.jsx' ],
transform: [
'babelify',
istanbul( {
ignore: [ '**/spec/**', '**/fixture/**' ]
} )
]
},
client: {
mocha: {
args: '--debug-brk'
}
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: [
'coverage',
'junit',
'mocha'
],
// reporter options - full, autowatch, minimal
mochaReporter: {
output: 'full'
},
// unit tests JUNIT reporter for Bamboo
junitReporter: {
outputFile: 'test-results.xml',
suite: ''
},
// coverage reporter
coverageReporter: {
instrumenters: { 'istanbul-react': require( 'istanbul-react' ) },
instrumenter: {
'app/**/*.{js,jsx}': [ 'istanbul-react', isparta ]
},
reporters: [
{ type: 'text' },
{ type: 'html', dir: 'coverage', subdir: 'reports' }
]
},
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: false,
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
browsers: [ 'PhantomJS' ],
// browsers: [ 'Chrome' ],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: true
} );
}; |
Also, there is an issue in the instanbul repo too: gotwarlost/istanbul#429 |
@fernandofleury |
@simongfxu: |
+1 |
for me helps using this one |
FYI, I made a new coverage instrumentation plugin for Babel 6: babel-plugin-coverage. Hope it helps 😄 |
Alternatives mentioned, like browserify-babel-istanbul still cause problems for me, but those are separate discussions, probably most appropriate for their repos instead of here. If anyone is interested in diving in for a fix in istanbul, @gotwarlost, is there a good candidate PR you'd like folks to work on? I see two or three related to the |
Encountered the same issue using mochify-istanbul with babelify, no luck so far. I tried the temporary workaround and it resulted with getting no coverage at all (so it would exit without error and generate 0/0, 100% coverage for tests that obviously don't cover 100%...). |
Solution (hopefully)Went through like 9,000 different comments in a bunch of issues, and found this example repo: https://github.com/kt3k/karma-browserify-isparta-example This works perfectly for me: browserify: {
debug: true,
transform: [
'babelify',
['browserify-istanbul', { instrumenter: require('isparta') }]
])
},
coverageReporter: {
reporters: [
{ type: 'html' },
{ type: 'text' }
]
}, |
fwiw, someone else had success with a similar setup as mine. |
Thanks, @thejameskyle. I've also been able to use |
Thank @thejameskyle this worked for me too. |
Thanks, @thejameskyle! |
Solution (..maybe)I can't even 😬For me, switching the position of the transforms got it working
karma.config.jsconst isparta = require('isparta');
const istanbul = require('browserify-istanbul');
...
transform: [
istanbul({
instrumenter: isparta, // <--module capable of reading babelified code I think
instrumenterConfig: { embedSource: true }, // what got it working
ignore: ['**/node_modules/**']
}),
['babelify']
], .babelrcNot sure if the babel config matters but {
"presets": ["es2015", "react"],
"sourceMap": "inline"
} |
Any news on this? For months now I am struggling to get coverage working with my setup:
I have all dependency versions up to date and still get this error. I tried all possible solutions in this issue, but nothing helped. It is very hard for me to believe that there is no way to get coverage working with this tools. The config of @boneskull was not far from working, but I had the browserified code in the HTML reports, not the ES6 code. And the coverage was also calculated based on the browserified and babelified code. So there was code marked as not covered, that was written for Browserify or Babel, not my own code. The approach with executing babely transform after browserify never worked, because browserify immediately complained about ES6 syntax (import). I am almost desperate. Maybe there are any recommendations about changing my toolchain? |
@dsuckau, see @thejameskyle and @aerze comments. I happened to do something similar and it worked fine for me. Here's my var istanbul = require('browserify-istanbul');
module.exports = function (config) {
config.set({
browsers: ['PhantomJS'],
frameworks: ['browserify', 'chai-as-promised', 'chai', 'mocha'],
reporters: ['spec', 'coverage'],
files: [
'node_modules/babel-polyfill/dist/polyfill.js',
'tests/spec/**/*.spec.js'
],
preprocessors: {
'tests/spec/**/*.spec.js': ['browserify']
},
coverageReporter: {
dir: './tests/coverage',
reporters: [
{ type: "html", subdir: "html" },
{ type: 'text-summary' }
]
},
browserify: {
debug: true,
// istanbul transform MUST go first
transform: [
istanbul({ instrumenter: require('isparta') }),
'babelify'
]
},
// if you want to continuously re-run tests on file-save,
// replace the following line with `autoWatch: true`,
// but watch out for `ENOSPC` error.
singleRun: true,
logLevel: config.LOG_INFO
})
}; what really helped to get past the error was this: { instrumenter: require('isparta') } |
For those working with webpack, you can find this useful: webpack-contrib/istanbul-instrumenter-loader#32 Basically work with older version of |
@aerze Your solution in my case prevented tests from being transpiled. The reason I thought it works is because my node version supports most ES6 features, but try to use ES6 import syntax or something from stage-* that's not in Node yet, and it'll break the bundle. The project in question uses browserify. |
FYI, I made it work and the solution was to use var isparta = require('isparta');
var browserifyInstanbul = require('browserify-istanbul')
var browserifyInstanbulConfig = {
instrumenter: isparta,
ignore: [
'**/node_modules/**',
'**/*.html',
'**/*.spec.js',
],
};
...
browserify: {
debug: true,
transform: [
[browserifyInstanbul(browserifyInstanbulConfig)],
['babelify'],
],
}, The order is important. You want to first instrument, then babelify. |
Running the code through Babelify before instrumenting it for code coverage resulted in incorrect statement location metadata. See the "statementMap" sections of the two files in https://gist.github.com/robertknight/4ec0f2c3509af61c31ffd574fe16cb74 Since Isparta already supports ES6 internally, it can instrument the original untranspiled source, the problem can be resolved by switching the order of the transforms. See karma-runner/karma-coverage#157 (comment)
Thanks for this tip. My understanding is that this works because "isparta" understands ES6 internally anyway, so it can instrument the untranspiled code? This seems preferable in any case because it avoids confusion due to what is a statement/function in the original code vs. the code that actually executes. |
We had this error and I spent yesterday morning fixing. Here's a write-up of what worked for us. Note that this solution does not involve browserify |
HTML reporter now no longer works with browserify-istanbul in v4.0
I was having issues with the html reporter not outputting anything in v3.1 where the index.html would be written, but no reports would be contained inside (while the text reporter was working fine) so after seeing issue #123 I jumped to 4.0. Now I am getting the following issue:
I made a temporary fix which now has the html reporter working properly (as far as I can tell, but certainly better than 0.3.1) - but I'm sure this isn't the proper solution
Sorry that I can't be of too much more help than this -- I don't really understand the inner-workings of instanbul and karma-coverage, all I can determine is that for some reason a faulty startline is being communicated occasionally causing istanbul to crash and burn.
karma.conf.js
package.json
One thing worth reiterating: I couldn't get html reports to work at all in 3.1 and now they work great with my hacky patch
The text was updated successfully, but these errors were encountered: