Report generation really really slow

[pschambacher]

I'm working on a Windows/JRuby project and the time to generate the HTML report is growing exponentially compared to the size of the project.

At this point I'm talking about 79 files in total, 1147 relevant lines and 333 tests (RSpec) and once the tests are done (about 1 minute) the report needs around 10 minutes to be written.

Is there something in the way we organize the files that would explain this ? I don't really think so because I've been working on a way bigger application (Mac/MRI) and it only needed a few dozen seconds. What's you opinion ?

[colszowka]

Not sure out of the blue. Please note that JRuby support is not official yet, although essentially it should work and I've had reports of such. As a linuxer I can't comment on Windows-related issues though. Maybe you could open up the (simplecov) source code (especially of the html formatter) and sprinkle in some puts statements to see which methods are taking so long and if there's some weird recursion going on.

[pschambacher]

OK I wanted to know if it was a known case and also have an end point if some others have the same issue.

Just to pick a blind guess I would say there is a call somewhere which, in JRuby, is translated into some nasty string concatenation. The way the time keeps increasing would match this kind of problem.

On Monday I'll try to put some logs here and there to identify the problematic method, until then I don't have access to the required configuration. I'll keep you informed.

[pschambacher]

OK so I've been tracing which part of the code was so damn slow and identified it in theses lines in simplecov-html

    File.open(File.join(output_path, "index.html"), "w+") do |file|
      file.puts template('layout').result(binding)
    end

I've been testing separately the call to template('layout').result(binding) and it's quite fast so it's the writing on the disk that is the problem. I'm going to see if another way of writing the file on disk solves the problem.

[pschambacher]

OK so the solution came from this post on StackOverflow: http://stackoverflow.com/questions/8064175/writing-unix-line-breaks-on-windows-with-jruby
Since I'm on windows some code tries to replace every \n with \r\n in some kind of bad way. Setting the mode to 'wb+' makes it bloody fast. The question is: how would this affect anything else ? Is it safe to just replace w+ with wb+ for everybody or should it only target JRuby+Windows ?
Based on your answer I can produce a pull-request if you want.

[colszowka]

As far as I remember the binary flag has quite a different meaning in Ruby on Windows vs. on Unix. I think it'd be better to have it open it in "wb+" only on Windows, possibly even only on JRuby. Do you have some way to test MRI behaviour with wb+ on Windows?

Pull request would be great :)

[pschambacher]

Here is the pull-request: simplecov-ruby/simplecov-html#16
I targeted JRuby/Windows. I don't really have a simple way to verify with MRI right now since my project is deeply linked to JRuby but I'll try to have something later to test it.

[pschambacher]

FYI I used this post to detect Windows: http://stackoverflow.com/questions/4871309/what-is-the-correct-way-to-detect-if-ruby-is-running-on-windows

[xaviershay]

recommend closing in favour of the linked PR.