guard-test with simplecov doesn't report coverage

[jmthomas]

I have several specs and several unit tests for a library I've developing. I have both guard-spec and guard-test installed and all my specs and tests run appropriately. However, I only get SimpleCov coverage data for my specs. It says it is generating coverage for Unit Tests but all I get is the method definitions as if it parsed the files but didn't run them.

When I run an individual test or my rake task that runs all the specs and test I get the coverage I expect.

My Guardfile:

guard :bundler do
  watch('Gemfile')
end

guard :test do
  watch(%r{^test/.+_test\.rb$})
  watch(%r{^lib/(.+)\.rb$})       { |m| "test/#{m[1].split('/')[-1]}_test.rb" }
  watch('test/test_helper.rb')    { 'test' }
end

guard :rspec do
  watch(%r{^spec/.+_spec\.rb$})
  watch(%r{^lib/(.+)\.rb$})       { |m| "spec/#{m[1]}_spec.rb" }
  watch(%r{^lib/tools/(.+)\.rb$}) { |m| "spec/tools/#{m[1].split('/')[-1]}_spec.rb" }
  watch('spec/spec_helper.rb')    { 'spec' }
end

My test_helper.rb

# Only require simplecov for Ruby 1.9 or greater
if RUBY_VERSION.split('.')[1].to_i >= 9
  require 'simplecov'
  SimpleCov.start do
    add_filter '/test/'
  end
end
require 'test/unit'

Using ruby v1.9.3p392, guard v1.8.0, guard-test v1.0.0, simplecov v0.7.1

[rymai]

Hi,

It seems that the filter you're using is excluding all files that match "/test/" in their path?

Edit: Sorry, that was silly! :P Running Guard in debug mode bundle exec guard -d will show you the exact command line run by guard-test, so you'll be able to run it outside of Guard and see what's the result then. Hopefully I'll have another look at this tomorrow (it's 11:30pm here now).

[jmthomas]

That's because I don't want to include the actual test files in my coverage report. In any case I can remove that line with no effect.

[jmthomas]

BTW, I'm on Windows. Here is the output from the -d:

16:14:52 - DEBUG - Command execution: bundle exec ruby -Itest -r bundler/setup -r C:/Ruby193p392/lib/ruby/gems/1.9.1/gems/guard-test-1.0.0/lib/guard/test/guard_test_runner -e "%w[test/system_definition_test.rb].each { |p| load p }" "./test/system_definition_test.rb" --use-color --runner=guard

I got it to work by eliminating the line which calls guard_test_runner:

bundle exec ruby -Itest -r bundler/setup -e "%w[test/system_definition_test.rb].each { |p| load p }" "./test/system_definition_test.rb" --use-color

[rymai]

Thanks!

It seems that requiring guard_test_runner causes an early exit (before the tests are actually run), I'll continue investigating tomorrow and hope to find a fix asap!

[rymai]

Hi @jmthomas,

Sorry for the delay! I've pushed a commit that might fix this issue. Could you try using the master branch from GitHub and let me know how it goes?

Thanks!

[rymai]

Hey @jmthomas! Have you had a chance to try my possible fix (gem 'guard-test', github: 'guard/guard-test')?

[rymai]

Can you confirm this is fixed (or not)?

[jmthomas]

I'm unable to reproduce my previous environment. When I try to back port to guard 1.8.0 and guard-test 1.0.0 I get a error in guard.rb:114 'setup_listener' undefined method 'change'.

I updated guard to the latest (2.2.2) and ran your new guard-test and it ran all my tests successfully but then just aborted back to the command prompt. It appears to have generated the coverage results so I guess it worked.

Here is the output from -d:

C:\test>bundle exec guard -d
13:48:31 - DEBUG - Command execution: emacsclient --eval '1' 2> NUL || echo 'N/A'
13:48:31 - INFO - Guard is using TerminalTitle to send notifications.
13:48:31 - DEBUG - Command execution: hash stty
13:48:31 - DEBUG - Guard starts all plugins
13:48:31 - DEBUG - Hook :start_begin executed for Guard::Bundler
13:48:31 - DEBUG - Command execution: bundle check
13:48:32 - INFO - Bundle already up-to-date
13:48:32 - DEBUG - Hook :start_end executed for Guard::Bundler
13:48:32 - DEBUG - Hook :start_begin executed for Guard::RSpec
13:48:32 - INFO - Guard::RSpec is running
13:48:32 - DEBUG - Hook :start_end executed for Guard::RSpec
13:48:32 - DEBUG - Hook :start_begin executed for Guard::Test
13:48:32 - INFO - Guard::Test 2.0.0 is running, with Test::Unit 2.5.5!
13:48:32 - INFO - Running all tests
13:48:32 - DEBUG - Command execution: bundle exec ruby -I"test" -r bundler/setup -r C:/GitHub/guard-test-master/lib/guard/test/guard_test_runner -e "%w[test/my_test.rb].each { |p| load p }" -- --use-color --runner=guard_test

<TEST OUTPUT>

13:48:43 - DEBUG - Hook :start_end executed for Guard::Test
13:48:43 - INFO - Guard is now watching at 'C:/cosmos1'
13:48:43 - DEBUG - Start interactor

C:\test>

[rymai]

Ok, awesome if it seems to work with latest Guard & guard-test!

Regarding the fact that Guard exits and goes back to command prompt, that's weird. Unfortunately,I don't have a clue as to why it happens...

[KjellMorgenstern]

I ran into the same behavior today (incomplete coverage when tests are run with guard). The problem was that guard requires minitest/autorun by default, which causes the coverage report to be finished early.
This could be solved by configuring guard:
guard :minitest, autorun: false do ...
Just mentioning it here because this thread was the first result when I googled the issue.