Minitest wrong coverage value

[plribeiro3000]

Im currently facing a problem i dont even know where to look. Im working on a project fog/fog-xenserver and i can't get simplecov to work on it.

It is resulting in 74.08% coverage when i have only 1 test and the lib is big, really big. I have tested both latest version (0.8.2) and the one stated on README (0.7.1).

Even if i remove require 'minitest/autorun' and no tests run, my coverage still 74.08%.

Ruby version : ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin13.0]
Minitest version : 4.7.5

In case it was something i did not properly configured, i have create a branch simplecov so you guys could check it.

I use rake to run my tests as you can see here

[bf4]

So, it's important understand how SimpleCov, and the stdlib Coverage lib it depends on works.

It tracks when code is evaluated for the first time (i.e. when loaded or run )

Thus, if you run SimpleCov.start, load a file, then run SimpleCov.result, you will see a certain amount of 'code coverage'. That is, running tests is a way to increase the coverage of a file above the coverage you get for just loading it.

In your test helper you have Dir[File.join(File.dirname(__FILE__), '../lib/**/*.rb')].sort.each { |f| require f } which loads all your ruby folders under lib. Thus, you see files 'covered' that weren't tested.

[plribeiro3000]

Hey @bf4 , Thanks for taking the time to look at the project. 👍

[bf4]

I've been referencing this question quite a bit. If anyone wants to help put together a 'standard response' doc, that would be great. See e.g. the one nokogiri uses https://github.com/sparklemotion/nokogiri/blob/master/STANDARD_RESPONSES.md

[geekq]

For me the key change was to require and start simplecov BEFORE minitest/autorun loads ./lib

# <BEGINNING OF test/test_helper.rb>
# Require and start simplecov BEFORE minitest/autorun loads ./lib to get correct
# test results. Otherwise lot of executed lines are not detected.
require 'simplecov'
SimpleCov.start do
  add_filter 'test'
end

require 'minitest/autorun'

class << Minitest::Test
  def test(name, &block)
...

I am maintaining a ruby library (as opposite to a Rails application)
and using plain Minitest and a minimal Rakefile