-
Notifications
You must be signed in to change notification settings - Fork 552
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Wrong coverage models result with Shoulda-Matchers #638
Comments
Ugh that doesn't sound good :'( Thanks for the report! Thanks for the report. Do you happen to have a small sample application to demonstrate the problem? |
Yes, I have this sample https://github.com/SofiaSousa/pets_api |
I'm having the same problem and thought I was covering everything :/ |
@SofiaSousa I checked with the repo you mentioned and had no issues. Tested with both master and develop branches. Can share your coverage results and mark a few specific lines you think should not be marked as covered? |
A very simple example:
Result: Models (100.0% covered at 1.0 hits/line) |
Your classes are still required by the acceptance tests:
Looking into your models - they are pretty simple (i.e has no method definitions, conditions, etc.), and all lines are actually executed when required. In the following example you can see two green lines:
Code within module/classes is executed just as any other. You can test this by adding a simple class Pet < ApplicationRecord
puts 'Pet is now defined!'
# more code
end Then when you run $ rake
/Users/nikolay/.rvm/rubies/ruby-2.4.0/bin/ruby -I/Users/nikolay/.rvm/gems/ruby-2.4.0/gems/rspec-core-3.7.0/lib:/Users/nikolay/.rvm/gems/ruby-2.4.0/gems/rspec-support-3.7.0/lib /Users/nikolay/.rvm/gems/ruby-2.4.0/gems/rspec-core-3.7.0/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb
Pet is now defined!
Pets
GET /pets
Listing pets
GET /pets/:id
Get a pet
POST /pets
Create a pet Here's what happens if we define a new method within Pet:
Please let me know what do you think of this. Maybe I'm missing something? |
@nbekirov thanks for all the help figuring this one out :) |
@nbekirov, thanks for your reply. I totally agree with what you described, of course it makes sense :) I prepared a branch with what I'm trying to explain: https://github.com/SofiaSousa/pets_api/tree/testing No acceptance tests, no factories, no Shoulda-Matchers config. Only 'empty' model specs. My goal was to check if models attributes validation is fully covered by my tests. Correct me if I'm thinking wrong: how 12.5% of the code could be covered if there aren't any tests? Yep, I don't have any method in the models yet. Does make sense check if validations are totally covered by tests? |
You haven't commented all of your specs code - for models Pet and Specie. Commenting the lines gets me to: Coverage report generated for RSpec to /Users/nikolay/Projects/pets_api/coverage. 0 / 91 LOC (0.0%) covered. This line triggers the framework to load RSpec.describe Pet, type: :model do ...then we proceed as described in my previous comment. You can add this code to the Pet class and see for yourself: class Pet < ApplicationRecord
puts '-' * 100
puts caller.grep(/pets_api/)
puts '-' * 100
# more code here
end and run $ rake
/Users/nikolay/.rvm/rubies/ruby-2.4.0/bin/ruby -I/Users/nikolay/.rvm/gems/ruby-2.4.0/gems/rspec-core-3.7.0/lib:/Users/nikolay/.rvm/gems/ruby-2.4.0/gems/rspec-support-3.7.0/lib /Users/nikolay/.rvm/gems/ruby-2.4.0/gems/rspec-core-3.7.0/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb
----------------------------------------------------------------------------------------------------
/Users/nikolay/Projects/pets_api/app/models/pet.rb:1:in `<top (required)>'
/Users/nikolay/Projects/pets_api/spec/models/pet_spec.rb:3:in `<top (required)>'
----------------------------------------------------------------------------------------------------
No examples found.
Finished in 0.00031 seconds (files took 1.37 seconds to load)
0 examples, 0 failures
Coverage report generated for RSpec to /Users/nikolay/Projects/pets_api/coverage. 14 / 91 LOC (15.38%) covered. ... see how it was Hope that helps! 😃 |
I left those lines so coverage can find some test code and run. I understand the code in the model is executed but can we say that is tested? |
Define "is tested" 😸
NoMethodError:
undefined method `belong_to' for #<Class:0x007fafe0d7a838>
Did you mean? belongs_to
Code coverage says relatively little about the quality of your tests. For me "100% covered" is just a good starting point, it's in no way the end of the journey. |
I was expecting to have to write tests for Pet |
Ruby's coverage module and SimpleCov which uses it only track C0 coverage. Any line that is evaluated when a file is loaded is considered 'covered', whether you not you executed it. #340 so def foo
puts "I am not covered unless you call 'foo'"
end but def foo; puts "I am covered when the file is required whether or not you call 'foo'"; end |
I struggle to find online a document that describe C0 in a satisfying (suitable for different audiences) way that can be linked in a FAQ. Here's the most useful ones I came across: |
But if we write like this: belongs_to :whatever Tests will be passed. And I think it is not good. |
Hi
I was trying to setup a rails api with RSpec, SimpleCov and Should-Matchers and I figured out that after running tests all models appears always 100% covered regardless of written tests, including models without tests.
If I comment the Should-Matchers setup, SimpleCov works correctly for models.
I added SimpleCov call to top of rails_spec.rb:
And Shoulda-Matchers config to support/shoulda_matchers.rb
Anyone with the same problem?
Thanks!
The text was updated successfully, but these errors were encountered: