After fighting with
simplecov for a little longer that I would like to admit; was attempting to get it to start analyzing a group of files that were the meat and potatoes of my application(Golaith application). Unfortunately none of the default configs (
Simplecov.start 'rails', etc) nor the filters were allowing my files to be tracked and printed in the handy coverage html file. Because of all this struggling I decided to go ahead and create my own crude coverage module; I’ll be using this post to discuss my learnings and share an early working iteration.
To get started I wanted to have the invocation of coverage be exactly the same as
simplecov; so let’s start with the goal of adding
CrudeCov.start inside of our
spec_helper.rb to keep track of the files we care about.
Before diving into the code I did a little research on how
Simplecov.start worked. I was mainly looking for information on how it was able to keep track of files with only a single invokation inside of the
spec_helper. Inside of
lib/simplecov.rb we find a definition of the
start method; which checks to see if the water is friendly (
SimpleCov.usable?) and then starts the tracking with a call to
Coverage.start. At this point during my investigation I was pretty sure that
Coverage was a
Module defined within the
simplecov source; after some grepping within the repo I only found one other reference to
Coverage inside of
lib/simplecov/jruby_fix.rb. Unfortunately that reference is just as the name implies, a
jruby specific fix for the
Coverage module that overrides the
result method. When I was that in the only reference to the module I ran off to google and was incredibly pleased to find that
Coverage is a
Ruby module! According to the Ruby 2.0 Coverage doc
Coverage provides coverage measurement feature for Ruby. This feature is experimental, so these APIs may be changed in future.
With that note about this being an experimental feature let’s be flexible and see what we can do (
simplecov uses it and it’s a pretty successful gem). The usage note in the doc also looks fairly promising:
require or load Ruby source file
::result will return a hash that contains filename as key and coverage array as value. A coverage array gives, for each line, the number of line execution by the interpreter. A nil value means coverage is disabled for this line (lines like else and end).
So we don’t have to worry about #1 (will be loaded by Ruby) and can start with #2 and call
Coverage#start, load all the files that matter, and then use
Returns a hash that contains filename as key and coverage array as value and disables coverage measurement.) to see how well the files have been covered.
As a note Coverage will pickup any file that has been required after
do ::start so it’s a good idea to have a way to selectively find the files that you want to get the coverage results on (e.g. Array of keys
Dir['./app/apis/*rb'] to grab the coverage results you want)
Since we don’t have any intention of supporting
JRuby we should be able to use
Coverage as is for our
CrudeCov example. Let’s start off with the
#print_result(used after our test suite finishes)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
CrudeCov module above is pretty straightforward and covers our basic needs of (1)Having a one-line call to add to our
spec_helper, and (2) a print method that we can call after our suite is finished running (ideally the module would figure out which test framework is being used and ensure that the hook is made to print results at the end of the suite). With the example above we will have to explicityly ensure that the
print_result method is called.
Assuming that we are testing with
spec_helper will look something like this
1 2 3 4 5 6 7 8 9 10 11
With that basic setup you will get a print out of the coverage percentages for all files that have been included in the
filelist. In less than 30 lines of code we were able to have an incredibly simple coverage module that we could use in a project to sanity check a file that may potentially lacking coverage or confirm proper testing. From that simple example you can start to see how a project like
simplecov would come into being and how something as simple as
CrudeCov could become a full ruby coverage suite.
With the legitimate need to get data on the effectiveness of your tests; SaSS solutions like
Coveralls (which did not recognize a Goliath application) + gems like
cover_me have all become relied upon staples for the TDD community.
What’s the point of even doing TDD if you aren’t covering new lines of code that could result in bugs down the road? For that reason alone I’d say it’s worthwhile to implement some sort of coverage tool when all the rest have failed.