Code Coverage Tools – Ruby

What is Code Coverage?

As the name suggests, we are measuring the degree to which the source code is tested by a particular test suite[s]. A higher source code coverage generally indicates thorough testing and has a lower chance for the existence of defects. There are intricacies of the above statement -that means, there are plenty of circumstances where the opposite is true. Meaning there might be high (lets say 80%) code coverage, but still more defects compared to a situation where the code coverage is lower (lets say 60%). But on an average, the higher code coverage, the less probability to find defects.

There are many ways in which code coverage metric can be misused too. Here is the article that describes it.

With that background, let’s see how can we measure code coverage with the Ruby code we got.

AGENDA

  • Available libraries/gems
  • Set up and configure for page object framework
  • Run and look at results
  • Code coverage dashboard on CI server

1. Available libraries/gems

We have gems viz. rcov, simplecov and simplecov-rcov

I found simplecov to be easy to set up and work with. That said rcov also has good number of reviews.

Note: Installing rcov on a windows machine was resulting in native extensions error, hence I moved to simplecov

simplecov-rcov is mainly a style formatter for the output of simplecov, that can result in pretty dashboards on CI servers like jenkins.

2. Set up and Configure

I am going to use the page-object_framework code to apply code coverage on. For reference, the code is available here – https://github.com/machzqcq/page-object_framework

Gemfile

Add the gems “simplecov” and “simplecov-rcov” in the Gemfile as below:

simplecov_gemfile

env.rb

First require the entry points into the gems i.e. the modules with the ‘require’ commands. Then enable the formatters. In the below case, I enabled both the HTML and rcov formatters. The HTML , so that we can directly view the html code coverage reports. The rcov, so that CI server can read that and draw pretty dashboards for us [using simplecov-rcov gem]

simplecov_env

That is it !

3. Run and Results

Run

Now run your project as you have been running always. In my case, I run the page-object_framework as below after navigating to page-object_framework root folder

Results

The results are created inside a folder called coverage within the project.

simplecov_coveragefolder

HTML Results

Open the index.html inside “coverage” folder and the results look as below

simplecov_html_results

Rcov Results

Open the rcov folder inside the “coverage” folder and open index.html

simplecov_rcov_results

4. Code Coverage on CI Server

In the CI job, configure the rcov plugin as below.

simplecov_configurejob

Run the CI job and the results should look as below after clicking “Rcov report”

simplecov_rcov_report