Archive Test Results

Context:

I am not a big fan of archiving test results, because eventually they will rot and nobody really has the time to go back and look at the results archived. There is not much value in it. Also with Continuous Test Automation, every execution might result in an archive, which might be an overkill. Also the pace at which technology is changing does not really justify the old archaic ways of archiving test results, documenting every little bit of information that you indeed validated a certain functionality. With Agile, trust on team members is of utmost importance and if it was not tested, it will anyways surface up and it is not hard to find who might be a slacker if I may say.

With that context, there might still be some cases where we might want to archive certain files, results, reports. So we are not overruling the concept altogether.

Use Cases for Archiving:

Some of the use cases for archiving are as follows:

  • Audit compliance to save screenshots or results 
  • Regulatory compliance as per acts like SOCKS
  • Financial and Healthcare firms have to record lot of artifacts and configuration items and yes test results [within certain scope] are part of thos configuration items.
  • Maybe I am testing on production and I don’t have the luxury to tear down environments as I have in non-prod, and I have limited time to execute, however want to analyse the results in detail

So as we see, there might be use cases where we might have to archive test execution results. Anyways, I will leave that decision to your discretion. This page shows how we can archive results using Cucumber/Ruby framework we have been discussing so far. 

Agenda:

  • Rubyzip gem
  • at_exit global hook
  • ENV[‘ARCHIVE_RESULTS’] environment variable 
  • Understand the intricacies of code

1) Rubyzip Gem:

Rubyzip is a module for reading and writing zip files. Find full details here.

We would not be going into extreme details of what the gem does because our focus is to zip the rest results and archive them for later perusal.

Add the gem to your project’s Gemfile as below:

rubyzip_gemfile

Now bundle update from RubyMine:

bundler_update

Now require the zip module in your env.rb

rubyzip_env

2) at_exit:

We will discuss the concept of Cucumber hooks in a separate page, however at this point all we need to understand is that there are scenario hooks and global hooks that Cucumber provides. Hooks are a way to execute certain block of code before/after an event [or we can also define custom hooks when an event occurs]

at_exit hook is a Kernel hook that executes as part of global teardown. For our purposes on this page, this helps us execute a certain piece of code (archiving results) after Cucumber scenarios execution and after the results file is generated.

Let’s place this code in hooks.rb file as follows:

Explanation:

We read the environment variable and set to ‘no’ if its value is nil

If environment variable is set to ‘yes’, then we enter the block [We set the value in cucumber.yml file , scroll down and see how it is set]

Create a directory “resultsarchive” if it is empty or doesn’t exist

We read the file results.html from current directory and then use the Zip api to create a zip file. The zip file name will have today’s date appended with “_results”. Here you can get creative if you want to name the zip file with time stamp or project name or maybe you want to name it something else. So you can assign any value to the zipfile_name. 

Then we loop through the files (in our case, there is only one file, but the code is generic enough to handle any # of files) and keep building the zip archive

3)cucumber.yml

Add ARCHIVE_RESULTS=yes to the default profile as below. I am sure you must have figured by this time that this is a configuration parameter i.e. if it is set to ‘yes’ , only then results are archived. So you can turn off if you don’t want to archive results.

Add ARCHIVE_RESULTS=yes to the default profile as below:

zip_cucumberyml

Output:

Now if you run any cucumber scenario in your project, there will be resultsarchive folder created and your execution results will be zipped and placed inside that folder. A screenshot after I executed a sample scenario looks like this:

zip_results