Cucumber Jenkins Plugins

Now that you have implemented Continuous Test Automation pattern, let’s build and enhance on top of it. We have seen in our previous post on Implement Continuous Test Automation Pattern , how to execute our Test automation suite on a Jenkins server. At the end of execution, we have looked into the console output and saw the BUILD SUCCESS.

However we can see that the text output is not pleasing to the eyes, meaning no colors ya! Right ?

In this section, we will see how to make use of some Jenkins plugins that are out there working hand in hand with cucumber formatting outputs.

Cucumber Test Results plugin

This plugin will take cucumber json output as input and publishes the results in polychrome, green when it has to be green and RED when it has to be so.

Step 1

Login to your Jenkins server as administrator and navigate to “Manage Jenkins”. You should see a page like this. Go ahead and click “Manage plugins

jekins_plugins

Step 2

On the manage plugins page, select “Available” tab and then type “Cucumber” in the filter at the right top corner. As of today, I found the below matches. Go ahead and check the box against Cucumber Reports plugin.¬†Then click “Install without restart

install_cucumber_json_plugin

The progress should look something like this below

installing_progress

Step 3

Click “Back to dashboard” in the left menu and navigate to the job we created in our previous section i.e. seleniumframework.com acceptance tests. Click “configure” against the job. Once you are in edit mode, go ahead and add “post-build step” and select “Publish cucumber test result report

configure_post_build

Step 4

Now add the below in the text box. Ignore the message at the bottom [The message appears since we don’t have any files yet, once we run the build they will get generated]

cucumber_json_reports

Step 5

Now lets generate the cucumber json reports. Open the cucumber.yml file in your page-object_framework and add the below line. Now in addition to the html reports, we will have reports.json output as well. This will be parsed by the cucumber plugin on Jenkins.

cucumber_yml_json

Execute

Now lets go ahead and “Build Now” from the left menu for the job. The build should be successful. At the end of the build, now instead of watching console output (as we did in the previous post), now select “Cucumber Test Result

results_output

Results

Go ahead and drill down into the Feature -> Scenario until you reach the below page, where you can see all steps green

final_results

 

Cucumber Reports (Graphical reports)

If you output the results to json format i.e. in cucumber.yml if we had options “-f json -out cucumber.json”, then this plugin can read the json output and produce pie charts for feature/scenario results and bar graphs for tags. This plugin has many advanced options to help us make decisions on the test results.

Step 1:

Add a post-build step and select the option as shown below

cucumber_reports

Step 2:

Now back to the job, we should see the menu option as below

cucumber_reports_piecharts

Step 3:

Now run the job and after clicking the reports, there will be swf charts similar to the below

cucumber_reports_piecharts_failed

Step 3:

We can drill down into the steps and the corresponding failures at the bottom of the report

cucumber_reports_piecharts_features

Cucumber perf plugin

This plugin shows you a history of how cucumber results were performing as a trend, so this will be helpful for you after you keep running the suite multiple times. There are more features to it, like “worst performing features” over time. This will help us focus on those particular features and debug further.

The plugin installation is pretty much the same process as we described above for cucumber test results plugin. Some screenshots below for you to make a decision whether to install or not. They are directly from the cucumber wiki website.

project_performance

worst_perform_feature