Headless Testing

What is Headless Testing?

Headless Testing means running a browser UI test without the head aka. no browser UI. It is extremely helpful when we don’t really care about the UI, but would like to execute out Automation tests as fast as possible and not involving any “draw” operations onto the screen that consumes it own time and memory. We all know that what we see on the monitor or screen is the result of video card (hardware) rendering those pixels to the screen. Also there is a decent amount of processing that goes on behind before it draws onto the screen. Headless Testing is a concept where we are ignoring those “draw” operations and the headless engines just run the same tests in the background [exactly doing the same operations of click, set etc ] what we do on the browser. Just that the objects reside in memory and we don’t see the video. However we know that as assertions that we baked into the scripts pass, that the application is behaving the way we expect it to. Some of the key benefits of headless testing are:

  1. If executing locally, get rid of those annoying browser popping up:)
  2. Faster execution (hopefully) since we removed any extra browser cache, cookies, resources which we may have unintentionally set on the browser since we use the browser for 100 other things.
  3. No specific monitoring required, just execute and come back and check the results (assuming that your tests passed on a headful browser)
  4. On a CI server, we might NOT always have a display output installed. Generally X-server is NOT installed on *nix systems that act as CI servers in enterprise settings. However if you are using Windows, it may be though. So in both cases, we want to be able to run our tests with lightning speed in a container that emulates display output.

Relevance to Test Automation:

We had HtmlUnitDriver in the good old past days to do headless testing, however over time we realized that though HtmlUnitDriver does headless, it did not have complete javascript support and did not give us the performance we expected. We have couple of options these days viz.

  1. Xvfb (X virtual framebuffer) , however goes only on *nix systems. There are lot of posts on search engines that will give you information on this.
  2. Phantomjs – phantomjs is close to emulating an exact browser and it goes on Windows and *nix systems. It can also take screenshots and render HTML, CSS, SVG and canvas. It uses webkit, a real layout and rendering engine. It also works with multiple frameworks, languages and settings. phatomjs support RemoteWebDriver protocol (via GhostDriver) that means it is very friendly with Selenium and WebDriver.Entire information on phantomjs website.

In this post, we will use phantomjs and execute tests headless.

Agenda:

  • Install and configure phantomjs on Windows
  • Parameterize BROWSER input to Cucumber
  • Execute existing Cucumber features with phantomjs
  • Output
  • Closing Thoughts and Video at the bottom.

1) Install and configure phantomjs:

Download phantomjs.zip from phatomjs website

download_phantomjs

 

Unzip and save it to any folder. I saved it to a folder as shown below

Extract_save

 

Set the folder path in Windows PATH variable so that it is available for environment. I am using a Windows 7, hence I open My computer – Properties – Advanced – Advanced System Settings – Environment variables – System variables section.

Add the folder to the PATH variable. In my case I appended “;C:\seleniumframework-softwares\phantomjs-1.9.8-windows” to the existing PATH variable.

path_setting

 

Verify that phantomjs.exe is available in environment by typing “phantomjs” in the command line window as below. The “phantomjs” prompt should appear and no errors.

cmd_phantomjs

2) Parameterize BROWSER variable:

Since we already wrote cucumber features and scenarios that execute on Chrome, Firefox and so on, let’s reuse those scenarios and execute them on phantomjs in a headless mode. In the below example, I am using the same code that was taught in Basic tutorial – WebElementOperations

Modify hooks.rb file as below. All we are doing is to read the BROWSER environment variable and if that is NOT set , we default to chrome. So now we can execute the scenario by specifying the browser at execution time.

Modify hooks.rb file as below. All we are doing is to read the BROWSER environment variable and if that is NOT set , we default to chrome. So now we can execute the scenario by specifying the browser at execution time.

hooks

3)Execute Cucumber feature as below:

cucumber BROWSER=phantomjs features\WebElementOperations.feature

Execution_cmd

4)Output:

The entire feature should execute. No browser should pop up and all scenarios should pass with GREEN status.

5)Closing Thoughts:

As we have seen, it is not a complex piece of code that we have to write to make our tests run headless. It is a simple matter of installing phantomjs and configuring and parameterizing the browser value inside hooks.rb. The video is posted below too.

Headless Testing