What is Headless Testing?

Headless Testing means running a browser UI test without the head aka. no browser UI. It is extremely helpful when we don’t really care about the UI, but would like to execute out Automation tests as fast as possible and not involving any “draw” operations onto the screen that consumes it own time and memory. We all know that what we see on the monitor or screen is the result of video card (hardware) rendering those pixels to the screen. Also there is a decent amount of processing that goes on behind before it draws onto the screen. Headless Testing is a concept where we are ignoring those “draw” operations and the headless engines just run the same tests in the background [exactly doing the same operations of click, set etc ] what we do on the browser. Just that the objects reside in memory and we don’t see the video. However we know that as assertions that we baked into the scripts pass, that the application is behaving the way we expect it to. Some of the key benefits of headless testing are:

  1. If executing locally, get rid of those annoying browser popping up:)
  2. Faster execution (hopefully) since we removed any extra browser cache, cookies, resources which we may have unintentionally set on the browser since we use the browser for 100 other things.
  3. No specific monitoring required, just execute and come back and check the results (assuming that your tests passed on a headful browser)
  4. On a CI server, we might NOT always have a display output installed. Generally X-server is NOT installed on *nix systems that act as CI servers in enterprise settings. However if you are using Windows, it may be though. So in both cases, we want to be able to run our tests with lightning speed in a container that emulates display output.

Relevance to Test Automation:

We had HtmlUnitDriver in the good old past days to do headless testing, however over time we realized that though HtmlUnitDriver does headless, it did not have complete javascript support and did not give us the performance we expected. We have couple of options these days viz.

  1. Xvfb (X virtual framebuffer) , however goes only on *nix systems. There are lot of posts on search engines that will give you information on this.
  2. Phantomjs – phantomjs is close to emulating an exact browser and it goes on Windows and *nix systems. It can also take screenshots and render HTML, CSS, SVG and canvas. It uses webkit, a real layout and rendering engine. It also works with multiple frameworks, languages and settings. phatomjs support RemoteWebDriver protocol (via GhostDriver) that means it is very friendly with Selenium and WebDriver.Entire information on phantomjs website.

In this post, we will use phantomjs and execute tests headless.

Agenda:

  • Install and configure phantomjs on Windows
  • Parameterize BROWSER input to Cucumber
  • Execute existing Cucumber features with phantomjs
  • Output
  • Closing Thoughts

Pre-requisites:

  1. You have completed the section “Set up Env” and have your environment ready with JDK, maven and IntelliJ
  2. You have read and understood HTML DOM
  3. You downloaded the project code base for class9 under – https://github.com/machzqcq/CucumberJVMExamples and imported into IntelliJ and the structure should look as below

class9_project_structure

If imported into eclipse, it looks as below

class9_eclipse_project_structure

1) Install and configure phantomjs:

Download phantomjs.zip from phatomjs website

download_phantomjs

Unzip and save it to any folder. I saved it to a folder as shown below

Extract_save

Set the folder path in Windows PATH variable so that it is available for environment. I am using a Windows 7, hence I open My computer – Properties – Advanced – Advanced System Settings – Environment variables – System variables section.

Add the folder to the PATH variable. In my case I appended “;C:\seleniumframework-softwares\phantomjs-1.9.8-windows” to the existing PATH variable

path_setting

Verify that phantomjs.exe is available in environment by typing “phantomjs” in the command line window as below. The “phantomjs” prompt should appear and no errors

cmd_phantomjs

2) Enable Project to use GhostDriver 

As we talked above GhostDriver has the java bindings to communicate with phantomjs. The good thing is ghostdriver abstracts it for us , so that all we have to do is instantiate the PhantomJSDriver that is provided as part of GhostDriver librarary.

Add ghostdriver GAV to pom.xml

In Step 1, we put phantomjs.exe in PATH variable, however if we would like to set it explicitly with an environment variable, I did it with PHANTOMJS_PATH. See in 3) how that is read and phantomjs.exe is accessed.

 

3) Parameterize BROWSER variable:

Since we already wrote cucumber features and scenarios that execute on Chrome, Firefox in previous post on parameterize browser, let’s reuse those scenarios and execute them on phantomjs in a headless mode. In the below example, I am using the same code that was taught

Modify hooks.java file as below. All we are doing is to read the BROWSER environment variable and if that is NOT set , we default to chrome. So now we can execute the scenario by specifying the browser at execution time.

As we can see above, we read the environment variable phantomjs_dir to get the directory and then explicitly set the phantomjs.exe using DesiredCapabilities and then instantiate the driver using PhantomJSDriver()

Execution Output

IntelliJ Output

We use the same methodology that we used in Parameterize Browser page, where we right click – edit configuration and then set the environment variable BROWSER to “phantomjs” and then run the scenario/entire feature

class9_run_scenario_phantomjs

class9_phantomjs_cmd_log

class9_intellij_phantomjs_pass

Command line Output

class9_phantomjs_cmd_execute

class9_phantomjs_cmd_pass

CI Server Output

class9_ci_output

class9_ci_output1

Closing Thoughts

As we can see, phantomjs run totally headless and it emulates a browser very well. It runs on a webkit engine with full JavaScript support and can take screenshots. There itself about 90% of functional testing is covered from browser perspective.