Pitfalls and Caveats

Now that we have covered 3 types of frameworks, each one incrementally building on the top of the other, let’s take a pause here and discuss what are some pitfalls and caveats.

Caveat 1:


I blogged about this a while ago especially around over abstraction and building layers and layers of abstraction. Here is the full blog entry. The summary is as follows:

In the Test Automation space with browsers , Selenium definitely is the leader at this point. The API’s exposed by selenium are in multiple languages and in ruby, the “real gem” is selenium-webdriver.

That said, I definitely appreciate watir-webdriver gem because it has abstracted some complexities in the syntax style. But end of the day, the watir-webdriver api’s call selenium-webdriver api’s for interacting with browser. Visually we can see something like this.



Too less abstraction: There are forces acting in opposite direction i.e. with too less layers, we have to write too much amount of code and have to sweat a lot to implement a simple functionality on application layer.

Too much abstraction: This is the one I want to focus on here with this caveat because I see too many gems on top of watir-webdriver and selenium-webdriver being implemented in the Ruby world. One of the major issues is the time it takes to debug, when things fail. Especially if the layer is NOT well built with exception handling and does NOT propagate the error up for easier root cause detection.

That is why on this website, we strove and built Page Object pattern using plain Ruby concepts

We DO NOT want to use more and more gems [thereby building more layers] and having dependencies that “will” become PITA over time.

It has also become more of a fashion statement for some folks to say “I can write a gem“.

Guys, if you hear this, raise your ears and evaluate properly before falling for this trick.

One of the strengths of Ruby world is “being humble”. Matz is humble, so we are ! I am sure you know that statement. Anyways, lets move on.

I highlighted the pitfall with a graphic below.



Caveat 2:

Page Object framework has its limitations.

There is often a misconception on the Test Automation world that Page Object framework is magic and can solve all problems and that is the answer for all browser automation issues. I would like to highlight the following to that notion.

  1. Page Objects represent the html source relatively well, however they have their own limitations on representing the page navigation and DOM current state
  2. With so many java script libraries that implement ajax calls these days, implementing call backs (aka. making our tests synchronous to handle the partial DOM state changes) has different strategies based on the strategy that was followed “by the application using that particular js library“. For example, on this website, we talked about Handling ajax with jQuery, angular,YUI and prototype.Recently I had an opportunity to work with parse api’s with backbone.js framework. Haven’t found a global listener tracker on parse api’s that I can wait before moving further in my tests [same strategy we followed with jQuery, angular etc.]
  3. The issues are generally around flaky tests and getting “Stale Element Exceptions“. One best practice to fix this is to model our page-objects very closely to align with the behavior and state transitions of the DOM and that is only as good until new js scripts get introduced into the web application js resource stack.
  4. On the Ruby side, if you get too many Stale Element exceptions, as a temporary workaround, put a begin..rescue block around the code that causes the exception and in the rescue block, re-poll the DOM to find that element. At this point, I found only this as a viable solution.

So summary is to say: If you are spending too much time on building the test Automation framework itself and fixing issues there, then when would you have time to focus on uncovering application defects?