Decision Models

Please skip to Choosing Automation Solution directly if you are jumper like me, however I would appreciate if you give me your eyes for this below content please.

I was always fascinated by the convergence of theories and practical applications and I am a believer that we go back and forth between dispelling theories and learn through “hit or miss” AND learn from experiences and get back to fundamentals (theories). This write-up is to get back to fundamentals after I gained some real-time “hit or miss” experiences for couple of years now.

Culture Change

Culture change is a big deal right. We all know that being a change agent is more than what meets the eye. After being in a couple of firms now, especially in the front line, articulating the strategy/vision and at the same time implementing it is a daunting task. More because, there are no superman’s except in movies. Most firms would hope for a culture change. Some of the images below should convey what I really believe in.

We will definitely need an early set of Innovators and early adopters. Typically this happens as choosing an area/ project/ set of folks inside a company who are naturally enthusiastic to be part of fast changes.

TheChasm

Putting an framework and organization into thought helps convey the message and ideas. Below infographic depicts the same in a very elegant manner.

 

How_cultures_change

 

 

empower_people

Often when we are making decisions, we have too many moving parts and competing priorities and constraints. For example, I might like some cool technology out there which does wonders, but questions that need to be answered before jumping on to the bandwagon are:

  1. Can I afford it
  2. Does my team have skill sets to adopt and maintain it
  3. How do I find staff that can get me jump started
  4. What is my budget for this year and what is my forecast
  5. What if it fails in my environment and what are my risk mitigation plans

I think you get the point here [A more exhaustive set of questions at the bottom of the page]

This is nothing but a classic Decision sciences problem that applies to every industry, every leader, every management and every situation in life. The same applies to be able to choose an Automation Solution too – it is no different. 

Giving it a little bit organized thinking for this problem (putting my MBA hat on), I view this as belonging to one of the following categories (if someone has more thoughts on this, please write to me, I will be glad to have this modified):

Linear:

The assumption we make here is that the output or effect caused by each input is linear (unfortunately that’s not the reality right;)

Decision Equation: An equation having certain variables that need to be assigned values and we want to figure out the optimal (max. or min.) variable inputs. Think of variables as money, staff, time etc. And better what, they are dependent variables, meaning if one changes, the other one might change too.

Constraints: The variables are governed by constraints meaning they cannot exceed certain values or cannot be below certain values etc. Obviously we cannot hire infinite staff, have infinite money or time etc. And combine this with the complexity of dependent variables.

Optimization: Based on our target or expected goal (maximize or minimize or some target #), we will need to figure out the inputs

 

Range:

The problem is pretty much the same described in Linear above, however the difference is that our constraints for the variables are within ranges [x,y] and the variables are again dependent on each other and we might not have simulated environments to figure out “what-if” scenarios

Probabilities:

The problem is pretty much the same described in Linear above, however the difference is that our constraints for the variables have a probability distribution sometimes we might know the distribution, most of the times we might not know.

 

Where are we going with this ?

As we can see there will be # of dimensions (variables) in real world that come into play and we are perplexed by the decision making process. The answer to most of such tough questions as we know is “It depends!”. Depends on what? Depends on how sure one knows what expected outcome is supposed to be. It is true. The more we know what we want, the closer we are getting to identify a solution to our problem. If we want to find a perfect predictive solution, I guess “Good luck” to you. 

We are not trying to find a silver bullet. We are trying to improve one step further over our current state. For me this mindset really helps make decisions go much smoother of course with a certain set of guidelines I would recommend:

  • Smallest bit of improvement we can do NOW (vs. boiling the ocean)
  • Ability to measure the progress
  • Continuous Feedback 
  • Build a thin slice and then expand
  • Let it go sometimes

Moving on, I will try to bring some organization to the thoughts and information on internet around being able to choose a Test Automation Solution

Frequently asked Questions

The below three sections of questions (faq) is experiential advice after working with small, medium and large scale companies. As we know the needs and requirements differ based on the budget, time and other constraints that each company has, however these questions help guide making your decision. Please click the ‘+’ sign to expand.

Stakeholder Frequently asked Questions - Management, investors and decision makers

  • How do we put a structure around the jargon and how do NOT end up comparing apples and oranges?
  • How do I make a case for budgeting a Test Automation solution?
  • How much money do I put in and what is the justification for the ROI?
  • What are the skill sets (existing) and what staffing needs?
  • What is the process and what are the technologies?
  • How do I measure the progress (aka. what metrics) and pivot and reset the direction if diverging?
  • How do I get employees excited about the change (organizational change)?
  • How to identify adopters, nay-sayers and staff?
  • In-sourcing vs. Out-sourcing vs. Vendors
  • Marketing messages and communication across the organization
  • Showing support and buy-in for the process and technical practices
  • Alignment from Business stake-holders

Technical Frequently Asked Questions - Languages, Programming, Frameworks

  • Programming languages can be Java, Ruby, C#, groovy, vbscript, php and so on.
  • Some of the libraries can be Selenium, Cucumber, testNG, Watij, Apachi-poi and so on.
  • Frameworks can be Data-Driven, Keyword-driven, Page-object, Hybrid and so on
  • Development models can be waterfall, spiral, Agile, TDD, BDD, ATDD and so on
  • Cloud Solutions like Sauce Labs that can do both Selenium based Web Automation and Appium based Mobile (device and simulator) automation
  • Windows or *nix based environment
  • API testing (SOAP or REST) and Tools
  • Service Virtualization with tools like IBM Test Virtualization, HP SV, Parasoft and whole lot of open source solutions
  • Test Data Handling – Obfuscate data, ingestion scripts, reset test data etc.

Gyan or Experiential Advice based on working with diverse set of clients

  • If you are a Java shop, it is a good idea to select Java Stack (Selenium-java, cucumber-jvm, testNG and so on)
  • If you are following ATDD and expect developers to help testers as they go through learning curve, again choose the stack that your source code is already written in. What your developers would be most comfortable with.
  • If you prefer that testers with no programming background be converted to automation engineers and that they would maintain the scripts over time, please bear in mind that Test Automation is a development project. It is going to take a significant amount of time and resources to get to self-service mode. Ruby might be a good stack to start in this case
  • If you are a high-tech shop with highly-skilled engineers and you don’t really differentiate dev and testing, pick JS + Node stack or you actually don’t really need much advice from this website 🙂
  • If you are .NET shop, again there is NUnit,C# Selenium and so on.
  • If you plan to introduce more open source in your .NET shop, choose either Java or Ruby or JS stack (it really pays dividends in long run)
  • Whatever said above, there should be leadership support and buy-in. Things will seem slow in the beginning
  • Hire Consultants, but please have success criteria well-defined in terms of your success metrics “continuously” tracked.