Designing a mobile app automation framework for all apps

Having click baited you with a blog post that claims to solve a problem universally for all apps, let me now proceed to use this post to excuse myself out of that situation. Just kidding!

After more than 6 months of silence on this blog, which you can be sure was because both Om and I were busy building RobusTest, we return with a post which hopes is relevant to you. Searching for "mobile app automation" on Google should give you good idea of the kind of problems that people are trying to solve in the space and the sheer number of people who are trying to solve it. Believe me, we know we are in an unenvious position. And therefore, we need to constantly think of that one approach which can help us solve our customers' problems better, faster.

It has been a year since we started building RobusTest and while the platform is (meant to be) a full-fledged solution for mobile app testing, one of the things that we focus on is the automation part which consists of creating tests through recording, running them on multiple devices and generating reports that help. Needless to say it is a work in progress.

Over the last few days we have been pondering on what would constitute an automation framework which would help solve the problem a lot of mobile app companies are facing today. How do I automate meaningfully not just efficiently?

Let us take up a few tenets or points and see how should an automation framework be designed keeping that in mind.


1. Easy creation and updation of tests

I am sure all of us have heard the "do not automate apps that are not stable" phrase. I have heard it so often that I think it ranks right along with grandma's sayings. I too belonged to the same "side" but I realize that apps do not exist to conform to process; they exist to solve a specific problem and if that requires them to change at a certain pace, well then the automation process, "bloody well", keep pace. It is in this context that recording comes across as a possible solution or as some non-recording types would grudgingly agree to as the bridge to the solution. Even if recording cannot be not perfect, it should definitely be designed in a way that it can be used by just about anyone including the managers (evil laugh). At the same time, it should ensure that we do not sacrifice technical capabilities for ease of test case creation. This can be done by ensuring that test cases are modularized which makes it for easier editing in case someone wishes to delve into the code.

2. Help users think in terms of test cases not test scripts

Instead of forcing test creators to think in terms of test scripts or programmatic actions on objects like buttons and text fields, a good automation framework should encourage thinking in terms of use cases and tests based on use cases by providing tools that enable abstracting out the technical aspects of automation in the same way as the suitcase stashed under your bed - you will see it only if you are looking for it. In the normal and expected cases, the framework should encourage test creators to think in terms of "ensure the right price is displayed to the user when s/he is looking for car insurance". The user can then break down the test case into reusable test stubs which could include "login to the application", "calculate insurance premium", "verify it is correct".

3. A clear relation between test step and test script

While abstraction is the ideal objective as stated in (2), it important that a relationship be maintained between test steps and test scripts to create a bridge between the code people and the civilized people. It creates a common ground for the functional experts to talk to the automation engineers. That would require every line of test code (or a bunch of lines) to be covered by a test step.

4. A neat separation between test case and test data

Yes, yes! I hear "data-driven" being thrown around. Of course, our tests need to take data. In fact they should be so designed that if tomorrow you need to change an data for an object - be it an input value or a verification value, you should be able to just do it by editing a file (or changing it and updating references). This also encourages app testers to think in terms of app data thus improving coverage by treating data creation as a specific activity.

5. Creating object repository as default

While recording is ensuring that tests are getting created easily and faster, an object repository ensures that the happy marriage continues long after the recording honeymoon. Simple point - things will change faster than you will like but an object repository will ensure that you are not continuously running to keep pace with the changes. In fact a good framework will abstract out the creation of object repository manifesting itself only when things are change and even then they provide an easy an intuitive way to update the object attributes.

I am sure there are many more things that go into creation of a good mobile app testing framework; in fact what I mentioned above may just as well apply to web automation frameworks as well. Do let me know what are your top tenets when it comes to designing automation test frameworks.

And the customary Dilbert truth capsule


Comments

  1. This comment has been removed by the author.

    ReplyDelete
  2. This comment has been removed by a blog administrator.

    ReplyDelete
  3. really you have posted an informative blog. it will be really helpful to many peoples. thank you for sharing such kind of an interesting article. so keep on sharing.
    dot net training in chennai

    ReplyDelete

Post a Comment

Popular posts from this blog

Startups should automate testing ... Now!

Marketing, Quality, Your App & the Horse