A product manager’s checklist for mobile app testing

As always, we like to begin our posts with some numbers. According to the data portal, Statista, as of June 2016, there were 2 million mobile apps on Apple’s App Store and about 2.2 million apps on Android.

And, these numbers are so large; it has given rise to a whole new category labeled the ‘app economy’. According to Gartner, by 2017, over 268 billion downloads of mobile apps will generate cumulative revenue of US $77 billion.

But, this data gives us only part of the big picture. The other side of the story is that if your app is not among the Top 1000 apps (especially in case of a consumer app) in your country, chances are your Monthly Active User (MAU) numbers, a key metric that translates into revenue, will be low.

Of course, the key to high MAUs depends on the purpose your app serves and the features it offers.

But, in addition to that, quality and user experience become crucial. If your app crashes a lot, has install issues or is inconsistent across devices/screen sizes, the word will quickly spread, and your users will move straight to your competitors. After all, there are tons of alternatives available.

Let us assume, as a product manager, you’ve crossed the first bridge; You’ve conceptualized and built an app that users want to download and use daily.

How do you ensure quality? What is the process you’re going to follow to ensure the app works in multiple scenarios? Will it work on both landscape and portrait?

What if the mobile device’s battery is low? What happens when a call comes? Most importantly, how secure is the user’s data on your app?

This post, aims to put together a checklist of sorts, for product managers to use during the mobile app testing process.

But, before that, here are some findings from the seventh edition of the World Quality Report, which is based on a global market research study conducted with 1,560 senior IT executives and testing leaders from 32 countries:

• Continuous and automated security testing will be a key strategy
• Agile and DevOps will take a front seat in testing
• Predictive Analysis will be a major enabler to ensure on-time and efficient application delivery
• Customer and business assurance will be a key focus area

Mobile App Testing Checklist

Keeping these macro trends in mind, we’ve put together a checklist that every product manager needs to go through, as he or she conceptualizes, launches, enhances and maintains the app. Here you go:

Security: 

The most crucial element to ensure is the security of a user’s data and his/her device. It goes without saying that whether your app is a game or an enterprise tool – the security needs cannot be compromised. Hence, mobile app security testing services is mandatory.

Install and uninstall process: 

For any user, installation is the first step and a bad experience will have him or her delete the app in a jiffy.

As a product manager, you need to realize that it is the app’s first touch point with a new user. Ensure a smooth install, irrespective of type of device, screen size, telecom carrier or wireless network.

Storage and Memory: 

There may be several app choices, but there is only a finite amount of space in any device.

Optimize your app for the amount of storage it takes up, the device memory it uses and ensure your testing process covers this.

Multiple Devices and Screen Sizes: 

This is actually the biggest challenge in mobile app testing. Therefore, as a Product Manager or QA Lead, it is paramount to have a robust Test Automation strategy in place.

Can you run a test automation process to test the app in parallel for multiple devices, operating systems and browsers? How much time can you save?

Are you using the right test automation framework, that enables you to do this seamlessly? (Shameless Plug: Check out Indium’s iMobi, a proven, widely used, tool-agnostic, Mobile Test Automation Framework).

1. Call interruptions: After all, your mobile device is primarily a phone. Therefore, one needs to pay careful attention to how your app reopens, after it is interrupted by a call.
2. Battery Life: Does your app slow down when the device is in low-power mode? How does it work when the mobile phone’s or tablet’s battery is low? As a product manager, one needs to pay careful attention of many such scenarios.
3. Older OS versions: A large number of users, often, don’t upgrade their operating systems. You may optimize your app for the latest iOS or Android, but are you checking and communicating this requirement to the user. Can you make it work in a previous version of the OS? Most importantly, how are you testing for older versions? Here’s where test automation services can really help save time.
4. The touch test: The latest devices use the most cutting edge touch technology. First of all, are you taking advantage of it in your app? Is your app’s response to a touch perfect? And, how do you go about testing your app for response to touch? These are key questions a product manager must answer.
5. The accelerometer: The latest mobile phones have the ability to gauge movement and gravity. If you’re developing an app that uses this functionality, then make sure it works across the different devices. If your app must not respond to movement, it is crucial to check that as well.
6. Testing for sound: If your app uses the phone’s or tablet’s audio function and headphone jack, have a test plan in place to check that.

The role of test automation

According to the 7^th edition of the World Quality Report 2015-2016, the No. 1 Technical Challenge in Mobile App Development Lifecycle is reliance on manual testing.

Over 39% of the 1560 CIOs and IT Leaders, responded that reliance on manual testing was their biggest challenge.

To give you a perspective, the CIOs who responded believed that this was a greater challenge than design complexity and over-burdened developers.

At Indium, we believe, our iMobi Mobile Test Automation Framework, can help CIOs and their teams tackle this major technical challenge.

iMobi, which is customizable and portable across almost all testing tools – both open source and commercial – can help jump start your test automation process and help save valuable time.

Why iMobi?

• Reduce test cycle time, by running tests across devices in parallel
• Save 2-4 weeks by jump-starting your test automation process
• Leave more time for features that have to be manually tested
• Our framework is tool agnostic
• Reduced test scripting time
• Huge repository of reusable components, minimizing valuable time