In today’s tech world, Mobile applications are widely used than regular desktop applications. The number of users who use smartphones for accessing most of their applications than using a desktop or a laptop is increased drastically.

Also, the end user will be expecting a good response from the application when he is using it. Hence, the developer who builds the mobile applications needs to keep this in mind while developing the application.

Now Performance Testing is critical either it is a regular web application or an application on a mobile device. If an application goes live without being performance tested and if the end user faces performance issues like a slow response, etc., then he will try to find another better application which will perform in a better way.

Any application has to be tested thoroughly before it is made available to the end user.It determines how the application responds under a particular number of challenges such as low battery power, out-of-coverage network, etc. In general, performance testing will test the speed, stability, and scalability of the application.

What is a Mobile Application?

Mobile applications are a set of predefined functions that are designed to work on mobile devices such as Smart Phone, Tablets and other feature phones.

Types of Mobile Application

• Native Mobile Applications
• Web-Based Mobile Applications
• Hybrid Applications

Native Mobile App

Native apps are built specifically for a mobile device’s operating system (OS). They provide high performance and are very reliable too. These type of applications also access the other applications on your smartphone device such as photos, address book etc.Examples of Native applications are a calculator, calendar, Facebook application etc.

Web-Based Mobile App

Web apps behave similarly to native apps but are accessed via a web browser on your mobile device. They’re not standalone apps in the sense of having to download and install code into your device.One kind of web app is the progressive web app (PWA), which is basically a native app running inside a browser.Examples of Web-Based Mobile Applications are online games, social networking sites etc.

Hybrid App

Hybrid Applications are similar to Native applications and they run inside the native container, they are a mix of both Native and Web applications.These are web apps that look and feel like native apps.Examples of Hybrid application are Instagram, Yelp etc.

Why Mobile Performance Testing?

• Validate the performance of an application during big workloads. Performance testing allows product owners to measure the performance of the system as the number of concurrent users or actions increases. A tester will know the response time and the amount of system resource consumption at the app’s peak points. This is highly helpful in case your app has traffic spikes (where, during a certain season or period of time, the number of visitors surges and decreases shortly after). 
• Validate the use of hardware. Ensure that low CPU usage or little server storage will not prevent the app from functioning according to the requirements. 
• Determine the capacity of the application. Determine if the current software is capable of matching the set of metrics designed beforehand (peak traffic load, server response time, and so on). 
• Measure and assess the performance of the app on real load. A tester will be able to get a realistic simulation of real traffic, create a load profile, and measure the response time.
• Check the app’s performance under critical conditions. To be able to predict and prevent system crashes. 

Strategy for Performance Testing Mobile Apps

Performance Test Approach differs for each type of mobile application. The approach varies from one application to the other as each application is developed with different components and the processing mechanism is different for each.

Browser Based Mobile Applications act differently than the regular web browser application. This behavior is due to the bandwidth from the data plan that the end user is using and it also depends on the configuration on the mobile devices.Hence for performance testing such as mobile browser-based applications, we need to replicate the user load on the browser. We can do this by using emulators or any other additional components. Apart from the user load on the mobile browsers we also need to test the web page rendering on the devices.

Native Mobile Applications are installed directly on the device which makes the user experience better, but the application is platform dependent.So if we want to test performance in these type of applications then we need to test all the platforms of the mobile devices like Android, ios, windows etc and also on all types of devices like smartphone, tablets etc.

Hybrid Mobile Applications are a combination of both Browser Based Mobile Applications and Native Mobile Applications, hence here we need to test the user load on the server and also we need to do full a stretch performance testing on all the mobile devices and platforms.

Step 1. Align test objectives and business requirements

Performance testing will only be as good as its goals. When setting objectives for each case, make sure they correspond to business scopes of building an app. A tester’s top priority would be to assess the functions of the app that are crucial for lead generation – the cart feature for e-commerce stores, the registration and contact form, and so on. 

Step 2. Identify test KPIs

By setting benchmarks for testing, you’ll be able to tell if the execution was a success or a failure. For mobile app testing, the following KPIs are worth consideration:

• Error rate
• Maximum response time
• Average response time
• Peak number of requests
• Average throughput and the peak number of concurrent active users per device and OS.

Step 3. Prioritize scenarios

Testing all the features of the app all at once without prior selection of cases is a common trap for a beginning tester to fall into. Instead, it’s better to outline multiple packet scenarios that are crucial for the app’s reliable performance. 

After you have assured crucial features & Business-critical scenarios work properly, feel free to move in deeper and test less significant scenarios. 

Step 4. Simulate a real-life testing environment

The testing environment is crucial in order to understand the user experience of the app the way people will after it’s live. In order to speed up the testing process, QA specialists mostly use system emulators. 

Such tools do a good job of emulating the basic parameters of an operating system and allow testers to get a sense of the look and feel of the interface. However, to assess the performance of features that require camera permission, GPS, and other device-specific functions, it’s better to use real hardware. 

Step 5. WorkLoad Modelling

As per requirement choose type of performance test and perform workload modelling as per the real world scenario.

Step 6. Keep the latency and the bandwidth of a carrier network in mind while testing

Due to the fact that mobile devices are connected to the internet through third-party carriers, the latency and bandwidth of the networks can vary. Keeping the carrier network differences in mind allows a QA specialist to optimize the app’s performance and improve user experience. 

Mobile Application Performance Testing Environment

Environment setup is capable of determining the accuracy of tests and the number of insights a QA specialist can gather out of any executed test. Setting up an environment for an app or mobile website performance testing consists of the following factors:

Emulators: Create an environment which consists of Emulators and real devices.An emulator is a software which can be installed on our laptops or desktop machines, and these act as mobile devices. We can generate huge load using these Emulators, hence they are best to use for load testing any mobile applications.

Operating Systems: What are different operating systems that will be used for mobile apps testing.

Devices :What are different devices & their availability on which we need to perform the testing – Physical or Cloud.

Geographic Locations:geographic location of the application devices, this is a mandatory and very important aspect to be considered or else our test results will not be as what happens in the real world scenario.

Isolate the environment. Make sure you’re the only one to use the system for testing and no other activity is performed on it simultaneously.

Isolate the network. This is done in order to eliminate timeout errors. It’s not a necessary guideline – if the bandwidth is wide enough to support both a tester and other users, there’s no need for isolation. For most networks, however, supporting a few concurrent activities without affecting the performance of the app is impossible. 

Remove proxy servers from the network path. A proxy between the client and the web server can put a strain on the performance of the app. 

Application monitoring: To monitor the application behaviour during the performance test.

Key Components in Mobile App Performance

Below are Key components in mobile performance testing.

1st Component : The Backend Processing Time

A mobile app’s backend architecture is generally based on an application server, a web server, and a database.When it comes to the backend, the things related to performance that are important to know when an app is under load are the server’s response times, database queries times, and the server’s resource usage. 

This is the total time taken by the request to reach the server, response generation time by server and the time to deliver the response back to the client. This time is affected by the total load on the server, network bandwidth, data transferred and the external component.

Using this information, it’s easier to detect issues such as: 

• High server response times
• Bottlenecks or breakpoints in the database and application server resources
• Poor implementation of escalation policies 

In general, the objective of these tests is to understand how the backend systems of an app behave and handle a certain volume of concurrent users.

2nd Component : Network Emulation

Networks have a wide range of variability when it comes to mobile internet usage and this makes network emulation important for mobile performance testing. We need to effectively emulate the network condition to replicate real-time traffic during performance testing.

With regards to the network that the device is connected to, there are two key things to measure: latency and bandwidth.

• Latency is the time that elapses when information is sent on the network (measured in milliseconds).
• Bandwidth is the maximum capacity (the amount of data) that can be transmitted through the network (measured in bits per second). 

An app’s performance can vary depending on, for example, whether it’s connected to a 3G network or a 4G network, and unfortunately, this is beyond an app developer or tester’s control. But, it is possible to incorporate the network during the mobile app performance testing process, simulating the different types of networks and measuring their impact on the response times, both on the server-side and the client-side.

3rd Component: Client Rendering Time

The time it takes to display the response delivered to the client on the device is measured. The platform type, configuration of the device and other hardware configurations affects this time. This time should be handled using an on-device monitoring tool.

Test mobile app performance as early as possible

If you start testing early on, from the first version of your app, you’ll give yourself more time to understand how it behaves, get familiarized with client side metrics, and run these tests alongside functional ones in each sprint.Another advantage of testing early is you can test on different devices at your disposal as well as understand how these metrics evolve over time.

Mobile Performance KPIs

Transaction response time (how long it takes to complete an action):

• Server response times 
• Database response time 
• Network latency
• Screen drawing time

Device resource usage (How the app uses the resources of the device it’s running on):

• % CPU
• Memory (Mb)
• Data sent and received by the application (Kb)
• % Battery usage

Rendering (Measures how the app is drawn on the device):

• Render time (ms)
• Frames per second (fps)
• Render lag (janks)

Performance Bottleneck in Mobile App

• Memory leaks: when an app doesn’t free up memory well, it increases until it runs out of available memory and then crashes 
• Screen freezes: due to very high rendering times
• Crashes: exceptions that jump in the app and cause it to suddenly close

Challenges in Mobile App Performance Testing

• When we need to test our application on different mobile devices, we need to organize all the devices like IOS, Android, Windows etc.
• Network simulation is another challenge we will face i.e we need to simulate networks like 3G, 2G, 4G and wifi networks.
• The memory available in the mobile devices to handle the test runs.
• The battery life of the Mobile devices.

References:

https://blog.apptim.com/how-to-test-mobile-app-performance-3-key-ways/

https://performancelabus.com/mobile-app-performance-testing/

https://www.testhouse.net/blogs/performance-testing-of-mobile-apps-how-is-this-different/