Performance Testing in an IoT World

The Internet of Things (IoT) revolution is going strong. A stroll through the aisles of a department store, home improvement center, or electronics store reveals a seemingly limitless assortment of connected smart devices. 

In 2017, 8.4 billion IoT devices were in use, a 31% increase from 2016. Gartner also anticipated that there would be 20 billion devices by the year 2020. Despite the pandemic having derailed this prediction temporarily, the IoT momentum is steadily gaining speed. 

Organizations seek to integrate IoT devices into their network infrastructures and explore new ways to utilize and manage the collected data. IDC forecasts that the amount of data created by IoT devices will reach 79.4 ZB (zettabytes) by 2025. According to Mordor Intelligence, the IoT technology market value is expected to rise to $1.39 trillion by 2026. Along with AI, connectivity (5G, LPWAN, Wi-Fi 6, etc.), EDGE computing, and wearables (smartwatches, wristbands, etc.), IoT is only set to soar ahead.

Due to their ability to connect to a broader network, IoT-enabled devices can achieve a high level of functionality. Nevertheless, this introduces a new difficulty: ensuring that devices perform as expected.

You might be interested to read on IoT Testing Approach in Devices

IoT use cases in different industries

Indium is pleased to have IoT as part of its digital assurance portfolio, along with our capabilities around IoT performance engineering. Let’s quickly touch upon how IoT is effectively implemented in different domains:

Healthcare: Wearables, defibrillators, oxygen pumps, connected inhalers, ingestible sensors, IoT in hospital management, monitoring patients remotely, monitoring glucose, heart rate, depression, Parkinson’s disease, etc.

Retail: Facility management, buyer behavior, supply chain, cold chain monitoring, personalized shopping, smart shelves, etc.

Manufacturing & Logistics:Real-time machine monitoring, predictive maintenance, production visibility, integrating systems, compiling, asset utilization, communications systems, vehicle tracking, location and route management, inventory tracking, etc.

BFSI:Real-time monitoring, fraud detection, improved analytics, better customer experience, cardless transactions, etc.

Travel /Automobile: Advanced Driver Assistance Systems (ADAS), blind-spot monitoring, lane departure warnings, parking assistance, object recognition, real-time line visibility, metal inspection, quality control, vehicle health check and inspection, etc.

Education & Media:Connected TVs & devices, facility monitoring and maintenance, data from other facilities, student-instructor response, performance and behavior, etc.

Energy:Optimizing energy spending, predictive analysis, smart grid systems, etc.

Also Read: IoT Testing Challenges & Approach for most of the cases

Focus areas of IoT performance testing

Clearly, IoT is making its presence across all verticals. However, the approach to IoT performance testing solutions is not as straightforward as web or API performance testing. Some of the focus areas of IoT performance testing can be segregated as follows:

IoT Platform

– Concurrent EDGE devices

– Bulk volume of message flow

– Batch performance

Business users

– E2E time for servers

– E2E time for users

– End-user applications

Messages from IoT devices (which can also be activated by emulators) are delivered to the Gateway using underlying protocols such as Zigbee, CoAP, Bluetooth, Wi-Fi, etc. Emulators are not always possible, and depending on the technology available, we may need to build some code to push bulk data.

Performance testing in action

IoT communication routes include HTTP/S, XMPP, TCP/IP, UDC, and others. The performance testing technique will span multiple layers, such as

Device layer 

• The key objective is to provide an end-to-end user experience and track event processing and malfunction, scalability of bidirectional devices, the energy usage of batteries, and so on. We can also conduct device profiling, API, and e2e performance testing.

Connectivity layer

• We can measure multi-geo performance, gateway scalability, network latency, and performance. We can also do batch performance tests, real-time analytics performance tests, and DB performance tests.

Data layer

• Includes event validation, data retrieval, batch performance, and predictive analytics.

End-user touchpoints

• Tests the end-user experience across the web and mobile, as well as scalability and reliability.

When it comes to tools, we can use any commercial or leading tools depending upon the technology stack. Typically, MQTT and web protocols are used. Tools like JMeter, LoadRunner, Neoload, etc., can be leveraged. The tool will be finalized depending on the IoT performance testing strategy. 

Fulfilling the expectations of customers and end-users is a high priority. Indium would like to recommend performance testing for all applications with a good customer base, irrespective of how it is being hosted. Imagine your frustration if your GPS directs you to take a turn after you’ve crossed it because of poor network connectivity. The moment the application slows down or the user receives data in a delayed fashion, the end-user experience will take a beating. Make sure you have done a performance assessment as one of the ‘gates’ before ‘going live’ and save your systems.

When it comes to tools, we can use any commercial or leading tools depending upon the technology stack. Typically, MQTT and web protocols are used. Tools like JMeter, LoadRunner, Neoload, etc., can be leveraged. The tool will be finalized depending on the IoT performance testing strategy.

Fulfilling the expectations of customers and end-users is a high priority. Indium would like to recommend performance testing for all applications with a good customer base, irrespective of how it is being hosted. Imagine your frustration if your GPS directs you to take a turn after you’ve crossed it because of poor network connectivity. The moment the application slows down or the user receives data in a delayed fashion, the end-user experience will take a beating. Make sure you have done a performance assessment as one of the ‘gates’ before ‘going live’ and save your systems.

Modernizing QA with life-cycle automation and AI practices to address scale, speed, security and innovation in the cloud is a prerequisite for Digital Transformation.