The Internet of Things or IoT has been around for more than two decades, but its foray into the mainstream business has gained prominence in the last decade. IoT refers to a network of interconnected physical devices collecting and sharing data through a network. Organisations around the world are using IoT to operate efficiently, make informed decisions with real-time data, and provide enhanced customer experience. As per Statista, iPriopertyManagement.com, in 2021, the number of IoT devices is projected to surpass 75 billion by the end of 2025. Testing these devices will pose many challenges for IoT testers in the coming years.
The main challenge which is faced in the testing of IoT is due to its vastness, integrating multiple devices, machines and sensors, like pulse-oximeter, electrocardiogram, a person with a heart monitor implant, a farm animal with a biochip transponder, an automobile that has built-in sensors to alert the driver when tire pressure is low, etc. These sensors and machines communicate in multiple protocols with an edge computing/gateway device which acts as a pipeline to collect the data from different sources. They send the collected information to the cloud, ensuring there is no loss of data, and the data is read in multiple ways by the user through phone/web interfaces etc.
In this article, we are going to discuss some of the common challenges in IoT that testers often encounter. We will also throw light on how to address them by offering an IoT testing approach before guiding you on how to select the right tools for IoT testing.
1.Devices/ Sensors: The data generated from a device/sensor doesn’t have intelligence on its own. The true challenge is to test data from ‘n’ number of devices that are sent at once. It can be wired/wireless, with a protocol that might differ as per business requirements. Hence, the verification of the data validation needs to be done in all these scenarios needs to be done to ensure only valid data flows in from the source. In the event of invalid data, there needs to be clarification of errors related to the same.
2. Edge computing/Gateway devices: Though this is a black box, the gateway devices are the heart of the IoT system. This has a complex configuration of the system, the schedulers, data processing and the business application firmware. The Gateway device needs to be verified to ensure that the device never goes offline. Even in case it goes offline, the reasons for failure need to be clear, and the data should not be lost but stored and then sent to the cloud as and when the device is On. The Gateway load needs to be verified to ensure that huge data transmission is not going to kill the memory of the device, but there is always a smooth transmission of data.
3.Data processing/Cloud: There are several cloud data processors available in the market. The verification of the cloud starts by ensuring that the configuration between the gateway device and the cloud is the same and secured. Voluminous data can have the same or multiple protocol communication methods. The verification of valid data points is key so that no data loss becomes crucial in this layer. Failure scenarios are well possible in case of data loss, and so the cloud also needs to be tested.
4. User Interface (UI): There are again many user interfaces that can vary from smart devices (watches, health monitors etc.), mobile applications, web interfaces, desktop interfaces, Kiosk, TV to anything where a human can read the information. With businesses trying to provide a seamless omni-channel experience, the burden on testers is huge as there are endless software and hardware devices. Testers need to ensure that the application works seamlessly on the required software. For those that are not supported, proper information needs to be given.
5. Security: Data security is pivotal. With numerous devices connected through various networks, testing and fixing vulnerabilities that can pose a security risk for business is crucial. Let’s discuss the security aspect by considering the following example.
You might be interested in : IOT Testing Approach on devices
In a mall, a fire sensor was replaced as it was not working. The sensor which was replaced is of the same brand but a different version type. In such a situation, testers need to ensure that:
From the above challenges, you must have understood that test scenarios are huge for functionality testing. But testers don’t always have time to test all the scenarios in an exploratory model, which may lead to delivery delays. So, it is important for testers to intelligently plan the test approach according to the product/project and minimize the time of test, and finish testing on time.
The following table gives an idea of what testing types will be applicable for the IoT layers.
|IoT Testing Types||Sensor||Application||Network||Backend|
The best practice to plan testing is to have a checklist or an acceptance criterion clearly defined for the whole product. This helps to set a clear boundary for the testing team and to avoid testing other scenarios.
The following are a few samples questionaries that must be considered here:
2. For Software and Hardware, it is good to start testing once the requirements are clear. Points that need to be considered here are:
c. As for the cloud, ensure that there will always be upgrades, and those upgrades are captured in the system. The lower version support also needs to be known.
Several open as well as paid tools are available in the market. Choosing the right tool depends on the design strategy of the IoT product, Saas, Paas or IaaS.
It is also difficult to have a single tool for a complete product and to design an automation framework for a product as a whole. So, it is good to split up testing according to the layers and use separate testing methods and automation tools.
However, among all the integration, testing will play a major role. While there are CI/CD pipeline models, integration testing brings in lots of possibilities and metadata and meaningful testing to the product. The Northbound of IoT (Cloud-> communication network ->UI) can be chosen to be automated separately. Whereas testing gateway and devices require apt tools according to the product/domain and architecture. As the investment in the hardware is quite high, choosing virtual simulators and emulators can prove helpful in generating the data points.
Another important aspect of IoT testing is pilot testing. In this, the product runs directly on the field post the verification runs are executed in the sophisticated lab environment with virtual simulations. Once the product is installed in the field, testing with the real data is very important. Only when this is done the reliability of the product is assured. In order to understand and collect the data, pilot testing ranging from 1 month to 6 months is advisable.
Even for a single IoT product, several components have to be tested. Hence it is always good to initiate testing in the design phase itself. Though exploratory testing takes a longer time compared to other testing methods, it is important in IoT as there are several users, and the same data is read by various users.
Remember that testing with requirements is never sufficient for the IoT, but exploring data and the reports and insights from the data source actually brings a great useful product in the end.
By Uma Raj
By Uma Raj
By Abishek Balakumar