Data processing and storage is a crucial part of working with data as part of the IoT. Once we have obtained the data, we must determine how we can analyze, interpret, and work with the data. In different use cases, data is processed in different places of the architecture; the device obtaining the data may process the data itself while in other use cases, it may transmit the data to other places first, such as gateway devices or cloud applications, for the purpose of conducting further analysis.
The processing power and storage size that an IoT application needs rely on the performance needed by the consumers of the data; consumers may use different services of applications that utilize the data either for further processing or as an end goal. These may include factors such as memory, number of cores, clock speed, or processor specifications; all determine the device’s capability for data processing.
Two distinct scenarios arise when considering where to process and store the data:
Battery-powered devices: Consider a simple device such as a temperature sensor powered by batteries. Such dummy devices aren’t typically designed for local data processing and storage. The simpler the design, the lower the power consumption and, subsequently, costs.
Externally powered devices: Devices with an external power source, such as machines or electricity meters, can afford to have local data processing and storage. This arrangement not only reduces service latency but also conserves wireless backhaul bandwidth, enhancing the overall efficiency.
On the capacity side, this is what makes the consideration of non-volatile memory—the temporary data storage before data is transmitted upstream—imperative; we want to account for being able to hold data while considering the different environments that the device may be in. Is the data in environments of intermittent internet connectivity meaning there may be a chance of data loss if it relies fully on the internet to continuously store data in an upstream provider? Do we do data processing within the device itself? These are all questions that we need to consider when carefully choosing the hardware we need.