Methodology of adaptive data processing in IоT monitoring systems with multilevel sensor data filtering and self-tuning

Abstract

The study focuses on the processes of collecting and preprocessing heterogeneous sensor data. The aim of the research is to develop a method of adaptive filtering and automatic trigger adjustment that ensures stable operation of IoT monitoring systems in the presence of noise, impulse outliers, and seasonal fluctuations.

A methodology for adaptive data processing is proposed, combining multi-level data filtering with automatic self-adjustment of control thresholds in monitoring systems. This approach not only improves the accuracy of real-time sensor measurements but also dynamically adapts the monitoring system parameters to changing operating conditions, thereby minimizing the number of false incidents.

Within the study, a model of multi-level filtering was formalized, based on a median filter, a moving-average filter, and an exponential smoothing method. The use of a multi-level filter provides comprehensive data cleansing, stabilization of time series, and extraction of key trends. A mechanism for automatic adjustment of control thresholds in the Zabbix monitoring system was developed, where threshold values are determined based on statistical parameters and trends identified at the multi-level filtering stage. This mechanism integrates into the subsequent data-processing pipeline, ensuring that the system automatically accounts for daily, seasonal, and other fluctuations of the dynamic data-collection environment.

Experimental studies involving various types of sensors confirmed improved measurement accuracy and a significant reduction in false alerts in the monitoring system. In particular, humidity-measurement accuracy improved by an average of 6.52%, while impulse temperature spikes were reduced by 53.06%. Compared to traditional approaches, the proposed methodology provides higher noise resilience and adaptability to changing environmental conditions, making it an effective solution for industrial, environmental, and other real-time IoT systems.