Ensembled Outlier Detection using Multi-Variable Correlation in WSN through Unsupervised Learning Techniques

Abstract

Outlier detection in Wireless Sensor Networks is a crucial aspect in IoT, since cheap sensors tend to be seriously exposed to errors and inaccuracies. Hence, there is the need of a solution to improve the quality of the data without increasing the cost of the sensors. In Big Data paradigms, it is difficult to exploit the temporal correlation of sensors since Big Data architectures and technologies do not process data in order. In this paper, a complete study of multi-variable based outlier detection is carried out. Firstly, three known unsupervised algorithms are analysed (Elliptic Envelope, Isolation Forest and Local Outlier Factor) and are tested in a big data architecture. Secondly, an ensemble outlier detector (EOD) is created with the outputs of these algorithms and it is compared, in a Lab environment, with previous results for different parameters of contamination of the training set. The analysis of the results show that for correlated variables, multi-variable EOD has a very good detection rate with a very low false alarm rate. Finally, the EOD is used in a real world scenario in the city of Barcelona and the results are analysed using spectral-decomposition techniques which indicate that EOD has a good performance in a real case.