An Empirical Study on Learning Models and Data Augmentation for IoT Anomaly Detection

Among many other security applications, anomaly detection is one of the biggest users of deep learning methods. This growing popularity is mainly driven by two common beliefs: (i) its ability to manage complicated patterns inside large datasets (given a large amount of data) and (ii) its no need of separate feature engineering (as it is done within the model learning). In this study, we question both of those beliefs and revisit the effectiveness of feature selection and data augmentation in the performance of popular deep-learning based anomaly detection approaches. Additionally, we study the impact of other important factors of any learning based anomaly detection approaches including learning models (both traditional Ml and deep learning), data balancing techniques, hyper parameter tuning, etc. on their performance. From this study, we first report that those common beliefs are not always true - which necessitates a framework that can evaluate the usefulness of features and data for specific use cases (varying the data and need). Then, we propose a new framework that can fill in this gap and assist the data users and anomaly detection tools to perform better by selectively choosing all the configurations (such as, features, models, hyper parameter, balanced data, augmented data). Finally, we demonstrate the effectiveness of our framework using two major IoT datasets.