Feature analysis and ensemble-based fault detection techniques for nonlinear systems

Abstract Machine learning approaches play a crucial role in nonlinear system modeling across diverse domains, finding applications in system monitoring, anomaly/fault detection, control, and various other areas. With technological advancements, today such systems might include hundreds or thousands of sensors that generate large amounts of multivariate data streams. This inevitably results in increased model complexity. In response, feature selection techniques are widely employed as a means to reduce complexity, avoid the curse of high dimensionality, decrease training and inference times, and eliminate redundant features. This paper introduces a sensitivity-inspired feature analysis technique for regression tasks. Leveraging the energy distance on the model prediction errors, this approach performs both feature ranking and selection. Additionally, this paper introduces an ensemble-based unsupervised fault detection methodology that incorporates homogeneous units, specifically long short-term memory (LSTM) predictors and cumulative sum-based detectors. The proposed predictors utilize a variant of the teacher forcing (TF) algorithm during both the training and inference phases. Additionally, predictors are used to model the normal behavior of the system, whereas detectors are used to identify deviations from normality. The detector decisions are aggregated using a majority voting scheme. The validity of the proposed approach is illustrated on the two representative datasets, where numerous experiments are performed for feature selection and fault detection evaluation. Experimental assessment reveals promising results, even compared to well-established techniques. Nevertheless, the results also demonstrate the need to perform additional experiments with datasets originating from both simulators and real systems. Further possible refinements of the detection ensemble include the addition of heterogeneous units and other decision fusion techniques.