Ensembles for Outlier Detection and Evaluation

Ensemble methods are widely used in many machine learning applications such as classification and recommender systems. However, ensemble methods have been slow to develop in unsupervised domains such as outlier detection[1]. The earliest methods for outlier ensemble analysis include techniques such as feature bagging and isolation forests[5,6]. Subsequently, theoretical foundations were developed for outlier ensembles[2], which turned out to be analogous to those used in classification. Therefore, many outlier ensemble methods from classification can be generalized to outlier detection. However, the unsupervised nature of the outlier detection problem necessitates some changes to these algorithms. For example, subsampling methods need to be replaced by variable subsampling in order to obtain the best results[2]. This is because variable subsampling methods implicitly explore the parameter space over different base detectors so that problems associated with lack of supervision are addressed. Outlier ensemble methods can be either data-centric (in which components use different subsets or subspaces of the data) or they could be model-centric (in which components use different variations of model-centric design). Examples of data-centric methods include methods like feature bagging and subsampling, whereas examples of model-centric methods include Isolation Forests[6], RandNet [4], and Subspace Histograms[7]. However, techniques like variable subsampling seem to have characteristics of both types of ensembles.