Typicality for Information Retrieval with Application on Sample Size Reduction of High Dimensional Data

This paper introduces typicality for information retrieval (TIR) for down-sampling training data with high dimensional feature space. High dimensionality and data size are often blockers to fitting complex machine learning models in various domains and can often contain noise. TIR allows for an efficient and explainable model training without compromising model performance. We leverage the notion of typicality, which is an empirically derived quantity that resembles probability. We use prototypes, which are the local peaks of the typicality as well as of the data density in a preprocessing step, as a way to extract most information from the training data. We apply the proposed approach on five diverse data sets including simulated and real industry data sets. Our results indicate that the downsampled dataset, obtained by our method, retains information content of the overall set and promises high model performance when compared with the existing preprocessing methods.