Impact of regularization methods and outlier removal on unsupervised sample classification

BACKGROUND: High-content assays (HCAs) have problems distinguishing biologically significant effects from those of technical factors, such as differences in materials, personnel, and non-repeatable aspects of cell culture environments. The number and heterogeneity of descriptors evaluated is thought to introduce additional problems. This meta-analysis assigned data to different classes and then determined whether and how preprocessing operations affected outcomes. METHODS: Batch effects that could affect reproducibility-i.e., signal/noise ratio, instrumental conditions, and segmentation-were controlled variables. Batch effects that cannot be controlled include variations in materials, personnel, and tissue culture environments. Descriptors' values were measured directly from images. Values of a low-dimensional descriptor, factor 4, which was identifiable and interpretable, were derived by exploratory factor analysis. In each of five trials, one sample was treated with the same chemical mixture (EXP) and another with the solvent vehicle alone (CON). RESULTS: Factor 4 means of CON and EXP repeats showed significant differences after within-trial data regularization. The mean of Trial 3 CON differed significantly from all other CONs. These differences disappeared after the data were regularized using comprehensive databases. Among EXP repeats, the Trial 2 mean differed from those of other repeats, but regularization to other databases had little effect. Surprisingly, the same classification pattern was found after regularization to any comprehensive database derived by the same protocol. Substitution of databases derived by different protocols produced patterns that only reflect an elevation of differences that had been marginal to statistical significance. Outlier removal was deleterious. Even with the most sparing definition of outliers, over 3% of a single sample's contents were removed from most trials. Elimination based on overall within-trial distributions caused type I and II errors. CONCLUSIONS: These results, based on real-world experiments, suggest that non-repeatability is an inherent feature of in vitro assays. Classification patterns were unaffected by several irreducible batch effects, including materials, personnel, and non-repeatable environmental variables. These effects, combined with small sample sizes and skewed distributions of descriptors' values, may account for non-repeatability. Non-repeatable factor 4 means in repeated trials had negligible influence on classification, however, suggesting that class assignment is a better indicator of assay quality.