Abstract ensembles for anomaly detection and beyond

Detecting anomalies is essential in domains such as cybersecurity, industrial monitoring, and scientific discovery, yet current methods often struggle with computational efficiency, robustness, reliability and interpretability. Similar to most machine learning tasks, ensemble methods have shown great promise here. Traditional ensemble methods combine powerful anomaly detection models, assuming that diverse models reduce individual errors. While this slightly improves performance, it comes at a high computational cost. In contrast, we explore ensembles composed of simple, specialized submodels that, individually, are weak detectors but collectively form an effective system. As we will show, this approach enables the construction of large and effective ensembles while maintaining computational efficiency, as each submodel focuses on solving a simple, abstract subtask. We refer to such ensembles as abstract ensembles. This thesis advances the study of abstract ensembles in four key ways. First, we introduce a novel deep-learning framework for anomaly detection based on abstract ensembles, providing an effective and easy- to-use training methodology. Next, we enhance this approach by proposing a shallow variant optimized for speed and a test-time training mechanism for improved performance. Furthermore, we analyze the advantages of abstract ensembles in terms of interpretability and adversarial robustness, demonstrating their potential for more transparent and resilient anomaly detection. Finally, we extend these ideas beyond anomaly detection to re-identification tasks, illustrating the broader applicability of abstract ensemble methods. This thesis establishes abstract ensembles as a principled, efficient, and robust approach to anomaly detection and related tasks with significant theoretical and practical implications.