Distance-based change point detection for novelty detection in concept-agnostic continual anomaly detection

Abstract Anomaly detection provides an effective decision support capability in several real-world domains. One limitation of conventional approaches is their inability to preserve knowledge as models are constantly updated with recent data, leading to catastrophic forgetting. Continual learning approaches overcome this limitation by providing strategies that provide a trade-off between model stability and plasticity. However, to deal with concept-agnostic scenarios, transitions between tasks/concepts must be detected and provided as auxiliary information to the models. While change point detection methods are a natural fit, the most effective ones for complex and evolving data rely on choosing an appropriate distance measure. However, a fundamental knowledge gap in current research stands in how distance measures for change point detection impact models’ ability to adapt and perform over time as new concepts emerge from evolving data. In this paper, we address this issue by proposing a modular approach to identify transitions in concept-agnostic scenarios and investigating how different distances in change detection affect the predictive performance of anomaly detection models in continual learning scenarios. We perform experiments with different continual learning strategies and compare them with concept-incremental scenarios across multiple real-world datasets. Our key results highlight that it is feasible to perform concept-agnostic learning with a small decline in anomaly detection performance compared to concept-incremental. Moreover, this decline can be mitigated with proper selection of the distance measure for change detection. Finally, our results reveal that even moderately accurate identification of changes can lead to competitive anomaly detection performance.