Are Exemplar-Based Class Incremental Learning Models Victim of Black-Box Poison Attacks?

Class Incremental Learning (CIL) models are designed to continuously learn new classes without forgetting previously learned ones, often relying on an exemplar set to retain a portion of knowledge from previously learned classes. However, their vulnerability to adversarial attacks under novel and unexplored conditions remains unstudied. In this work, we are the first to evaluate the robustness of exemplar-based CIL models using a non-overlapping dataset, where the dataset is independent of the training and test sets of the target model. We propose and implement a novel black-box attack framework targeting the exemplar set of class incremental learning models using zero-overlapping data. Specifically, we focus on scenarios where the target model provides only hard-label predictions with-out interactive access. Our experimental evaluation covers a range of exemplar-based incremental learning algorithms, different surrogate models, and black-box attack options. Our findings reveal significant vulnerabilities in exemplar-based CIL models to poisoning-based attacks using a non-overlapping dataset.