Incremental SliceLine for Iterative ML Model Debugging under Updates

SliceLine is a model debugging technique for finding the top-k worst slices (in terms of conjunctions of attributes) where a trained machine learning (ML) model performs significantly worse than on average. In contrast to other slice finding techniques, SliceLine introduced an intuitive scoring function, effective pruning strategies, and fast linear-algebra-based evaluation strategies. Together, SliceLine is able to find the exact top-K worst slices in the full lattice of possible conjunctions in reasonable time. Recently, we observe an increasing trend towards iterative algorithms that incrementally update the dataset (e.g., selecting samples, augmentation with new instances). Fully computing SliceLine from scratch for every update is unnecessarily wasteful. In this paper, we introduce an incremental problem formulation of SliceLine, new pruning strategies that leverage intermediates of previous slice finding runs on a modified dataset, and an extended linear-algebra-based enumeration algorithm. Our experiments show that incremental SliceLine yields robust performance improvements up to an order of magnitude faster than full SliceLine, while still allowing effective parallelization in local, distributed, and federated environments.