Detectable Mislabeling – Can Faulty AI Models be Recognized from Incomplete Memory Traces?

Machine Leaning outsourced to remote cloud environments is prone to injections attacks when, e.g. in the communication channel, some fraction of a training set is contaminated, i.e. replaced with some "malicious" data or just mislabeled. In this paper, we research the detection of such injections attacks, based on inevitable, slight but detectable changes of weights in contaminated models. This involves the design of a Convolutional Neural Network super-model, called doctor, able to distinguish between healthy networks - trained on clean data, vs unhealthy networks - trained on data contaminated in injection attacks. Our approach for analyzing the network involves extraction of post-training weights, which undergo a CNN classification. The research proves that the reliability of remotely trained models can undergo efficient post-training verification based on images arbitrarily sliced from tested model weights.