Model Poisoning Detection via Forensic Analysis

In today’s modern battlefield, where drones, missiles, and other autonomous systems are beginning to rely on machine learning (ML) models, the importance of detecting and mitigating model poisoning cannot be overstated. A compromised model could lead to catastrophic outcomes, making robust detection strategies essential for national security and defense. Model poisoning poses a critical threat to the security and functionality of ML models, especially in domains where data integrity is crucial. This paper introduces a novel forensic analysis methodology for detecting model poisoning. Unlike traditional approaches that scrutinize input data for tampering, our strategy identifies signs of compromise within the ML model itself, determining whether it has been trained on manipulated data. Our forensic analysis methodology, which we demonstrate to detect model poisoning of a convolutional neural network trained using the CIFAR-10 dataset, uses a unique pipeline composed of model reverse engineering, topological data analysis, Shapley values, and Benford’s analysis to detect subtle anomalies indicative of poisoning. This work aims to provide a robust foundation for future research and practical applications in safeguarding ML models against sophisticated poisoning attacks.