Reliable Privacy Risk Testing for AI Systems

The field of AI systems is quickly evolving. There have been a number of important discoveries and improvements over the last couple of years. One area that has become increasingly important is that of AI risk assessment and testing. Understanding how data leaks from a machine learning model and what can be done to prevent leakage is important for AI systems to comply with regulations such as GDPR. In this thesis, model leakage is assessed using two Membership Inference Attacks: the Likelihood Ratio Attack and the Robust Membership Inference Attack. Robustness for machine learning models can be proven empirically with confidence bands created using these attacks. The primary aim of this work is to investigate cost-effective methods for estimating model robustness, and to examine how assumptions influence the resulting confidence bands. Three methods for creating the confidence bands were tested: Brute Force, Leave-One-Out, and Bootstrapping. The Brute Force method served as a baseline for evaluating the estimations produced by the Leave-One-Out and Bootstrapping methods. When making these estimations certain assumptions about their parameters were made. An analysis was conducted on the impact that the number of shadow models, the size of the model pool, and the inclusion of augmented queries had on the resulting confidence bands. The results suggest that the robustness of both attacks can be estimated fairly accurately, the Likelihood Ratio Attack using Leave-One-Out and the Robust Membership Inference Attack using Bootstrapping. When using Bootstrapping, the model pool should contain approximately three times as many models as those used in the membership attack to achieve optimal results. These findings are preliminary. Several assumptions remain that require further examination, along with potential improvements to the techniques used in this work.