Detecting Black-Box Model Probing Attacks Through Probability Scores

In the black-box model probing attack, the attacker sends a series of model inference requests to a victim model to map out the classification boundary of the model. This attack is considered critical because it helps the attacker gain a better understanding of the model and launch follow-up attacks. While most existing works defend this attack through the input of the victim model, this paper addresses the problem by utilizing the output of the victim model, i.e., the sequence of probability scores generated through the model inquiries. Our research discovered that the sequences of top-2 probability scores generated by different model probing attacks form unique patterns that can be detected through time-series classification methods, such as Transformer and Long Short-Term Memory. Our experiments showed that two deep learning-based time series classifiers, Transformer and Long Short-Term Memory(LSTM), can accurately detect and identify six OARS-enhanced black-box model probing attacks. Particularly, compared with existing defense methods, our classifiers can reduce the Attack Success Rate (ASR) of the OARS-enhanced attacks from 93% to 1.78%. To facilitate further study on the black-box model probing attack, we released our collected probability score sequences as an open dataset: https://www.kaggle.com/datasets/drvoyager/model-probing-attack-dataset.