Rethinking security: the resilience of shallow ML models

Abstract The current growth of machine learning (ML) enabled the commercialization of several applications, such as data analytics, autonomous systems, and security diagnostics. These models are becoming pervasive in most systems and are deployed into every possible domain. Hand in hand with this growth are security and privacy issues. Although such issues are being actively researched, there is an evident fragmentation in the analysis and definition of the ML models’ resilience. This work explores the resilience of shallow ML models to a relevant attack of data poisoning, as poisoning data attacks pose serious threats, compromising ML model integrity and performance. Our study aimed to uncover the strengths of shallow ML models when facing adversarial manipulation. Evaluations were performed in a CAPTCHA scenario using the well-known MINIST dataset. Results indicate remarkable resilience, maintaining accuracy and generalization despite malicious inputs. Understanding the mechanisms enabling resilience can aid in fortifying future ML systems’ security. Further research is needed to explore limits and develop effective countermeasures against sophisticated poisoning attacks.