Resilient Automatic Model Selection for Mobility Prediction

In order to avoid extensive machine learning models selection and optimizations, Automated Machine Learning (AutoML) has arisen as a practical and efficient way to apply machine learning to many different application areas. Data poisoning is a real threat to the accuracy of machine learning models in different settings, and it has in recent research studies been shown that the usage of AutoML can be even more sensitive to data poisoning than is the case for non-AutoML generated models. On the other hand, the usage of AutoML also has the potential of improving the robustness of a model by adapting the model to adversarial patterns. In this way, good accuracy can be maintained despite the attacker’s efforts to poison the data. However,no previous studies have investigated these effects. In this paper, we examine the risks associated with adversarial trajectory attacks in mobile systems, specifically looking into mobility prediction problems. By using mobility data from two different simulation frameworks: a simulator developed by Ericsson, which is based on a real-world deployment of Airtel’s open-network topology, and the ONE framework, we investigate three different AutoML frameworks and how the mobility accuracy for the frameworks is affected by a mobile trajectory attack. Our results show that re-running AutoML at every retraining is vulnerable to adversarial mobility poisoning and shows high accuracy variance. By contrast, using a single, well-chosen model from an initial AutoML search achieves more stable performance across adversarial conditions, even when the training set includes up to 10% adversarial mobility data.