Detecting changes and anomalies in nonstationary contextual bandits with an application to task categorisation

As society becomes increasingly connected, the demands placed on telecommunications systems will only grow. To meet these demands network providers want to deploy automated tools that make decisions based on available network information. Furthermore, there is a need for these tools to be agile, so that they can react to changes, or identify unexpected outcomes, as they occur in this rapidly evolving digital landscape. To address this challenge the first nonstationary contextual bandit method that simultaneously monitors the observed rewards for both changes and anomalies, SCAPA-UCB, is introduced. In addition to incorporating change and anomaly detection, the proposed approach relaxes common nonstationary bandit assumptions on the reward distribution for an arm, allowing contextual information to be incorporated using a broad range of statistical models. Furthermore, the method provides a faster retraining process once a change is detected. Extensive simulation studies are performed to establish the favourable performance of SCAPA-UCB, and an application categorising maintenance tasks on a telecommunications network is presented. • A novel nonstationary contextual bandit algorithm, SCAPA-UCB, is proposed. • The first contextual bandit method to detect changes and anomalies in rewards. • Relaxes common contextual reward assumptions such as linearity or Gaussian noise. • Simulations show excellent performance in a range of nonstationary settings. • Presents an application categorising telecoms network maintenance tasks.