Survival-Critical Machine Learning

Autonomous systems must be able to survive in adversarial or hostile environments where threats evolve and morph. Under conditions in which a class of adversarial agents is novel but rare, these systems must rapidly learn and adapt. We introduce Survival-Critical Machine Learning (SCML), a new ML paradigm that defines how autonomous systems that rely on machine learning can negotiate such adversarial environments. Inspired by the ability of a biological entity's immune system to develop defenses against new viruses, SCML systems leverage the workflow of Live Learning to iteratively improve ML models for threat detection. Beyond the conceptualization of SCML, the main contributions of this dissertation are an analytical model, a prototype implementation, and experimental results of the SCML design tradeoff space. We evaluate the impact on survivability of the various design parameters and demonstrate the intimate relationship between SCML and Live Learning. Notably, we evaluate the impact of the availability of finite countermeasures (CMs), the CM deployment threshold, the number of deployed systems, and the average threat arrival rate, among others, on the probability of survival of a given mission duration. We also examine various mission success criteria and the effects of divergent performance metrics between SCML and Live Learning. Additionally, we model SCML as a Markov Decision Process (MDP) to demonstrate how it can be analyzed within existing, well-understood ML frameworks such as MDPs and Reinforcement Learning (RL). We also demonstrate Live Learning's extensibility to domains other than visual data, such as short range radar, critical in many environments where SCML systems will need to operate. Our experimental results confirm that learning can indeed improve survivability in an SCML system. It further shows that the CM deployment threshold and the number of available CMs have a significant impact on survivability. Allowing flexibility in the CM deployment threshold during the mission enhances such survivability under most conditions. Similarly, Live Learning improves the probability of mission success by increasing the likelihood of accurately classifying actual threats (true positives) and decreasing the likelihood of wasting CMs on non-threats (false positives). By defining an SCML MDP, we also show how an SCML system can optimally adjust its CM deployment threshold as a function of state, defined by the number of remaining CMs and the time until mission completion.