UR-018 Towards Bounding the Behavior of Deep Neural Networks

Recent advances in Artificial Intelligence (AI) have unlocked many new possibilities but have also brought with it many new challenges. While modern AI systems have been continuously exceeding expectations, our ability to interpret and understand their behavior lags behind. For example, an AI model trained to detect pneumonia from X-rays may fail in new hospitals because it learned to recognize hospital logos instead of medical patterns. Why do some succeed while others fail? Do they truly understand their tasks, or are they relying on patterns that may not always hold? To enumerate the most informative explanations of a neuron’s behavior, we developed an improved approach to bounding the behavior of individual neurons within artificial neural networks. In this research we demonstrate, both theoretically and empirically, the utility of our approach.