Comparison of Explainability Approaches for Reinforcement Learning for Autonomous Flight Maneuvers

Autonomous drone movement is an important problem area that has various real-world applications including surveillance and flight combat scenarios. Reinforcement learning (RL) based approaches can be an obvious choice to solve this problem. However, a lack of explainability to these mission-critical scenarios can bring catastrophic mistakes. For instance, a regular deep Q Network (DQN) based technique is not efficient enough at explaining the agent's behavior after it learns the optimal policy. Therefore, we propose a reward decomposed DQN (rdDQN) framework that is efficient at providing minimal sufficient explanations (MSE) of an aircraft agent's maneuvers when the agent is trained in a simulated flight combat environment. Furthermore, the RL-trained flight agent is expected to learn flight maneuvering strategies beyond the common training tactics. Our explainable RL (XRL) algorithm will be able to answer ‘actual’ and ‘counterfactual’ questions on the flight agent's behavior and the knowledge can be shared with the human counterpart. Explainability is a key aspect of trustworthy AI and our effort will aid in the development of trustworthy autonomous solutions.