Explainable Decision-Making: From Formal Logic to AI Systems with Explainable Behavior

This thesis makes the claim that logic-based frameworks can serve as an explainability layer atop AI systems, capable of generating rigorous and flexible explanations for human users across diverse problem domains. We support this claim through a progression of novel theoretical frameworks and practical implementations, starting with a general logic-based framework for generating explanations from the knowledge bases of an AI system and a human user and showing how it can be used on a diverse set of problem domains. We then systematically extend this framework with capabilities crucial for real-world applications: probabilistic reasoning for handling uncertainty, personalization through vocabulary-based abstraction, and dynamic interaction through argumentative dialogues. Building on these foundations, we address additional challenges by developing privacy-aware explanations for multi-agent systems and exploring explanation-guided approaches to belief revision that better align with human cognitive processes. To make our methods more accessible, we demonstrate how it can be effectively combined with large language models to generate natural language explanations while maintaining formal guarantees. Our theoretical contributions are complemented by efficient computational methods that make these frameworks more practical, as demonstrated through extensive evaluations across diverse problem domains. Recognizing that the ultimate test of explanatory frameworks lies in their effectiveness with real human users, we validate our approaches through several human-subject studies that show high comprehension of the explanations as well as high overall satisfaction with the explanation process, thus providing some evidence for the effectiveness of our approaches in enhancing human-AI interaction. By showing how logic can serve as a robust explainability layer that bridges the decision-making processes of AI systems and human understanding, this work aims to contribute to the development of AI systems that are not only powerful but also understandable, trustworthy, and above all, human-aware.