Building Teachable AI from Explanations, Interactions, and Demonstrations

Recent advancements in AI have led to development of powerful AI assistants that can tackle a plethora of tasks ranging from simple queries like asking about weather, to complex queries that seek help with programming and creative writing. However, the efficacy of these AI systems suffers in certain domains/tasks due to lack of abundant high quality annotated data. Thus, it is important to develop frameworks that can leverage different forms of supervision under limited data scenarios and make the most use of the available annotations. Motivated by how humans learn, I present frameworks to teach AI systems by extracting helpful supervision signals from explanations, interactions, and demonstrations. First, natural language explanations are an efficient medium to convey concepts (by stating how different attributes map to a class) and can effectively model fuzziness of concept by using quantifiers (e.g. `likely’). I explore learning generalizable language-guided classifiers by presenting (1) a large-scale resource of classification tasks paired with language explanations and (2) techniques that encompass multi-task learning, curriculum learning, and data-driven learning of quantifier semantics. Additionally, I explore combining explanation supervision from multiple experts and effects of adversarial explanations when learning such classifiers. Second, interactions between AI system and an expert can enirch the AI system's understanding of the task. This is akin to learning in humans, where we typically improve our understanding of a task by interacting with others, like experts and peers, to reduce uncertainties thus improving our task performance. I present a framework that leverages interactions between an AI system (powered by a language model) and an expert, where the AI system actively asks for clarifications in forms of labels and explanations to improve its performance. Third, demonstrations for sequential decision making tasks can capture several desirable characteristics in the outputs (typically a sequence of actions/commands) in addition to serving just as "labels" for the task. Extracting helpful inductive biases from demonstrations that emulate these desirable characteristics can lead to improved performance, especially in limited data scenarios. I present a framework that formulates several helpful inductive biases and learns their importances in a data-driven manner. This thesis takes a step towards developing intuitive and simple frameworks that can produce robust generalizable AI systems by leveraging different sources of supervision.