Training robots with natural and lightweight human feedback

Abstract Generalist robot models promise broad applicability across domains but currently require extensive expert demonstrations for task specialization, which is a costly and impractical barrier for real‐world deployment. In this article, which summarizes the author's presentation in the New Faculty Highlights Track of the 39 th annual AAAI Conference on Artificial Intelligence, we present algorithms that enable non‐expert users to adapt and continually improve robot policies through natural and lightweight feedback modalities, such as preference comparisons, rankings, ratings, natural language, and users' own demonstrations, combining them with active learning strategies to maximize data‐efficiency. We further introduce methods for leveraging real‐time human interventions as rich training signals, modeling both their timing and absence to refine policies continually. Our approaches achieve substantial gains in sample‐efficiency, adaptability, and user‐friendliness, demonstrated across simulated and real‐world robotic tasks. By aligning robot learning with how humans naturally teach, we hope to move toward autonomous systems that are more personalized, capable, and deployable in everyday environments.