Novel framework for automated testing of ill-defined human–robot interaction environments

As automated systems advance in complexity, comprehensive testing becomes crucial, particularly for human–robot interaction (HRI) environments where human unpredictability creates ill-defined testing domains that challenge conventional software testing approaches. In interactive robotics, evaluation criteria extend beyond performance to include critical safety considerations. This paper introduces a novel automated testing framework combining runtime monitoring with constraint-based techniques for HRI environments. The framework employs a three-level cognitive oracle architecture – observation, interpretation, and diagnosis – that automatically evaluates the correctness of human and robot actions without requiring expert human intervention. The approach uses constraint-based modeling to handle the non-deterministic nature of HRI scenarios while ensuring safety compliance. Validation through five test cases in a refrigerator disassembly simulation demonstrates the framework’s effectiveness in detecting safety violations and procedural errors under environmental uncertainties. • AI framework for automated human-robot interaction testing without supervision. • Three-level cognitive framework for diagnosing human-robot interaction environments. • Constraint-based interpretation and diagnosis handling uncertain behavior in HRI. • Framework avoids state explosion problems vs. traditional model checking. • Successfully validated on simulated human-robot interaction disassembly tasks.