A Metacognitive Evaluation Framework for Embodied Intelligent Agents

The study introduces a novel evaluation system designed to measure the metacognitive abilities of embodied agents. The system incorporates multiple metricsincluding task success rate, self-monitoring accuracy (measured by AUC), error detection speed, and confidence calibration errorto provide a comprehensive assessment of an agents internal monitoring and self-regulatory processes. Experiments were conducted in simulated environments (using Meta-World, etc.) and on a real robotic platform performing target grasping tasks. Two types of agents were compared: baseline agents relying solely on external feedback and agents enhanced with integrated metacognitive modules. The results demonstrate that agents with metacognitive capabilities consistently achieve higher performance, exhibit more precise self-monitoring, and respond more swiftly to unexpected events. This evaluation system serves as a robust tool for assessing metacognitive functions and offers promising implications for the development of more adaptable and reliable autonomous systems in dynamic environments, thus significantly enhancing overall system performance continuously.