Metacognitive Reliability and Calibration in Synthetic Agents: An Experimental Study

Reliability and calibration are fundamental properties of intelligent systems, determining whether internal confidence estimates align with external performance. While calibration has been studied extensively in machine learning, much less is known about how synthetic agents form and act upon metacognitive beliefs about their reliability. In this work, we investigate metacognitive reliability and calibration in synthetic agents across eight controlled conditions, including varying complexity, ablations, and adversarial structures. We introduce metrics such as reflection reliability, confabulation rate, calibration error, and policy alignment to quantify how self-beliefs correspond to observed outcomes. Our findings show that agents maintain moderately accurate self-models under normal conditions but become systematically overconfident under stress, deception, and high-entropy perturbations. Communication reliability degrades under misaligned incentives, though auditing partially restores calibration. Adaptive methods reduce long-term bias but trade off short-term accuracy. Together, these results provide a framework for assessing metacognitive reliability in synthetic agents and highlight its importance as a foundation for trustworthy, self-assessing AI systems.