Systems Engineering With Architecture Modeling, Formal Verification, and Human Interactions for Learning‐Enabled Autonomous Agent

ABSTRACT With the rapid advancements in Artificial Intelligence (AI), autonomous agents are expected to handle increasingly complex situations. However, learning‐enabled algorithms, which are critical to these systems, present significant challenges, including complexity, difficulty in verification, and a lack of certification pathways. A systematic approach integrating architectural analysis with human–machine interaction is crucial to ensuring the safety of these systems. This research emphasizes the early incorporation of human interactions in the design of architectural models to meet safety requirements. These interactions are modeled in the Soar cognitive architecture, which combines symbolic decision logic and numeric decision preferences, refined by reinforcement learning. The agent is then automatically translated into the formal verification environment, nuXmv, where its properties are verified. Our framework integrates systems modeling, formal verification, and simulation to check operational correctness, enhancing the reliability and safety of learning‐enabled autonomous agents.