Formal Interpretation of RL Agent Strategies in Cybersecurity Tasks Based on Temporal and Epistemic Logic

The paper presents a formal approach to explainable reinforcement learning aimed at developing trusted intelligent systems in the field of cybersecurity. A mathematical model of an RL agent is proposed, which operates in an uncertain network environment and makes decisions on traffic inspection or blocking based on reward function optimization and partial observation of the network state. The subject of the research is the logical‑mathematical interpretation of RL agent strategies, which allows linking its behavior with formally verifiable properties of security, reachability, and causal sequence of actions. The method provides interpretation of the agent’s strategy using temporal logic, enabling formal verification of its compliance with specified security and reachability properties. The proposed mathematical model connects the decision‑making process regarding traffic blocking or inspection with causal structures and logical inference. This approach is aimed at creating trusted and auditable cyber‑physical systems whose behavior can be rigorously verified. The research methodology in this paper is based on the synthesis of machine learning techniques and formal logic. This approach enables a transition from empirical optimization to formal, verifiable behavior specifications, which constitutes its key innovation. The effectiveness of the methodology is confirmed by a computational experiment. The scientific novelty lies in the integration of reinforcement learning with logical models of LTL (Linear Temporal Logic) and EL (Epistemic Logic), which enables transition from empirically derived policies to formal specifications subject to automatic verification. It is shown that the combined use of temporal and epistemic logic allows describing the agent’s response to network anomalies both in the temporal dimension and from the perspective of the agent’s knowledge level under partial observability. The obtained results confirm that the logical‑mathematical interpretation of RL agent strategies is an effective tool for enhancing transparency, predictability, and trust in intelligent cybersecurity systems.