Risk-sensitive safe reinforcement learning for joint cyber insurance and cybersecurity investment planning in cyber-physical power systems

Abstract Cyber insurance and cybersecurity investment are tightly coupled in smart grids because cyberattacks can produce physical outages, financial losses, regulatory exposure, and insurance claims through the same operating event. This study develops a risk-sensitive safe reinforcement learning framework for joint cyber insurance and defense investment planning in cyber-physical power systems. The formulation treats the utility as a sequential decision maker that observes physical grid states, cyber exposure, insurance contract status, posterior threat beliefs, and budget conditions, and then selects coverage, deductible, hardening, monitoring, redundancy, backup-control, and incident-response actions. The framework combines a constrained Markov decision process, conditional value-at-risk loss control, an optimal power flow (OPF) safety shield, and a Benders-style decomposition between long-term policy learning and short-term security correction. The mathematical presentation separates model-specific cyber-physical-insurance equations from standard reinforcement-learning components, so that the novel coupling among insurance, defense, physical feasibility, and tail-risk control remains explicit. A simulation benchmark on IEEE test systems evaluates expected cost, 95% CVaR, physical-security violations, shield corrections, policy sensitivity, and belief-update degradation. Under the extreme-threat case, the safe-RL policy reduces 95% CVaR from 130.63 MUSD under the no-insurance stress case to 68.21 MUSD, lowers total cost from 36.8 MUSD to 17.1 MUSD, and reduces total security violations to four events per 1000 evaluation episodes. Across five random seeds, the total-cost and tail-risk results remain stable, with a mean total cost of 17.16 MUSD and a mean extreme-threat CVaR of 68.34 MUSD. The results show that cyber insurance is most effective when premiums are priced from verified defense posture and coordinated with physical safety constraints rather than treated as a substitute for cybersecurity investment.