A Probabilistic Mechanism for Safe Reinforcement Learning

This paper introduces a modification of the Critic as A Lyapunov Function approach, which provides formal goal-reaching guarantees for reinforcement learning agents. We enhance the original method with a probabilistic acceptance mechanism inspired by simulated annealing, enabling temporary exploration of potentially unsafe actions while preserving the theoretical stability guarantees. The proposed framework offers a flexible rule-based mechanism to control the trade-off between safety and performance, creating a continuous spectrum from conservative to more aggressive exploration strategies. Theoretical analysis confirms that our approach maintains goal-reaching properties of the original method while significantly improving performance. Experimental evaluation demonstrates superior sample efficiency compared to conventional reinforcement learning methods, achieving near-optimal performance in a single episode without compromising safety. Our extensive validation on both pendulum stabilization and mobile robot navigation tasks reveals that our approach consistently outperforms baseline methods across various environmental conditions, making it particularly valuable for safety-critical applications where both performance optimization and formal guarantees are essential.