Certificate-Driven Learning for Safety-Critical Systems: Reward-Free Policy Synthesis via Adversarial Certificate-Margin Optimization

Abstract Safety-critical learning systems require controllers that can be audited for constraint satisfaction under adverse operating conditions, not only policies that perform well on average. This paper proposes Certificate-Driven Learning (CeDL), a reward-free policy-synthesis framework that jointly learns a control policy and an accompanying mathematical certificate. CeDL replaces expected-return maximization with worst-case certificate-margin maximization over a declared admissible scenario domain. The certificate unifies Lyapunov stability margins, control-barrier safety margins, temporal-logic robustness, and feasibility margins into a single training objective. An adversarial falsifier searches for scenarios in which the current policy-certificate pair is closest to violation, and the resulting low-margin cases are retained in a hard-scenario bank for counterexample-guided optimization. A runtime safety shield is further introduced as a final feasibility-preserving mechanism when the nominal policy approaches the certified boundary. The framework is developed for general stochastic discrete-time systems and instantiated in three representative domains: URLLC-aware wireless resource allocation, energy-safe routing in wireless sensor networks, and robotic obstacle avoidance. Simulation results show that CeDL improves minimum certificate margins, reduces observed constraint violations, lowers URLLC tail delay, extends sensor-network lifetime, and increases robotic safety distance compared with reward-based reinforcement learning, constrained reinforcement learning, model predictive control, shielded reinforcement learning, and classical controllers. These results indicate that optimizing certifiability directly can provide a principled and practical path toward auditable learning-based control in safety-critical networked and cyber-physical systems.