Körningsverifiering av DRL-baserade säkerhets- eller uppdragskritiska system
Deep Reinforcement Learning (DRL) is increasingly used to control autonomous robots that operate over unreliable wireless links and navigate cluttered indoor spaces. Although DRL policies can be trained to high average performance, they still fail in rare, unpredictable circumstances. This is a risk that traditional offline verification cannot mitigate once the agent is deployed. Ensuring runtime safety in such uncertain environments therefore remains an open research and engineering challenge. This work proposes a lightweight runtime verification layer that wraps an already-trained DRL controller. The monitor embeds a Kalman predictor that extrapolates the robot’s state several steps ahead while a rule engine checks temporal–logic safety properties on the trace. When a potential violation is detected, the monitor overrides the agent with a minimal corrective action. The design is intentionally model-agnostic so that it can run in the control loop of typical mobile robots. Evaluation in a network-aware simulator shows that the monitor cuts collision incidents from 49 % to 6 % across 100 diverse evaluation episodes and reduces missed warnings by 80 %, while adding minimal millisecond overhead of processing time per control cycle and <1 % resource overhead. The approach therefore achieves real-time assurance without sacrificing task throughput. The success rate rises from 51 % to 69 % because many episodes that would have crashed are recovered. This introduces episodes where it might not reach the goal but it does not collide either. The thesis delivers a modular implementation and results that practical, scalable runtime verification for DRL agents is achievable. Future developers can integrate the monitor into training loops, extend it to network Quality-Of-Service constraints, or port it to physical robots, confident that the safety framework will hold without heavy computational cost.
Institutions
- Studi (SE)
- WSP Sverige (Sweden) (SE)
Publication Details
- Journal
- KTH Publication Database DiVA (KTH Royal Institute of Technology)
- Published
- 2025-01-01
- Primary Topic
- Formal Methods in Verification
- Type
- article
- Field-Weighted Citation Impact
- 0.00