Körningsverifiering av DRL-baserade säkerhets- eller uppdragskritiska system

Deep Reinforcement Learning (DRL) is increasingly used to control autonomous robots that operate over unreliable wireless links and navigate cluttered indoor spaces. Although DRL policies can be trained to high average performance, they still fail in rare, unpredictable circumstances. This is a risk that traditional offline verification cannot mitigate once the agent is deployed. Ensuring runtime safety in such uncertain environments therefore remains an open research and engineering challenge. This work proposes a lightweight runtime verification layer that wraps an already-trained DRL controller. The monitor embeds a Kalman predictor that extrapolates the robot’s state several steps ahead while a rule engine checks temporal–logic safety properties on the trace. When a potential violation is detected, the monitor overrides the agent with a minimal corrective action. The design is intentionally model-agnostic so that it can run in the control loop of typical mobile robots. Evaluation in a network-aware simulator shows that the monitor cuts collision incidents from 49 % to 6 % across 100 diverse evaluation episodes and reduces missed warnings by 80 %, while adding minimal millisecond overhead of processing time per control cycle and <1 % resource overhead. The approach therefore achieves real-time assurance without sacrificing task throughput. The success rate rises from 51 % to 69 % because many episodes that would have crashed are recovered. This introduces episodes where it might not reach the goal but it does not collide either. The thesis delivers a modular implementation and results that practical, scalable runtime verification for DRL agents is achievable. Future developers can integrate the monitor into training loops, extend it to network Quality-Of-Service constraints, or port it to physical robots, confident that the safety framework will hold without heavy computational cost.

Institutions

Publication Details

Journal
KTH Publication Database DiVA (KTH Royal Institute of Technology)
Published
2025-01-01
Primary Topic
Formal Methods in Verification
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Körningsverifiering av DRL-baserade säkerhets- eller uppdragskritiska system

KTH Publication Database DiVA (KTH Royal Institute of Technology)
Formal Methods in Verification
article

Körningsverifiering av DRL-baserade säkerhets- eller uppdragskritiska system

article en

Abstract

Deep Reinforcement Learning (DRL) is increasingly used to control autonomous robots that operate over unreliable wireless links and navigate cluttered indoor spaces. Although DRL policies can be trained to high average performance, they still fail in rare, unpredictable circumstances. This is a risk that traditional offline verification cannot mitigate once the agent is deployed. Ensuring runtime safety in such uncertain environments therefore remains an open research and engineering challenge. This work proposes a lightweight runtime verification layer that wraps an already-trained DRL controller. The monitor embeds a Kalman predictor that extrapolates the robot’s state several steps ahead while a rule engine checks temporal–logic safety properties on the trace. When a potential violation is detected, the monitor overrides the agent with a minimal corrective action. The design is intentionally model-agnostic so that it can run in the control loop of typical mobile robots. Evaluation in a network-aware simulator shows that the monitor cuts collision incidents from 49 % to 6 % across 100 diverse evaluation episodes and reduces missed warnings by 80 %, while adding minimal millisecond overhead of processing time per control cycle and <1 % resource overhead. The approach therefore achieves real-time assurance without sacrificing task throughput. The success rate rises from 51 % to 69 % because many episodes that would have crashed are recovered. This introduces episodes where it might not reach the goal but it does not collide either. The thesis delivers a modular implementation and results that practical, scalable runtime verification for DRL agents is achievable. Future developers can integrate the monitor into training loops, extend it to network Quality-Of-Service constraints, or port it to physical robots, confident that the safety framework will hold without heavy computational cost.

KTH Publication Database DiVA (KTH Royal Institute of Technology)
Studi (SE), WSP Sverige (Sweden) (SE)
Openalex Percentile: Top 47%
Formal Methods in Verification
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Körningsverifiering av DRL-baserade säkerhets- eller uppdragskritiska system · KTH Publication Database DiVA (KTH Royal Institute of Technology) (2025) | TGRS Research Map | TGRS