Uncertainty-Aware Constrained Q-Learning for Online UAV-Based ILS Localizer Flight Inspection
Unmanned aerial vehicle (UAV) flight inspection requires online decisions that respond to changing measurement evidence while sustaining task progress. An uncertainty-aware framework maps heterogeneous observations to normalized quality indicators and uses temporal intuitionistic fuzzy sets to represent reliability support, unreliability support, and hesitation. Aggregated evidence enters the decision state, while post-action uncertainty defines a cost under an expected discounted cumulative-cost constraint. The resulting constrained Markov decision process is addressed using primal–dual constrained Q-learning (CMDP-Q). Measured Instrument Landing System (ILS) localizer profiles motivate retaining local variation, and controlled simulations examine its influence on learned decisions. Compared with unconstrained Q-learning, CMDP-Q selects Retest and Reposition more often and lowers mean step uncertainty, with longer episodes. Under the main evaluation setting, its mean discounted cumulative cost exceeds the budget and is slightly higher than that of GreedyProgress, without a statistically supported reduction relative to unconstrained Q-learning. A tested fixed-Lagrangian variant achieves higher reward and lower discounted cumulative cost than CMDP-Q, but higher mean step uncertainty. The state derived from temporal intuitionistic fuzzy evidence shows no statistically supported advantage over direct aggregation in reward, discounted cost, or step uncertainty. These findings characterize trade-offs between uncertainty exposure and execution burden under explicit cost regulation.
Authors
- Yuan Jiang (ORCID: https://orcid.org/0009-0003-5282-8715)
- Yu Zhang
- Yihao Shu
- Honghao Li
- Haoxiang Xu
Institutions
- Hohai University (CN)
- PLA 306 Hospital (CN)
- Tsinghua University (CN)
Publication Details
- Journal
- Drones
- Published
- 2026-11-16
- DOI
- https://doi.org/10.3390/drones10110860
- Primary Topic
- Reinforcement Learning in Robotics
- Type
- article
- Field-Weighted Citation Impact
- 0.00