Multi-Agent Reinforcement Learning for Anomaly Detection with Hierarchical Adaptive Sampling Strategy
The proliferation of sensor technologies has enabled real-time monitoring across diverse domains. In many practical systems, sensors are organized into geographically or functionally distinct regions, and continuous monitoring is constrained by limited sensing budgets. Such systems exhibit a hierarchical and resource-constrained data layout, where resources must be intelligently allocated across regions to capture the most informative data streams. We refer to this as the System-Region-Stream (SRS) structure. To support sequential anomaly detection in SRS systems under resource constraints, we formulate the monitoring task as a leader-follower decision-making problem, where a central planner allocates limited resources across regions, and regional detectors autonomously sample data streams for local monitoring. To optimize this hierarchical process, we propose a two-stage trust region learning framework that stabilizes policy updates and establishes monotonic-improvement properties for ideal trust-region updates of both planner and detectors. Building on this scheme, we develop HIPPO, a practical multi-agent deep reinforcement learning (DRL) algorithm that scales to large SRS systems by integrating neural function approximation with constrained action parameterizations for integer-valued resource allocation and budget-aware local sampling. Extensive simulation studies and real-world case studies demonstrate that our proposed method outperforms existing baselines significantly.
Authors
- Ziqian Zheng (ORCID: https://orcid.org/0000-0002-8135-8623)
- Kaibo Liu (ORCID: https://orcid.org/0000-0003-2863-5748)
- GUO Jie
- Xin Xia
- Chen Zhang
- Yujie Ma
Institutions
- University of Wisconsin–Madison (US)
- Tsinghua University (CN)
Publication Details
- Journal
- Technometrics
- Published
- 2026-11-13
- DOI
- https://doi.org/10.1080/00401706.2026.2763807
- Primary Topic
- Anomaly Detection Techniques and Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00