Multi-Agent Reinforcement Learning for Anomaly Detection with Hierarchical Adaptive Sampling Strategy

The proliferation of sensor technologies has enabled real-time monitoring across diverse domains. In many practical systems, sensors are organized into geographically or functionally distinct regions, and continuous monitoring is constrained by limited sensing budgets. Such systems exhibit a hierarchical and resource-constrained data layout, where resources must be intelligently allocated across regions to capture the most informative data streams. We refer to this as the System-Region-Stream (SRS) structure. To support sequential anomaly detection in SRS systems under resource constraints, we formulate the monitoring task as a leader-follower decision-making problem, where a central planner allocates limited resources across regions, and regional detectors autonomously sample data streams for local monitoring. To optimize this hierarchical process, we propose a two-stage trust region learning framework that stabilizes policy updates and establishes monotonic-improvement properties for ideal trust-region updates of both planner and detectors. Building on this scheme, we develop HIPPO, a practical multi-agent deep reinforcement learning (DRL) algorithm that scales to large SRS systems by integrating neural function approximation with constrained action parameterizations for integer-valued resource allocation and budget-aware local sampling. Extensive simulation studies and real-world case studies demonstrate that our proposed method outperforms existing baselines significantly.

Authors

Institutions

Publication Details

Journal
Technometrics
Published
2026-11-13
DOI
https://doi.org/10.1080/00401706.2026.2763807
Primary Topic
Anomaly Detection Techniques and Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Multi-Agent Reinforcement Learning for Anomaly Detection with Hierarchical Adaptive Sampling Strategy

Ziqian Zheng, Kaibo Liu, GUO Jie, Xin Xia et al.
Technometrics
Anomaly Detection Techniques and Applications
article

Multi-Agent Reinforcement Learning for Anomaly Detection with Hierarchical Adaptive Sampling Strategy

Ziqian Zheng, Kaibo Liu, GUO Jie, Xin Xia, Chen Zhang, Yujie Ma
article en

Abstract

The proliferation of sensor technologies has enabled real-time monitoring across diverse domains. In many practical systems, sensors are organized into geographically or functionally distinct regions, and continuous monitoring is constrained by limited sensing budgets. Such systems exhibit a hierarchical and resource-constrained data layout, where resources must be intelligently allocated across regions to capture the most informative data streams. We refer to this as the System-Region-Stream (SRS) structure. To support sequential anomaly detection in SRS systems under resource constraints, we formulate the monitoring task as a leader-follower decision-making problem, where a central planner allocates limited resources across regions, and regional detectors autonomously sample data streams for local monitoring. To optimize this hierarchical process, we propose a two-stage trust region learning framework that stabilizes policy updates and establishes monotonic-improvement properties for ideal trust-region updates of both planner and detectors. Building on this scheme, we develop HIPPO, a practical multi-agent deep reinforcement learning (DRL) algorithm that scales to large SRS systems by integrating neural function approximation with constrained action parameterizations for integer-valued resource allocation and budget-aware local sampling. Extensive simulation studies and real-world case studies demonstrate that our proposed method outperforms existing baselines significantly.

Technometrics
University of Wisconsin–Madison (US), Tsinghua University (CN)
Anomaly Detection Techniques and Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon