Real-time dynamic monitoring of rockfalls with PTZ cameras

Rockfall is a prevalent geological hazard threatening lives and infrastructure. Beyond static assessment, real-time dynamic monitoring is crucial to capture sudden events, ensuring timely warning and risk mitigation. Leveraging low cost, flexible deployment, and high spatiotemporal resolution, PTZ (Pan-Tilt-Zoom) cameras combined with deep learning provide a potential solution. However, the small size, background similarity, rapid and complex motion of rockfall objects present significant challenges for reliable monitoring. To tackle these issues, this paper proposes a rockfall detection and tracking framework integrating motion awareness and physical constraints. First, a lightweight temporal module is incorporated into YOLOv8 to enhance perception of small and dim moving objects. Second, a novel matching strategy based on Euclidean distance and motion direction replaces the traditional IoU in ByteTrack to improve tracking stability. Trajectory fragmentation caused by missed detections is mitigated via a frame-difference-based completion mechanism. Adaptability to unseen environments is further bolstered through a generalization training strategy. A custom dataset combining real-world surveillance and simulated rockfalls was constructed. Experiments demonstrate superior detection, robust tracking, and strong generalization under complex backgrounds and limited frame rates. Deployment on NVIDIA Orin edge platforms facilitates real-time monitoring and supports practical PTZ-based early warning for geological hazards.