Toward accurate and scalable rainfall estimation using surveillance camera data and a hybrid deep-learning framework

Rainfall measurement at high quality and spatiotemporal resolution is essential for urban hydrological modeling and effective stormwater management. However, traditional rainfall measurement methods face limitations regarding spatial coverage, temporal resolution, and data accessibility, particularly in urban settings. Here, we show a novel rainfall estimation framework that leverages surveillance cameras to enhance estimation accuracy and spatiotemporal data coverage. Our hybrid approach consists of two complementary modules: the first employs an image-quality signature technique to detect rain streaks from video frames and selects optimal regions of interest (ROIs). The second module integrates depthwise separable convolution (DSC) layers with gated recurrent units (GRU) in a regression model to accurately estimate rainfall intensity using these ROIs. We evaluate the framework using video data from two locations with distinct rainfall patterns and environmental conditions. The DSC-GRU model achieves high predictive performance, with coefficient of determination (R²) values ranging from 0.89 to 0.93 when validated against rain gauge measurements. Remarkably, the model maintains strong performance during daytime and nighttime conditions, outperforming existing video-based rainfall estimation methods and demonstrating robust adaptability across variable environmental scenarios. The model's lightweight architecture facilitates efficient training and deployment, enabling practical real-time urban rainfall monitoring. This work represents a substantial advancement in rainfall estimation technology, significantly reducing estimation errors and expanding measurement coverage, and provides a practical, low-cost solution for urban hydrological monitoring.