Real-time rainfall estimation using deep learning: Influence of background and rainfall intensity

Traditional methods fall short in providing the necessary spatiotemporal resolutions for real-time urban flood forecasting, advancements suggest that surveillance camera image-based techniques are promising alternatives. However, most studies focus on static images, overlooking the impact of moving objects. To address this gap, we introduce the ResNet (Residual Network)-LSTM (Long Short-Term Memory) and compared its performance against two common models, CNN (Convolutional Neural Network) and CNN-LSTM. We used YOLO (You Only Look Once) to classify videos by the proportion of moving objects and rainfall intensity to validate the models across different scenarios and analyze the impact of these factors. Our results show that ResNet-LSTM performs reliably in complex, dynamic backgrounds and significantly outperforms CNN and CNN-LSTM across all evaluated scenarios using the four metrics. This approach allows for high-precision rainfall intensity data collection using surveillance cameras, without additional sensors, providing a cost-effective solution for real-time, high-resolution rainfall data crucial for flood prevention and control. • ResNet-LSTM model accurately estimates rainfall intensity from traffic surveillance camera images. • Resnet-LSTM, CNN-LSTM, and CNN show degraded performance with increased moving objects in the video frame. • Moving objects most affect rainfall intensity estimates at medium rainfall intensities (12–35.9 mm/h). • At low rainfall (0–12 mm/h), video noice can impair estimates due to limitations of the filming equipment.