I nternet of things and artificial intelligence‐based structural health monitoring and damage evaluation of concrete dams using vibration and vision analysis

Abstract The safety and durability of concrete gravity dams play an important role in providing the necessary resilience of water infrastructure systems. In order to provide reliable service of structure health monitoring (SHM) without and disruptions, we present a real‐time multi‐modal SHM system based on the application of internet of things sensing techniques, edge‐computing and artificial intelligence (AI)‐based algorithms for visual analytics in this research. Using a 1:56 laboratory‐scale prototype of a concrete gravity dam built according to Cauchy similitude ( S f = 56) to maintain geometric, inertial, and elastic similarities, we demonstrate how vibration analysis can be conducted through Fast Fourier Transform‐based modal analyses. Moreover, our in‐house lightweight convolutional neural network (TinyDeepCrack) conducts pixel‐level crack segmentations along with their width quantifications by overcoming traditional threshold‐based approaches. Unlike conventional methods of image processing that rely solely on thresholds, our novel AI approach allows accurate semantic segmentations regardless of varying surface conditions, showing promising precision metrics of 0.959 precision value with recall value up to 0.946 and F1‐score 0.952. Conclusive validation through confusion matrix, receiver operating characteristic‐curve, precision‐recall‐curve, and comparative analysis proves that TinyDeepCrack confirms the superior segmentation over the conventional U‐Net architecture in accuracy, classification capability, and overall computational performance. Our entire analytical procedure has been coded using open‐source Python libraries (e.g., SciPy and NumPy). Additionally, we propose an analytic hierarchy process‐based multi‐sensor fusion scheme which seamlessly combines all collected data types including those related to vibrations' degradation trends, cracks' severities, fatigue indicators, etc. Finally, after performing experimental validations, we found that the natural frequencies decreased by around ~10%, corresponding to near‐stiffness degradation by ~19%. Crack widths reaching as large as 1.2 mm cause our model's remaining useful lifetime estimation (~predicted remaining useful life) values close to 9.2 ± 1 years accounting uncertainties due to sensor noise as well as geometric scaling factors.