Application of a Multi-Teacher Distillation Regression Model Based on Clustering Integration and Adaptive Weighting in Dam Deformation Prediction

Deformation is a key physical quantity that reflects the safety status of dams. Dam deformation is influenced by multiple factors and has seasonal and periodic patterns. Due to the challenges in accurately predicting dam deformation with traditional linear models, deep learning methods have been increasingly applied in recent years. In response to the problems such as an excessively long training time, too-high model complexity, and the limited generalization ability of a large number of complex hybrid models in the current research field, we propose an improved multi-teacher distillation network for regression tasks to improve the performance of the model. The multi-teacher network is constructed using a Transformer that considers global dependencies, while the student network is constructed using Temporal Convolutional Network (TCN). To improve distillation efficiency, we draw on the concept of clustering integration to reduce the number of teacher networks and propose a loss function for regression tasks. We incorporate an adaptive weight module into the loss function and assign more weight to the teachers with more accurate prediction results. Finally, knowledge information is formed based on the differences between the teacher networks and the student network. The model is applied to a concrete-faced rockfill dam located in Guizhou province, China, and the results demonstrate that, compared to other knowledge distillation methods, this approach exhibits higher accuracy and practicality.