DA-Transformer: Building Damage Assessment Under Complex Changing Conditions Through Multimodal UAV Data

Accurate damage assessment is critical for reconnaissance and rescue missions. Damage assessment based on unmanned aerial vehicle images is a relevant topic. However, sharing pre- and post-damage knowledge is hindered by the lack of same-viewpoint images in damage assessment for building restoration and reconstruction work. We propose a novel method for generating images based on the same view before and after damage occurs. A point cloud registration method based on serialized matching is presented for spatial transformation with complex damage variations, achieving lower errors and reducing training time. Then, the transformed data are utilized to train a pre-damage differentiable rendering model, generating the undamaged images of buildings from post-damage drone views. Therefore, damage assessment using a large language model (LLM) is conducted by comparing pre- and post-damage images and point clouds. We also construct pre- and post-destruction datasets based on videos. Our method overcomes the limitation of obtaining paired image data from the same perspective, improving the efficiency of evaluation and reducing the difficulty and cost of data collection. We validated the effectiveness of the method through scaling experiments and real explosion scenarios.