Automated Virtual Earthquake Reconnaissance Reporting Using Natural Language Processing

Posthazard reconnaissance is essential following natural hazards, such as earthquakes, strike a community. Conducting reconnaissance efforts with efficiency and accuracy is necessary to expedite recovery and support decision-making processes. The Structural Extreme Events Reconnaissance (StEER) Network provides a natural hazards engineering (NHE) community-centered approach to accelerate the Data to Knowledge (D2K) life cycle and emphasize the impact of reconnaissance efforts. Within StEER, the virtual assessment structural teams (VASTs) summarize and disseminate the knowledge learned from investigated events by focusing on the available online data. Despite the well-designed workflow of VASTs, their efficiency is often hindered by the difficulty of tracing the rapidly updated information in practice, particularly in the initial phase of the reconnaissance effort. This paper leverages state-of-the-art natural language processing (NLP) techniques, including large language models (LLMs), to facilitate virtual reconnaissance. We propose a framework empowered by the NLP techniques to partially automate the workflow of VASTs through generating event briefings with rapidness, completeness, and accuracy. This practical approach of integrating the framework into the VAST workflow further distinguishes it from other automatic event reporting systems. Key steps of VAST workflow requiring repetitive and continuous manual labor are restructured into NLP tasks, with LMMs employed as auxiliary tools to process the visual data. The intention of autogenerated briefings is to serve as a starting point for VASTs and rapidly guide them toward key observations with consistency as demonstrated through two case studies.