Research on Construction Site Safety Q&A System Based on BERT

Abstract This paper aims to utilize the pre-trained language model BERT from deep learning to construct A question and answer system specifically targeting safety knowledge in construction sites, thereby enhancing safety management on-site and increasing workers’ awareness of safety issues. Through extensive reading of literature related to construction site safety and the integration of practical case studies, this research compares various pre-trained language models such as word2vec, Pre-trained RNN, GPT, and BERT, analyzing their respective advantages and disadvantages. Despite the fact that word embedding methods such as word2vec have improved the effectiveness of natural language processing to some extent, their ability to understand context is limited. Pre-trained RNNs, although capable of handling sequential data, suffer from the problem of gradient disappearance when dealing with long-range dependencies. In contrast, the GPT model performs well in generative tasks; however, due to its reliance on a unidirectional language model, it falls short in understanding bidirectional contexts. Ultimately, it was determined that a method based on BERT would be most suitable for improving the model to meet the safety needs of construction sites. The system can accurately understand and respond to safety-related questions posed by workers, thereby preventing accidents and ensuring the safety of construction site personnel. This study not only explores the optimization and adjustment of the BERT model but also evaluates its performance in practical application scenarios, providing new technological means for safety education and management within the construction industry.