Automatic analysis and anomaly detection technology of intelligent operation and maintenance log integrated with natural language processing
The traditional manual log analysis approach is increasingly inadequate for meeting the demands of efficient and accurate operation and maintenance. To address this challenge, this paper proposes an intelligent log analysis framework capable of automatically processing large-scale log data, accurately identifying anomalies, and providing timely and effective decision-making support for operation and maintenance teams. The proposed system architecture comprises five core modules: log acquisition, data preprocessing, natural language processing (NLP)-based feature extraction, anomaly detection, and result visualization and alerting. By leveraging NLP techniques, the system can deeply interpret and extract valuable information from log texts, construct log template vectors, and support precise anomaly detection. For anomaly detection, this study employs deep learning models, specifically long short-term memory (LSTM) networks, to automatically learn complex patterns in log data and accurately identify anomalous events. Experimental results demonstrate the superior performance of the LSTM model in log classification tasks, achieving an accuracy of 97.5%, with precision, recall, and F1 scores of 80.0%, 75.0%, and 77.4%, respectively, significantly outperforming traditional statistical anomaly detection methods. This research highlights the effectiveness of integrating NLP and LSTM-based methods for intelligent automated log analysis and anomaly detection. The proposed approach not only enhances operational efficiency but also improves system stability, offering a robust solution for modern operation and maintenance challenges.
Authors
- Zepeng Wen
- Shuai Xu (ORCID: https://orcid.org/0000-0002-1393-3702)
- Han Wang (ORCID: https://orcid.org/0000-0001-5448-9903)
- Yakun Zhang (ORCID: https://orcid.org/0000-0002-9636-8576)
- Ziyang Meng (ORCID: https://orcid.org/0009-0005-4750-4155)
Institutions
- China Academy of Engineering Physics (CN)
Publication Details
- Published
- 2025-05-09
- DOI
- https://doi.org/10.1117/12.3067399
- Primary Topic
- Advanced Data Processing Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00