BatteryGPT: Battery Anomaly Detection Based on Multimodal Large Language Model

In the context of pursuing carbon neutrality goals, the importance of battery energy storage systems is becoming increasingly prominent. Battery defect detection has become critical to ensuring the safety and enhancing the efficiency of these storage systems. Current detection methods face challenges such as limited sample availability, insufficient recognition accuracy, low sensitivity to defect details, and a lack of domain-specific knowledge. To address these issues, this paper proposes a lithium-ion battery defect detection technology based on the multimodal large language model, BatteryGPT. BatteryGPT integrates a pre-trained image encoder, a multimodal text encoder, and a large language model, utilizing simulated anomaly data to locate and describe anomalies in battery images while supporting multi-turn conversations. By aligning the features of the visual and text encoders, the system derives localization results and fine-tunes the input encoding of the large language model. This approach enables the detection of unseen anomalies in batteries with only a small number of normal samples. BatteryGPT offers a more efficient solution for battery anomaly detection, with the potential to further improve the research and production efficiency of electrochemical energy storage systems.