A Keypoint-Driven Robust Pointer Meter Reading Method With Enhanced Structure for Complex Environments

With the rise in popularity of intelligent systems, deep learning-based methods for meter reading recognition are gradually replacing traditional image processing methods. However, most of the meter reading methods currently available in the market have limitations. Existing methods overlook the interference of complex environments with data acquisition devices (e.g., inspection robots and fixed-point cameras), such as light changes, weather effects, and meter tilting. In addition, most of the reading methods for pointer meters rely on accurate segmentation of scales and pointers, which can be challenging in practical situations. Therefore, this study proposes a complete pipeline to address the above problems. First, our approach focuses on completing the reading by simply identifying the four keypoints of a pointer meter. We propose a keypoint-structure-enhanced CenterNet (KSE-CenterNet) that can ensure accurate reading and identification of the meter even in complex situations. Specifically, the KSE-CenterNet is a multihead network consisting of a keypoint structure attention (KSA) module and an attention fusion module (AFM). The KSA module allows the network to enhance the spatial structure of keypoints during feature extraction by adaptively learning the spatial relationships of localized regions, while the AFM module is utilized to enhance the semantic information from the deeper layers and adaptively adjust the features during feature fusion. In addition, we have developed a keypoint reading algorithm (KRA) to correct the tilted and rotated meters by utilizing perspective transformation algorithms that identify the keypoints. Experimental results show that our proposed method achieves state-of-the-art performance with an average relative error of only 1.32%. Since we do not need the complete semantic information of the meter, we are able to perform meter readings well in various complex situations.