MFFTD: A Multiscale Feature Fusion Transformer Detector for Electricity Theft Based on Semi-Supervised Learning

The wide deployment of advanced metering infrastructure (AMI) in power systems allows utility companies to automatically and accurately collect and process the time-series load profiles of households, but meanwhile incurs the severe electricity theft (ET) that some illegal residential users may manipulate their electricity consumptions to reduce their billings. Although, due to the powerful ability of modeling long-range dependencies in sequential data, transformer has been widely used for time-series modeling including ET detection (ETD), the significant weakpoint lies in that it only considers the attention weights between either points or prepatched subsequences (i.e., patches) of fixed size within the input sequence, which cannot fully characterize the relationships among multiscale temporal patches and lead to suboptimal detection performance. To address the above issue, based on self-supervised feature extraction and supervised fine-tuning, our work proposes a novel multiscale feature fusion transformer encoder (TE)-based ETD framework, MFFTD. Specifically, our work’s contributions are following. First, in MFFTD, a hierarchical patching enhanced TE (HPTE) is explicitly designed, in which each layer patches the input sequence with variable patch size. Then, through hierarchically stacked multiple HPTE layers, the feature combining multiscale patches can be effectively extracted. Second, considering the constraint that only a small percentage of labeled theft samples are practically available, our work first pretrains the structure parameters of MFFTD through a self-supervised pretext task of forecasting the randomly masked segments in time series. Then, the small percentage of labeled anomalous samples is used to fine-tune the MFFTD model. Extensive experiments on multiple real datasets demonstrate our proposed MFFTD scheme outperforms the state-of-the-art (SOTA) transformer-based supervised and semi-supervised ETD methods.