Dual-stream background modeling network with anomaly suppression for hyperspectral anomaly detection

Autoencoder (AE)-based hyperspectral anomaly detection aims to model pure background so that anomalies can be identified via reconstruction errors. However, (1) due to the generalization of AEs, some anomalies may be reconstructed well, resulting in missed detections, and (2) conversely, certain complex backgrounds may be poorly reconstructed, thereby increasing the false alarm rate. Towards this end, a novel dual-stream background modeling network (DSBMNet) with anomaly suppression is devised to suppress anomaly reconstruction while enhancing background reconstruction. Specifically, for the first challenge, a multi-directional masked convolution (MDMC) module is designed to destroy the spatial correlation of anomalies along horizontal, vertical, diagonal, and center-block directions, ultimately suppressing anomaly reconstruction. For the second challenge, a dual-stream network with local and global branches is developed to enhance background reconstruction. Specifically, in the local branch, an attention-driven multi-scale grouped convolution (AMGC) module is designed by combining multi-scale grouped convolution with the designed coordinate-aware spatial attention (CSA) module to accurately model fine-grained background features. In the global branch, a difference-enhanced spatial–spectral attention (DSSA) module is developed, which integrates the designed spatial difference adaptive attention (SpaDAA) module and spectral attention (SpeA) module to jointly optimize spatial–spectral global background features. Additionally, SpaDAA also assists reduce attention to potential anomalies by leveraging difference attention scores. Ablation studies confirm the effectiveness of MDMC, AMGC, and DSSA, while comparison experiments on six hyperspectral datasets demonstrate the superior performance of DSBMNet. • A multi-directional masked convolution module is designed to suppress anomaly reconstruction by destroying spatial correlations of anomalies. • An attention-driven multi-scale grouped convolution module is proposed to model local background features by integrating coordinate-aware spatial attention. • A difference-enhanced spatial–spectral attention module is developed to optimize global background features and reduce attention to anomalies.