A Hybrid 3D-2D Convolutional Neural Network with Spatial Attention for Enhanced Spectral-Spatial Classification of Hyperspectral Images

Hyperspectral image (HSI) classification using deep learning (DL) has proven to be highly effective. However, efficiently learning highly discriminative features from HSI is an open issue for the large dimensionality and complex spectral-spatial properties. The 2D Convolutional Neural Network (CNN) is an effective method for classifying Hyperspectral Images based on their spatial or spectral features. But HSI depends on both spectral and spatial classification. On the other hand, the 3D CNN is superior to the 2D CNN as it leverages both spectral and spatial features of the HSI. Nevertheless, in this case, computational complexity increases. This paper introduces a novel approach that utilizes a hybrid 3D-2D CNN structure with a spatial attention mechanism that specifically emphasizes spatial attributes. We utilize Z-score normalization followed by Incremental Principal Component Analysis (IPCA) for efficient dimensionality reduction and optimized memory usage on large datasets. Next, we use a spatial attention mechanism and a hybrid 3D-2D CNN to extract the features. In the hybrid CNN architecture, the 3D-2D branch merges the 3D CNN’s capability to capture detailed spectral-spatial features with the 2D CNN’s spatial refinement, further enhanced by a spatial attention mechanism. This approach balances comprehensive feature extraction with computational efficiency, improving accuracy and reducing overfitting. This results in great accuracy at a limited number of parameters. We have implemented the proposed hybrid CNN with attention mechanism on three hyperspectral datasets and it showcases exceptional performance in both accuracy and computational efficiency than other state-of-the-art methods.