FishermaskFormer: Lightweight Remote Sensing Scene Classification With Masked Transformer

Remote sensing scene classification (RSSC) is to accurately assign semantic labels to remote sensing images by analyzing scene contents. Recently, many algorithms have made significant progress in improving the classification accuracy of RSSC. However, a large number of parameters and floating point operations are needed to achieve that end in those approaches, resulting in high complexity. To address the issue, we propose a novel RSSC algorithm, dubbed FishermaskFormer, which aggressively decimates features in the convolutional backbone via a novel masking operation with a proposed Fisher Discriminant Analysis Criterion, and then designs a lightweight transformer block to drive the classification loss. This is aimed at offering a flexible and effective framework for preserving classification accuracy while significantly reducing the complexity. The proposed transformer design employs a new grouping index that assigns multi-headed transformer groups by maximizing the information interactions in each group. Compared with leading lightweight RSSC methods, experimental results show this proposed framework achieves higher classification accuracy while having the similar low complexity.