SpectralX: Efficient Global-local Representation via Fourier Gating and Prototype Refinement for Weakly Supervised Thorax Disease Classification

The task of diagnosing diseases related to the Thoracic region from Chest X-rays (CXRs) is a challenging multi-label classification task that faces issues such as class imbalance, label noise, and the need for global context in images to diagnose diseases such as Emphysema and Infiltration. The current State-of-the-Art (SOTA) methods in the field involve the use of Vision Transformers (ViTs) or Convolutional Neural Networks (CNNs) which are computationally expensive and are likely to overfit on noisy labels. We propose a novel computationally efficient model Named as a SpectralX that is well-suited for deployment on hardware with limited VRAM capacity ( \(<12\) GB). The key contributions of our model are: (1) a Spectral Gating Unit (SGU) that utilizes a linear complexity Fast Fourier Transform (FFT) to replace quadratic complexity self-attention motivated by the latest advances in token mixing operations; and (2) a Prototype Refinement Head (PRH) that utilizes centroids to improve label noise robustness, a technique that has been shown to be effective in the context of weakly supervised classification in medical image classification, especially in the presence of a long-tailed distribution. We present comprehensive ablation experiments that isolate the contribution of each proposed module and compare against recent state-of-the-art methods published between 2022 and 2025. Our experiments on the challenging NIH ChestX-ray14 dataset reveal that our model achieves a leading performance with an average AUC score of 0.842 under per-class optimized thresholds while being computationally efficient and employing a backbone network such as EfficientNet-B0. We further validate SpectralX on the CheXpert dataset to demonstrate cross-dataset generalization. This work highlights a promising direction for developing efficient yet powerful models for complex medical imaging tasks.