Segmentation Aided Multiclass Classification of Lung Disease in Chest X-ray Images using Graph Neural Networks

Deep learning (DL)-based medical image classification has become a pivotal research area in computer vision, significantly enhancing the diagnostic process across the medical field. DL-based image classification can be aided by supplementary data, further improving its performance. One prominent example of such data is gaze-points, which involve eye-tracking techniques to document radiologists’ interaction with images. Recently, gaze-point data has been used as a pre-processing method to map an image into a graph, enabling the use of graph neural networks (GNNs) in computer vision domain. Such techniques have demonstrated significant improvements compared to conventional convolution neural networks (CNNs); however, their reliance on human involvement makes them labor-intensive, thus limiting their applications. To address this, we propose to leverage DL-based automatic segmentation mask generation to prepare image data as input for the GNN. This technique uses the segmentation mask as the attention information to guide the classifier. The results demonstrate that the proposed segmentation-aided classification model surpasses conventional CNN models and delivers the same performance as the existing supplementary data-aided techniques while reducing manual labor.