NES-DM: Representative Attention-based CNN Network for Medical Image Classification

Various lesions in different body parts have different sizes and, in particular, different representations, which leads to great challenges for medical image classification tasks. To avoid incorrect classification, it is critical to design a model that has the ability to extract descriptive and discriminatory features. The attention mechanism is a method that can utilize layer weight adjustment to enhance feature extraction capability for various deep learning models. In this study, an attention-based CNN framework, NES-DM, is proposed to adapt to different lesions. Specifically, for the channel attention, we propose to leverage a neuron energy separability module, which aims to aggregate more representative features in a channel; for the spatial attention, we design to decompose the feature map along with two dimensions and then input them into a multiple convolution module, respectively, which aims to reduce the feature redundancy by the decomposition and capture long-range dependencies of features by the multiple convolution. The effectiveness of the proposed NES-DM based CNN framework is evaluated on three medical image datasets and NES-DM achieves at most 7.8% improvement on sensitivity compared to baselines.