DenLsNet-C: a novel model for breast cancer classification in pathology images based on DenseNet and LSTM

Abstract In the contemporary world, breast cancer is a common malignancy, whose early detection and timely treatment can increase the patients’ survival prospects. The automated classification of breast cancer types based on histopathological images is a challenging endeavor, wherein computer-assisted diagnosis serves as a reference for pathologists’ decision-making. Addressing the automated breast cancer classification task, this paper proposes a novel DenLsNet neural network model, featuring a combined DenseNet−LSTM architecture for efficient feature extraction and classification. First, the feature extraction process is optimized by incorporating squeeze-and-excitation (SE) blocks into a pretrained improved dense convolutional network (DenseNet). Next, iterative convolutional feature fusion (iCFF) blocks are introduced for deep and shallow feature fusion. To enhance the classification performance, the original DenseNet classifier is replaced with a specially designed long short-term memory (LSTM)-based classifier, which proves effective in capturing long-distance relationships in image sequences, improving the model’s sensitivity to breast cancer variations. Performance evaluation experiments, conducted on the BreakHis and BACH public datasets, demonstrate significant performance enhancement in the multi-class classification task, with DenLsNet exhibiting superior performance compared to state-of-the-art models. Additionally, the proposed model achieves commendable results in the binary classification task, indicating strong generalization capabilities.