Lightweight Deep Learning Model Optimization for Medical Image Analysis

ABSTRACT Medical image labeling requires specialized knowledge; hence, the solution to the challenge of medical image classification lies in efficiently utilizing the few labeled samples to create a high‐performance model. Building a high‐performance model requires a complicated convolutional neural network (CNN) model with numerous parameters to be trained which makes the test quite expensive. In this paper, we propose optimizing a lightweight deep learning model with only five convolutional layers using the particle swarm optimization (PSO) algorithm to find the best number of kernel filters for each convolutional layer. For colored red, green, and blue (RGB) images acquired from different data sources, we suggest using stain separation using color deconvolution and horizontal and vertical flipping to produce new versions that can concentrate the representation of the images on structures and patterns. To mitigate the effect of training with incorrectly or uncertainly labeled images, grades of disease could have small variances, we apply a second‐pass training excluding uncertain data. With a small number of parameters and higher accuracy, the proposed lightweight deep learning model optimization (LDLMO) algorithm shows strong resilience and generalization ability compared with most recent research on four MedMNIST datasets (RetinaMNIST, BreastMNIST, DermMNIST, and OCTMNIST), Medical‐MNIST, and brain tumor MRI datasets.