MCBIR: Deep Learning based Framework for Efficient Content Based Image Retrieval System of Medical Images

Content-Based Image Retrieval (CBIR) in computer vision applications, enables retrieval of images reflecting user intent. Traditionally CBIR is based on image processing techniques. With the emergence of Artificial Intelligence (AI), it is now possible to realize CBIR using learning-based approaches. Particularly deep learning techniques such as Convolutional Neural Network (CNN) are efficient for image analysis. In this paper, we proposed a framework known Medical Content Based Image Retrieval System (MCBIRS), which exploits pre-trained CNN variants for retrieving medical images based on image input. The framework has an offline phase for extracting visual features from training data and an online phase for processing given user queries. The descriptors obtained by CNN variants in the offline phase are persisted in a database. These are later used in the online phase to compute the distance between persisted descriptors and input image descriptor. A set of closely matching images are returned against the query image based on similarity. We proposed an algorithm known as Learning-based Medical Image Retrieval (LbMIR) to realize MCBIRS. We also implemented a re-ranking of results retrieved by the framework using other techniques. The performance of LbMIR is evaluated and compared with the state-of-the-art methods such as Bag of Visual Words (BoVW) and Histogram of Oriented Gradients (HOG). Empirical results using medical image dataset revealed that CNN variants outperformed BoVW and HOG methods. On test data, the highest performance is achieved by the proposed system with 90% mean top-k precision, demonstrating its practical implications. On the training data highest performance is achieved by proposed system (CNN variants) re-ranked with HOG with 92.30% mean top-k precision.