CCBIR: A concept-based system for geospatial image retrieval

Content Based Image Retrieval (CBIR) systems play a crucial role in efficiently organizing large image datasets for efficient image retrieval based on visual content. In this work, we focus on Natural Disaster Management (NDM) scenarios, where rapid and accurate image retrieval can assist in monitoring disaster events and supporting effective emergency response. We present a novel Concept-Based CBIR (CCBIR) pipeline designed to enhance the CBIR process by integrating semantic image concepts derived from image features. The pipeline extracts features using a task-specific DNN backbone and applies Non-Negative Matrix Factorization (NMF) to derive semantic image concepts. For efficient image retrieval based on concept based image similarity, the FAISS library is leveraged, which enables fast approximate nearest neighbor search in high-dimensional spaces. By transforming complex, high-dimensional features into a structured and interpretable concept space, the proposed CCBIR approach facilitates a more precise alignment between computational image representations and human semantic understanding. The pipeline has been evaluated on three datasets featuring different CBIR problems in natural disaster scenarios such as forest fires and floods, comparing favorably against state-of-the-art CBIR methods. This kind of performance showcases the effectiveness and adaptability of the CCBIR system, highlighting its potential to improve disaster response and recovery operations. • A Concept-Based CBIR (CCBIR) pipeline for content-based image retrieval is proposed. • Interpretable image concepts are extracted using Non-Negative Matrix Factorization. • Our method improves retrieval accuracy and interpretability compared to state-of-the-art. • The pipeline achieves superior performance in flood and wildfire retrieval tasks.