Fine-tuned deep models for niche datasets — People detection in UAV building images to aid rescue operations

Flood victims are often trapped inside buildings or stranded on rooftops, making their detection a critical aspect of effective rescue operations. Unmanned Aerial Vehicles (UAVs) have emerged as an essential tool for disaster response, particularly in locating stranded individuals during flooding events. However, existing datasets do not specifically cater to this application, limiting the effectiveness of state-of-the-art (SOTA) object detection models in such scenarios. To bridge this gap, a novel public dataset called the NITC person rescue dataset is introduced. It consists of UAV-captured images of individuals stranded within buildings and on rooftops with two types of labels: bounding boxes and image captions. This dataset presents challenging environmental conditions that significantly impact the performance of pre-trained models. In particular, these models struggle with detecting individuals behind windows, highlighting the limitations of applying general-purpose object detection models to domain-specific tasks. This performance deterioration emphasizes the need for fine-tuning to adapt these models to specialized rescue scenarios. This work systematically evaluates various fine-tuning strategies and their effectiveness in improving model performance. The advantages of model-structure-based fine-tuning are illustrated using YOLOv9 and YOLOv11 as case studies. The best-performing model achieves an AP@50 of 87% for detecting person instances on unseen data. State-of-the-art vision language models (VLMs), BLIP2 and Qwen2-VL are fine-tuned using ground-truth image captions to obtain meaningful descriptions of the seen.