Multi-target classification and localization using thermal camera and deep learning

Reliable detection and localization of small objects such as unmanned aerial vehicles, birds, and other aerial vehicles is highly challenging. Further, recent studies have mostly focused on utilizing RGB cameras for the detection and classification of these objects. Nevertheless, extreme weather conditions like fog, snow, and dust complicate RGB-based localization and identification. To overcome these challenges, in this work, a thermal imaging-based target detection and localization method is proposed that performs well even in poor lighting conditions. Initially, we created a dataset of thermal images corresponding to humans, cars, and drones. Thereafter, the YOLOv5 model is utilized by the proposed method for the classification and localization of these targets. In contrast to the state-of-the-art, the proposed method localizes the target based on the angle at which it is situated within the field of view of the camera. Through numerical results, the performance of the proposed method is evaluated for target detection. Further, the k-fold cross-validation results are presented for obtaining the angle at which the target is located. Finally, root mean square error is used to find the error between the true angle and the predicted angle of the target using the proposed method.