PyraSegNet: A Novel Framework for Thermal Facial Image Segmentation

Thermal facial region segmentation is critical for applications such as physiological monitoring, stress detection, and human-computer interaction. Despite its importance, progress in this domain has been hindered by challenges such as occlusions, the lack of thermal annotated datasets and the absence of tailored, high-performing models on thermal images. In this paper, we address these challenges through four key contributions. First, we introduce the TFR dataset, a novel thermal face dataset that includes 5050 annotated thermal images extracted from 166 video sequences using 163 different subjects, which is, to our knowledge, the largest thermal dataset with comprehensive annotated regions in terms of the number of subjects, captured under diverse conditions, including variations in lighting, occlusions, and demographics, ensuring robustness and adaptability. Second, we propose PyraSegNet, an advanced deep learning architecture combining a customized encoder-decoder structure with a Feature Pyramid Network (FPN) to enhance multi-scale feature learning. Third, we compare PyraSegNet to other state-of-the-art methods. Finally, we perform an in-depth comparative analysis of loss functions, including Dice Loss, Tversky Loss, and Jaccard Loss, providing actionable guidance on optimizing thermal facial segmentation performance. Our findings underscore the significance of the TFR dataset, the effectiveness of PyraSegNet, and provide insights into optimal loss function selection for thermal facial region segmentation. This research paves the way for enhanced accuracy and reliability in applications leveraging thermal facial analysis.