Heat stress assessment in chickens via key head region temperature measurement using semantic segmentation and cross-modal RGB-IR collaboration

Surface body temperature is an important indicator for assessing heat stress in chickens, and infrared (IR) thermography is an effective method for obtaining it. However, current methods for selecting temperature measurement regions of interest (ROIs) in IR images often struggle to accurately and comprehensively reflect the actual size and shape of the measured areas, inevitably introducing noise from background or irrelevant areas during the temperature acquisition process. This study proposes a cross-modal RGB-IR collaborative segmentation framework. It utilizes the Mask2Former model to remove the background from RGB images and the key chicken head region semantic segmentation (CHSFormer) model to extract the ROIs of the comb, eye, beak, and wattle. The ROIs are then mapped to the corresponding regions in the IR images using the cross-modal coordinate mapping algorithm (CCMA), which enables accurate temperature measurement of the key chicken head regions. RGB-IR images of chickens were used in experiments to evaluate the performance of the framework. The results showed that the Mask2Former for background segmentation achieved mean intersection over union (MIoU) and mean pixel accuracy (MPA) of 99.05 % and 99.47 %, respectively. The CHSFormer for key chicken head region segmentation achieved MIoU and MPA of 93.21 % and 96.49 %, respectively. The CCMA accurately mapped the ROIs of key chicken head regions from the RGB images to the corresponding regions in the IR images. The temperature measurement results indicated that the temperatures of the comb and wattle significantly increased under high-temperature environments, which effectively reflected the heat stress in chickens.