Image-free single-pixel semantic segmentation for complex scene based on multi-scale U-Net

Abstract Single-pixel imaging (SPI) receives widespread attention due to its superior anti-interference capabilities, and image segmentation technology can effectively facilitate its recognition and information extraction. However, the complexity of the target scene and plenty of imaging time in SPI make it challenging to achieve high-quality and concise segmentation. In this paper, we investigate the image-free intricate scene semantic segmentation in SPI. Using “learned” illumination patterns allows for the full extraction of the object’s spatial information, thereby enabling pixel-level segmentation results through the decoding of the received measurements. Simulation and experimentation show that, in the absence of image reconstruction, the mean intersection over union (MIoU) of segmented image can reach higher than 85%, and the Dice coefficient (DICE) close to 90% even at the sampling ratio of 5%. Our approach may be favorable to applications in medical image segmentation and autonomous driving field.