EAPEM: An edge-aware-prototype-based efficient MaskFormer for identifying rock mass structure of cantilever roadheader tunnel face

• Edge-aware and prototype-based segmentation model designed for rock mass structure identification in cantilever roadheader tunnel faces. • Multi-scale pixel decoder, masked cross-attention, and edge-aware module integrated to enhance boundary localization and semantic consistency. • 93.54% accuracy and 85.23% boundary IoU achieved in karst cave segmentation using only 52 annotated images. • Lightweight framework enables small-sample learning and real-time deployment in complex tunneling environments. The rapid and accurate identification of rock mass structures in tunnel face is a fundamental prerequisite for optimizing dynamic excavation strategies and ensuring safety decision-making in cantilever roadheader construction. To address the poor performance of existing image segmentation methods in handling fuzzy boundaries, texture interference, and data scarcity, this study proposes a novel semantic segmentation framework named edge-aware-prototype-based efficient MaskFormer (EAPEM). Built upon the MaskFormer architecture, EAPEM integrates an efficient multi-scale pixel decoder, a prototype-based masked cross-attention, and an edge-aware module to achieve collaborative optimization from global semantic parsing to local boundary refinement of tunnel face structures. Experimental results demonstrate that the proposed model achieves a mean pixel accuracy of 84.56% and a mean intersection over union (mIoU) of 75.58% with only a small number of high-quality training images, significantly outperforming existing segmentation networks. Specifically, the model achieves 93.54% accuracy and over 85% boundary IoU in karst cave recognition tasks, showing its significant advantage in handling complex boundary structures. The proposed EAPEM framework offers a lightweight, efficient, and edge-aware segmentation solution for rock structure identification under complex construction scenarios.