Dynamic Fine-Tuning Rotation Network for Semantic Segmentation of Rock Paintings

The scale features of rock art exhibit significant diversity and graduality. Among the existing semantic segmentation methods for rock art, although some models have taken note of the scale differences in rock art patterns and the complexity of directional features, and proposed targeted improvement strategies, most of these methods view scale adaptation and directional representation as unconnected problems. They fail to model the intrinsic correlation between the scale adaptation and directional representation, and particularly overlook the restrictive effect of scale accuracy on the extraction of directional features. This ultimately leads to the problem of "spatial representation misalignment" in the semantic segmentation of rock art. To address the above problems, this paper proposes a Dynamic Fine-tuning Rotation Network (DFTR-Net), which aims to solve the problems of imprecise scale feature extraction and directional misalignment for rock art patterns with arbitrary orientations. The network consists of a dynamic selective convolution structure and a shape-aware spatial feature extraction module. Specifically, the dynamic selective convolution dynamically adjusts the coverage range of the receptive field through inter-layer feature aggregation. It uses stacked small dilated convolution kernels to replace large convolution kernels with the same receptive field for extracting the neighborhood details of patterns. Then, by combining with feature aggregation, it constructs spatial feature differences and realizes intra-layer dynamic weighted fusion, thereby achieving accurate scale feature extraction. After obtaining fine-grained scale features, the shape-aware module first corrects the initial segmentation candidate regions of the patterns to generate directional guide boxes. Subsequently, it drives the rotational sampling of convolution kernels based on the angles of the guide boxes, forming region-constrained deformable convolutions that adapt to the shape of the patterns. These convolution kernels obtain strong supervision based on pixel-level annotations, which enhances the sensitivity to the directional features of the patterns and effectively alleviates the problem of directional misalignment. Extensive experiments show that DFTR-Net can achieve higher performance on the 3D-pitoti and Petroglyph Annotation datasets compared with the existing methods.