Fusion of Convolution Neural Network and Visual Transformer for Lithology Identification Using Tunnel Face Images

This study proposes an intelligent method for recognizing the lithology of a tunnel working face by combining a convolutional neural network and visual transformer. First, an efficient method for collecting high-resolution images of the tunnel working face after construction blasting is developed. Based on relevant geological data, the lithology labels of the tunnel face images are manually prepared. A data augmentation technique is then applied to expand the number of original image samples. Given the established sets of tunnel face images and corresponding lithology labels, the performances of ResNet18 and VIT-4 (which contains four transformer encoding layers) developed in this paper in identifying lithology is compared and analyzed. Subsequently, the efficiencies of using ResNet18 and VIT-4 in both parallel and successive manners is evaluated. The experimental results show that the accuracies of ResNet18 and VIT-4 are 95.7% and 95.4%, respectively. However, stacking ResNet18 and VIT-4 in a parallel manner achieves significantly improved performance in lithology recognition, with an accuracy rate of 98.3%. In contrast, the performance achieved from combining ResNet18 and VIT-4 in a serial manner depends on their structures. Achieving optimal classification performance hinges on minimizing the number of convolution blocks in ResNet18 and concatenating appropriate transformer blocks. The highest accuracy achieved by the method for deploying ResNet18 and VIT-4 in a serial manner using the optimal network structure is 98.5%.