Geologically Constrained Multi-Scale Transformer for Lithology Identification Under Extreme Class Imbalance

Accurate identification of lithology is considered very important in oil and gas exploration because it has a direct impact on the evaluation and development planning of any reservoir. In complex reservoirs where extreme class imbalance occurs, as critical minority lithologies cover less than 5%, the identification accuracy is severely constrained. Recent deep learning methods include convolutional neural networks, recurrent architectures, and transformer-based models that have achieved substantial improvements over traditional machine learning approaches in identifying lithology. These methods demonstrate great performance in catching spatial patterns and sequential dependencies from well log data, and they show great recognition accuracy, up to 85–88%, in the case of a moderate imbalance scenario. However, when these methods are extended to complex reservoirs under extreme class imbalance, the following three major limitations have been identified: (1) single-scale architectures, such as CNNs or standard Transformers, cannot capture thin-layer details less than 0.5 m and regional geological trends larger than 2 m simultaneously; (2) generic imbalance handling techniques, including focal loss alone or basic SMOTE, prove to be insufficient for extreme ratios larger than 50:1; and (3) conventional Transformers lack depth-dependent attention mechanisms incorporating stratigraphic continuity principles. This paper is dedicated to proposing a geological-constrained multi-scale Transformer framework tailored for 1D well-log sequences under extreme imbalance larger than 50:1. The systematic approach addresses the extreme imbalance by deep-feature fusion and advanced class-rebalancing strategies. Accordingly, this framework integrates multi-scale convolutional feature extraction using 1 × 3, 1 × 5, 1 × 7 kernels, hierarchical attention mechanisms with depth-aware position encoding based on Walther’s Law to model long-range dependencies, and adaptive three-stage class-rebalancing through SMOTE–Tomek hybrid resampling, focal loss, and CReST self-training. The experimental validation based on 32,847 logging samples demonstrates significant improvements: overall accuracy reaches 90.3% with minority class F1 scores improving by 20–25% percentage points (argillaceous siltstone 73.5%, calcareous sandstone 68.2%, coal seams 65.8%), and G-mean of 0.804 confirming the balanced recognition. Of note, the framework maintains stable performance even when there is extreme class imbalance at a ratio of up to 100:1 with minority class F1 scores above 64%, representing a two-fold improvement over the state-of-the-art methods, where former Transformer-based approaches degrade below. This paper provides the fundamental technical development for the intelligent transformation of oil and gas exploration, with extensive application prospects.