Automated Scene-Adaptive Rock Fragment Recognition Based on the Enhanced Segment Anything Model and Fine-Tuning RTMDet

Abstract The particle-size distribution (PSD) of rock fragments is crucial for various engineering practices. Recently, there has been increasing attention towards new methods for analysing rock fragment PSD based on artificial intelligence and computer vision. However, most existing image-based studies on this topic are case-based, while heavily relying on manual annotation. The proposed algorithms or trained models often exhibit poor performance across different geological and lighting conditions. In this research, an automatic scene-adaptive framework for real-time rock fragment recognition (FragAdaptixAI) is introduced. First, a diverse foundation image set of rock fragments is built, and an automated annotation module based on an improved segment anything model (SAM) is employed to create the foundation rock fragment dataset, which is then utilised for training the foundation model (RTMDet). When faced with new cases, the foundation model is rapidly fine-tuned using a small amount of current case data to derive an application model, supporting real-time automation in rock fragment recognition. FragAdaptixAI was tested using two cases, and the test results demonstrate that (1) the automated annotation module, developed based on the improved SAM, is effective, and (2) FragAdaptixAI achieves excellent results with short tuning time for new cases. The approach introduces a new concept to address the generalisation issues in rock fragment recognition by combining the zero-shot capability of large models with the real-time processing of small models.