Transformer-based multimodal learning for predicting mechanical properties in heat-treated stainless steel

• Innovative Multimodal Learning Approach: Introduces a novel multimodal learning framework utilizing a Transformer-based model to predict the mechanical properties of vacuum carburized martensitic stainless steel. • Integration of Diverse Data Sources: Successfully combines microstructure images, material composition, and process parameters to construct a comprehensive model for predicting hardness and wear resistance. • Enhanced Prediction Accuracy: Achieves significant improvements in prediction accuracy, with an R 2 value of 0.98 and a mean absolute error (MAE) of 5.23 HV for hardness prediction. • Application of Variational Mode Decomposition (VMD) : Implements VMD to process wear curves, effectively reducing noise and enhancing the accuracy of wear performance predictions. • Potential for Broader Applications: Demonstrates the potential of multimodal learning in materials science, offering a robust framework for future advancements in predictive modeling for material design and processing optimization. Accurately predicting mechanical properties of heat-treated materials is critical for intelligent process control and advanced manufacturing. This study proposes a Transformer-based multimodal learning framework for predicting the hardness and wear behavior of carburized steel after vacuum carburizing. By integrating microstructural images, material compositions, and process parameters, the proposed model effectively captures complex cross-modal relationships. Experimental results show that the multimodal model achieves high prediction accuracy, with an R 2 of 0.98 and MAE of 5.23 HV for hardness prediction. Furthermore, Variational Mode Decomposition (VMD) is introduced to preprocess the wear curve, reducing noise and improving the robustness of friction performance prediction. The results demonstrate the effectiveness and generalizability of the proposed approach, offering a practical AI-based solution for intelligent material property evaluation and process optimization.