A prediction model for the mechanical properties of SUS316 stainless steel ultrathin strip driven by multimodal data mixing

• The ability of a ML model driven by multimodal data to predict the mechanical properties of SUS316 strips is demonstrated. • A framework is presented for including both process and images info into property predictions of SUS316 strips. • The addition of CBAM can fully extract the microstructure features to improve the training accuracy of multimodal model. • The combination of multiple microstructure images of SUS316 strip has a significant impact on the accuracy of the model. Constructing a mapping relationship among material preparation process, microstructure, and mechanical properties is a challenge in material research and development. In this work, a deep learning framework for multimodal data fusion is constructed that couples a multi-layer perceptron (MLP) and a residual neural network (ResNet) to predict mechanical properties of SUS316 stainless steel ultrathin strips. Specifically, the MLP branch is used to extract the rolling process data features, and the ResNet with the addition of a convolutional block attention module (CBAM) is used to extract the microstructure features. Six models are constructed for comparison under the comprehensive consideration of factors such as unimodal network, the multimodal network and input form of image samples. The results show that the multimodal data model fused with the ResNet and MLP after adding the CBAM using both rolling process data and four types of microstructure image data as model inputs has the most accurate prediction results. The R 2 , MAPE, RMSE and MAE are 0.998, 0.727, 4.440 and 3.359, respectively. In addition, the proposed model is used for predicting yield strength and elongation, and the results show that the R 2 values of both models on the test set are greater than 0.980, fully confirming that the multimodal data model has high prediction accuracy and good generalizability.