Self-Supervised Learning by Element Shuffling for Material Property Prediction

Materials Informatics (MI) using machine learning has been widely adopted for materials discovery. Material structures are naturally represented as graph structures, which makes Graph Neural Networks (GNNs) a prevalent method for predictions. However, Artificial Intelligence (AI) models require large volumes of high-quality labeled data, which is costly to obtain through simulations or experiments. To address this, Self-Supervised Learning (SSL) methods have been developed to utilize unlabeled structure data, allowing models to learn essential data representations using automatically generated pseudo-labels. An SSL method for materials, which involves replacing atoms with others, has successfully improved predictions for inorganic material energies. Despite its success, this method suffers from easily detectable atom replacements because elements not originally present in the structure are used for replacement, making effective SSL challenging. Building on previous work, this study introduces a novel SSL method for predicting material properties. This method is based on shuffling atoms, ensuring that the processed structure contains only elements present in the original structure. Furthermore, it is designed to prevent hint information from interfering with the learning process. The proposed method was evaluated using pre-training with SSL and semi-supervised learning. In the pre-training validation, the accuracy increase during fine-tuning was confirmed to be up to 0.366 eV compared to a state-of-the-art method. Furthermore, the effectiveness of the method for semi-supervised learning was observed, achieving approximately a 12% improvement in energy prediction accuracy compared to using only supervised training.