Heterogeneous Graph Neural Network for Fashion Outfit Compatibility Prediction

To solve the problem of the inflated effectiveness of current clothing compatibility prediction models, as well as the lack of consideration for the type of clothing when modeling clothing relationships and insufficient model training because of imbalanced samples, we propose a clothing compatibility prediction model based on heterogeneous graph neural networks. The model first adopts a new negative sample sampling strategy, which reconstructs the negative sample set based on the principle of using similar items for replacement to cope with the problem of the inflated model’s effectiveness. Then, a heterogeneous graph neural network is used, together with graph global pooling techniques, to model the complex relationships between different types of items within the same outfit. Finally, through hard example learning, clothing items with fewer samples have more balanced training opportunities. Experimental results on the Polyvore dataset show that the model achieves an AUC value of 0.838, with significant advantages in comprehensive performance compared to other methods.