A Multimodal Approach for Prediction of Plastic Color Formulation with Transformers and Convolutional Networks

Color formulation prediction aims to get different target colors based on a specified color range. It was difficult for existing color matching methods to achieve accurate nonlinear mapping from target colors to formulations due to the overlook of the global and local features of color information. Global featuress represent the long-range variation trends of the input information, while local features represent the numerical characteristics of the input data. To address this issue, a novel feature extraction block based on Transformer and parallel convolutional neural network was proposed for the first time. By integrating this block with a color feature extractor after the fusion of color features, a multimodal color feature fusion network framework was constructed to comprehensively learn the nonlinear mapping between multi-modal inputs and formulations, formulation reflectance curves. This network framework achieved a total loss of 0.0421 and a formulation loss of 0.0391 on test set, indicating more accurate color formula predictions.