Data Characterization and Multi-Plant Data Integration Strategies for Artificial Neural Network Modeling in Palm Oil Refining

Abstract Real-time quality monitoring in palm oil refining is often hindered by laboratory delays, which can be mitigated through the development of Artificial Neural Network (ANN) soft sensors. However, the effectiveness of these models is constrained by the characteristics of available process data. To address this, this study establishes a quantitative framework for assessing data set suitability by analyzing industrial data from three refineries using coefficient of variation (CV) statistics and self-organizing map (SOM) visualizations, with a feedforward neural network (FFN) used as a baseline model. Results show that process variables with CV exceeding 50% consistently yield better FFN performance than those below 50%. An integration strategy between different refineries was introduced to overcome the limitation of single-plant data and improve model performance to above 80% prediction accuracy for FFA content and Red Lovibond color. To test whether this improvement resulted from model capacity rather than data variability. The results further showed that data variability significantly affects model performance rather than model architecture. These findings establish CV as a practical criterion for data set suitability in ANN modeling for palm oil refining, offering an approach other chemical process industries facing similarly low-variation data may adapt.