Hybrid Convolutional Neural Network–Bidirectional Long Short-Term Memory Model with Whale Optimization Algorithm for Error Prediction in On-Machine Measurement

On-machine measurement (OMM) enables real-time dimensional feedback in production, yet accuracy is often degraded by thermal drift, sensor noise, and environmental disturbances. This motivates intelligent error-prediction methods to ensure reliable, high-precision machining. This study proposes a hybrid deep learning model integrating a Convolutional Neural Network (CNN), a Bidirectional Long Short-Term Memory (Bi-LSTM) network, and a Whale Optimization Algorithm (WOA) for precise OMM error prediction. Initially, raw measurement data underwent preprocessing to remove noise and outliers. Subsequently, we use a CNN to extract features and a Bi-LSTM to model time-dependent patterns. Finally, WOA optimizes model hyperparameters globally, further boosting predictive accuracy. Comparative experiments show that the proposed model reduces RMSE, MAE, and MAPE by approximately 53.58%, 54.96%, and 57.65%, respectively, while improving the R2 score by about 11.17% over baseline methods. Results confirm the method’s superior nonlinear prediction capabilities, significantly enhancing machining accuracy and production efficiency, and demonstrating promising industrial application potential.