Time series data cleaning based on sliding window and linear fitting method for vibrating wire strain gauges

Abstract Vibrating wire strain gauges (VWSG) are extensively utilized in civil engineering, yet conventional data cleaning methods inadequately address trend-associated anomalies such as bias and gain. To address these challenges, this study proposes a novel data cleaning method based on particle swarm optimization enhanced sliding window and linear regression (PSO-SW-LR) feature extraction. First, the overall data is divided into segments using a SW, with LR applied to each segment. Next, slope and intercept coefficients are extracted to generate slope and intercept streams, with slope streams reflecting trend information for each segment. Then, by eliminating abnormal slope values and corresponding intercept values, abnormal data with trend change characteristics can be effectively removed. Finally, regression calculations and the median method are applied to reconstruct the data. Validated on real VWSG monitoring data, SW-LR demonstrates superior performance over eight existing techniques, achieving an 88% root mean square error reduction and 79% mean absolute error improvement in composite anomaly scenarios. The proposed approach offers a robust and efficient solution for cleaning long-term time series data.