Time-Aware Rotary Transformer for Soft Sensing of Irregularly Sampled Industrial Time Sequences

As modern industrial processes increase in integration and scale, there exist intricate dynamic time variability and nonlinearity within process data. Deep learning-based nonlinear dynamic models, such as Transformer, are frequently applied to soft sensor modeling for industrial time sequences due to their powerful ability in learning temporal feature representations. Nevertheless, process data gathered from industrial plants are usually sampled at irregular intervals, posing a challenge for most mainstream dynamic models to handle the resulting temporally changeable relations in process sequence data. Thus, this paper proposes a time-aware rotary Transformer (TART) for soft sensor modeling of irregular sampled time series in industrial processes, which adaptively and efficiently model the temporally changeable dynamics among series data. Specifically, a sampling interval embedding layer is devised to simultaneously encode the process variables, positional information and sampling intervals, facilitating the efficient extraction of temporal dynamic features. Accordingly, a non-increasing function-based time-aware rotary attention mechanism is proposed to deal with the temporally changeable sampling intervals via assigning proper weights to the corresponding feature representations. Theoretical analysis shows that the proposed time-aware rotary attention mechanism can adaptively assign temporal similarities according to the irregular sampling intervals. The TART-based soft sensor is applied to an actual-run industrial sugar crystallization process to predict the supersaturation and purity of mother liquor. In comparison with recent predictive modeling methods, the proposed TART achieves state-of-the-art performance, demonstrating its feasibility and efficacy for soft sensing of practical industrial processes.