Lazy Fusion of Multimodal Sensors for Cost-Effective Process Monitoring

Advances in sensing technologies and AI have resulted in new in-line and online process measurements based on video, vibration, chromatograms, and other high-dimensional data that can complement common process measurements such as pressure, temperature, and flow rates. These sensors can be beneficial for process monitoring; however, their continuous use is often highly expensive or even impractical. In this work, we propose a novel fusion strategy to integrate insights from these sources when needed while predominantly relying on the less expensive common measurements. A hierarchical organization of sensors based on a generalized cost metric serves as the basis for the fusion. The fusion process intelligently utilizes the least expensive data first. Costlier data are used by the fusion scheme only if found necessary in real-time to improve performance. Through this lazy fusion strategy, heterogeneous multimodal sensors can be utilized within a unified framework to improve decision timeliness, accuracy, and reliability while being robust to data delays, sensor failures, and computational limitations. The proposed fusion technique has been tested on two case studies, a simulated CSTR process and an experimental data set obtained from a multiphase flow facility. The obtained results show a significant reduction in diagnostic delay compared to traditional process monitoring while utilizing costly video and high-frequency measurements only 15–30% of the time.