Learning-based operational deflection shape extraction and magnification via latent-space spectral processing

Phase-based Video Motion Processing, a video motion magnification (VMM) method, is an image-based technique that amplifies small motion. This approach has been integrated into Operational Deflection Shapes (ODS) analyses of vibrating structures, producing accurate visualisations that are robust to noise. However, despite advances in computing power, in some cases, this method still suffers from artefacts related to amplification and long processing times when dealing with numerous and high-resolution frames. In this paper, we propose a novel approach that adapts an existing Convolutional Neural Network (CNN), originally trained for RGB-video motion magnification, to the ODS characterisation workflow, in order to inherit its advantages of faster computation, lower noise sensitivity, and reduced artefacts. While the original network architecture captures and directly amplifies motion between every frame and a fixed reference, producing a new video where the objects move or deform within a prescribed, amplified dynamic bandwidth, our technique directly extracts the animations of the ODS at the frequencies of interest. We validated our approach in a series of case studies, performing both qualitative and quantitative analyses to evaluate the clarity of the results, the presence of artefacts, the physical accuracy, and other characteristics, and conducting a parametric study on how varying the magnification factor influences image quality and the production of artefacts. Finally, we compared processing times with those of conventional pipelines, demonstrating that our learning-based method achieves a significant advantage.