LSTS: Periodicity Learning via Long Short-Term Temporal Shift for Remote Physiological Measurement

Remote physiological measurement from facial videos, exemplified by the remote photoplethysmograph (rPPG) technology, has attracted considerable attention for its potential in many applications. While recent advances in remote physiological measurement have achieved great success, what is often overlooked in previous studies is the periodic nature of physiological signals. In this study, we present long short-term temporal shift (LSTS), a novel neural network designed to effectively model the periodicity in physiological signals. We propose the periodic channel shift (PCS) mechanism to represent the periodic nature of physiological signals by selectively shifting channels between frames in adjacent periods. Additionally, to help models focus more on inter-period variations in the videos, we also propose TSAug, a shift-based data augmentation technique to suppress intra-period variations. Furthermore, we propose a simple input preprocessing scheme through color space transformation, termed multi-scale plane-orthogonal-to-skin (MPOS), to better capture the rPPG clues in videos. Extensive experiments show that the proposed LSTS model not only achieves superior or on par state-of-the-art performance on four benchmark datasets, but also exhibits outstanding generalizability across people of different races and skin tones, making LSTS an inclusive model that can benefit a wide range of users. The code will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Promisery/LSTS</uri>.