@LM DeceptionNet: A multimodal approach for efficient transfer learning-based deception detection

In terms of deception detection, traditional contact-based techniques often require collecting physiological signals, which can negatively impact device accuracy and participant comfort. While multimodal features extracted from audio and video modalities have been shown to outperform human observers on public datasets, the generalizability of existing audio and visual-based deception detection methods in different scenarios remains insufficiently explored. To narrow this gap, this work proposes a novel domain knowledge transfer learning method for deception detection in cross-scenario applications, which enhances its generalization and adaptability. Additionally, we designed a multimodal framework that filters out irrelevant information from other modalities when a particular modality yields reliable results, further improving overall system accuracy and robustness. We evaluate the proposed method on different public datasets, achieving promising generalizability results with consistent enhancements using four variations and networks. Apart from this, the proposed @LM DeceptionNet demonstrates better generalization capacity in computational efficiency, feature extraction, and adaptability compared to a larger model when employing fewer parameters.