Buyer–Seller-Deception-Game Dataset: A new comprehensive dataset for facial expression based deception detection in economic contexts

Automated deception detection using only video and audio modalities remains a challenging problem in computer vision. Deceptive behavior is highly context dependent, shaped by factors such as the specific scenario, cultural background, and the associated stakes. To explore the potential of future deception detection tools in online interactions, such as virtual sales meetings, we present a new high-quality low-stakes dataset specifically tailored for this task. It currently includes 500 annotated video samples, with a planned extension to around 1000 before public release. Participants engage in incentivized online interactions, in which sellers attempt to persuade buyers to choose a specific card, creating naturally motivated deceptive and truthful behavior. We evaluate a variety of visual and audio based feature sets, such as gaze, head pose, facial Action-Units (AUs), and prosodic features, on our dataset as well as on a high-stakes in-the-wild deception dataset. Our results show that OpenFace based AU features perform best on our clean and controlled recordings, while CNN based AU predictors outperform others in the more challenging dataset with lower video quality and unstable head pose. Multimodal approaches slightly outperform the best unimodal features in both cases. We will make the dataset freely available to support future research in automated deception detection.