Improving human performance in deepfake detection: limits of aggregation and human–AI collaboration

Abstract Given that deepfake videos are now easy to generate and increasingly high in quality, the present research investigated human detection performance and several routes to improvement. Across four experiments, participants demonstrated above-chance detection accuracy when viewing YouTube clips and their corresponding deepfakes, but performance remained modest and error-prone. In Experiment 1, aggregating responses across the full sample produced reliable improvements in accuracy, consistent with a ‘wisdom of the crowd’ effect. However, in Experiment 2, participants showed no evidence of metacognitive sensitivity. Although confidence was modestly higher for correct than incorrect decisions, it did not reliably discriminate between them at the trial level. Consistent with this finding, confidence-weighted fusion of participant pairs reduced rather than improved accuracy. In Experiments 3 and 4, we examined human–AI collaboration, with participants making decisions before and after receiving simulated algorithmic guidance. Performance improved following guidance when it was highly accurate, but this benefit was offset by a tendency to follow incorrect advice when it occurred. Notably, this reliance on simulated algorithmic guidance was unaffected by the prevalence of deepfakes within the stimulus set. Taken together, these findings suggest that, while both aggregation and human–AI collaboration can enhance performance under certain conditions, their effectiveness is constrained by poor metacognitive insight and a general tendency to align decisions with external advice. Improving deepfake detection may depend not only on advances in algorithmic performance, but also on a better understanding of how humans interpret and use both perceptual and algorithmic information.