RFEA: Reliable Forensic Evidence Aggregation for Deepfake Video Detection

Deepfake video detection can benefit from multiple forensic cues, but the resulting responses are not always equally reliable or directly comparable. Moreover, informative forgery evidence may be concentrated in only part of a video. Consequently, indiscriminate fusion can allow unreliable sources to interfere with useful evidence, amplify differences in response scales, and dilute informative temporal regions. We propose Reliability-Filtered Evidence Aggregation (RFEA), a framework for converting heterogeneous forensic responses into robust video predictions without retraining or modifying the upstream detector. RFEA restricts fusion to a retained set of evidence sources, maps their responses into a comparable relative space through rank normalization based on the empirical cumulative distribution function (ECDF), and concentrates the final decision on frames with strong forensic evidence. These operations address source instability, response scale differences, and temporal variation in evidence strength. Experiments on five public deepfake video datasets show that indiscriminate fusion reduces the average area under the receiver operating characteristic curve (AUC) at the video level from 92.05% to 85.58%, demonstrating that additional forensic evidence is not necessarily beneficial when source reliability is ignored. RFEA instead achieves average AUC and average precision (AP) values of 92.68% and 95.16%, respectively. Compared with the same base response using the same temporal aggregation strategy, RFEA improves AUC and AP by 0.63 and 1.16 percentage points. A complementary replicate-level macro paired-bootstrap analysis further supports these positive aggregate gains. These results show that effective forensic evidence aggregation depends not only on the amount of available evidence, but also on its reliability, comparability, and temporal distribution.