Elastic Weight Consolidation and Transfer Learning: A New Paradigm for Deepfake Detection

The rampant proliferation and escalating sophistication of deepfakes underscore the critical need for effective detection mechanisms. Addressing the persistent challenge of generalization in deepfake detection, this work presents a continual learning strategy, fostering model adaptability across evolving datasets while safeguarding knowledge from prior tasks. Central to our methodology is the utilization of Elastic Weight Consolidation (EWC) and Layer-Wise Scaling (LWS) of learning rates. By calculating the Fisher Information Matrix (FIM), our bespoke EWC implementation adeptly counters the catastrophic forgetting dilemma. It penalizes drastic alterations in pivotal weights during incremental task training, enshrining legacy know ledge. An interesting facet of this work is the combination of EWC with transfer learning concepts. By leveraging the pre-trained Xception network, the model is imbued with a baseline of knowledge and adaptively fine-tuned for deepfake detection.