A Soft-Contrastive Pseudo Learning Approach Toward Open-World Forged Speech Attribution

Anti-spoofing of deepfake or forged speech is an important technique for the security usage of generative artificial intelligence. Beyond binary classification of real and forged speech, method attribution of forged speech is becoming a practical solution of interpretable anti-spoofing strategies. However, existing related methods have poor performance on analyzing speech forgery methods unseen in their training data, which is inefficient in open-world scenarios with emerging new forgery methods. In this paper, Open-World Forged Speech Attribution (OW-FSA) is firstly defined towards the attribution of forged speech on the methods generating it, where the recognized methods are not limited to the seen ones in training data and the properties of the unseen methods should also be depicted adequately. A novel algorithm, Soft-contrastive Pseudo Learning (SPL), is proposed to address the challenges outlined in OW-FSA, which introduces two key innovations: 1) Based on similarities between features at different scales, the proposed similarity-based soft filtering module filters and matches utterances from the same forgery class to enhance the intra-class compactness of features through contrastive learning. 2) The proposed similarity-based soft pseudo-labeling module integrates label-smoothing-like and similarity weighting techniques to mitigate possible errors in pseudo-labeling. Besides, an iterative algorithm based on SPL is proposed to predict the number of unseen classes. Extensive experiments have validated the superiority of the proposed algorithm over other recently proposed methods on the task of OW-FSA with or without the knowledge of the number of unseen classes. Intuitive visualization and ablation studies have also been conducted to illustrate the advantages of the proposed algorithm. The newly defined task OW-FSA and the proposed algorithm SPL in this paper will help advance the research in speech anti-spoofing.