An intelligent method for automatic detection of face substitution in an image

Deepfake images have become a major problem in today’s digital landscape. Such images are usually created using advanced machine learning techniques. These fake images can deceive viewers, posing risks to privacy, security, and trust. In this paper we introduce an innovative approach to generating a target feature set for identifying deepfakes and consequently to detect a deepfake, generating a dataset of labeled substitution face images based on a public dataset using a face substitution generator. Facial keypoints are calculated and analyzed in the volumetric format for the images in the dataset. Besides, the corresponding features identified by an expert method are calculated and several classifier models based on three feature sets including facial keypoints, expert features, and their mixture are trained. Also, the performance of the classifiers is evaluated , confusion matrices and tracking patterns depending on the target feature set are constructed chieving an accuracy of 0.682 and an F1-measure of 0.680 on the combined feature set. Using the coefficients of the obtained models, the importance of features was assessed and the most significant of them were identified for a more in-depth approach to forming data sets when training machine-learning models. For the calculated expert features, the most important were facial features characterizing its geometric shape, and in the case of key points of the face, the most significant areas were the boundaries of facial objects: the contour of the face, the areas around the eyes and the corners of the lips.