Identity-Driven Multimedia Forgery Detection via Reference Assistance

{"Recent":[0],"advancements":[1],"in":[2,27,74,107,148],"\\"deepfake\\"":[3],"techniques":[4],"have":[5,43],"paved":[6],"the":[7,17,32,67,79,82,100,104,134,143,150,177,207,213,222,231,236,241],"way":[8],"for":[9,34,206],"generating":[10],"various":[11],"media":[12,22,36,70,129],"forgeries.":[13],"In":[14],"response":[15],"to":[16,124],"potential":[18],"hazards":[19],"of":[20,69,81,103,172,186,224,238],"these":[21,108],"forgeries,":[23],"many":[24],"researchers":[25],"engage":[26],"exploring":[28],"detection":[29,223,243],"methods,":[30],"increasing":[31],"demand":[33],"high-quality":[35],"forgery":[37,62,146,159],"datasets.":[38],"Despite":[39],"this,":[40],"existing":[41,111],"datasets":[42,48,112],"certain":[44],"limitations.":[45],"Firstly,":[46],"most":[47],"focus":[49],"on":[50,133,230,240],"manipulating":[51],"visual":[52],"modality":[53],"and":[54,76,145,191],"usually":[55],"lack":[56],"diversity,":[57],"as":[58,138,202],"only":[59],"a":[60,203],"few":[61],"approaches":[63],"are":[64,95],"considered.":[65],"Secondly,":[66],"quality":[68],"is":[71,84,89,247],"often":[72],"inadequate":[73],"clarity":[75],"naturalness.":[77],"Meanwhile,":[78],"size":[80],"dataset":[83,246],"also":[85],"limited.":[86],"Thirdly,":[87],"it":[88],"commonly":[90],"observed":[91],"that":[92],"real-world":[93],"forgeries":[94,109],"motivated":[96],"by":[97],"identity,":[98],"yet":[99],"identity":[101,117],"information":[102,118],"individuals":[105],"portrayed":[106],"within":[110],"remains":[113],"under-explored.":[114],"For":[115],"detection,":[116],"could":[119],"be":[120],"an":[121,156],"essential":[122],"clue":[123],"boost":[125],"performance.":[126],"Moreover,":[127],"official":[128],"concerning":[130],"relevant":[131],"identities":[132],"Internet":[135],"can":[136],"serve":[137],"prior":[139],"knowledge,":[140],"aiding":[141],"both":[142],"audience":[144],"detectors":[147],"determining":[149],"true":[151],"identity.":[152],"Therefore,":[153],"we":[154,211,234],"propose":[155,212],"identity-driven":[157],"multimedia":[158,242],"dataset,":[160,233],"IDForge,":[161],"which":[162],"contains":[163],"249,138":[164],"video":[165,181,200],"shots":[166,182,201],"sourced":[167],"from":[168,176],"324":[169],"wild":[170],"videos":[171],"54":[173,208],"celebrities":[174],"collected":[175],"Internet.":[178],"The":[179,245],"fake":[180],"involve":[183],"9":[184],"types":[185],"manipulation":[187],"across":[188],"visual,":[189],"audio,":[190],"textual":[192],"modalities.":[193],"Additionally,":[194],"IDForge":[195],"provides":[196],"extra":[197],"214,438":[198],"real":[199],"reference":[204],"set":[205],"celebrities.":[209],"Correspondingly,":[210],"Reference-assisted":[214],"Multimodal":[215],"Forgery":[216],"Detection":[217],"Network":[218],"(R-MFDN),":[219],"aiming":[220],"at":[221],"deepfake":[225],"videos.":[226],"Through":[227],"extensive":[228],"experiments":[229],"proposed":[232],"demonstrate":[235],"effectiveness":[237],"R-MFDN":[239],"task.":[244],"available":[248],"at:":[249],"https://github.com/xyyandxyy/IDForge.":[250]}

Authors

Institutions

Publication Details

Published
2024-10-26
DOI
https://doi.org/10.1145/3664647.3680622
Citations
5
Primary Topic
Digital Media Forensic Detection
Type
article
Field-Weighted Citation Impact
1.25
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
article

Identity-Driven Multimedia Forgery Detection via Reference Assistance

Yu–Gang Jiang, Jingjing Chen, Han Feng, Xue Song et al.
5 citations
Digital Media Forensic Detection
1.25
article

Identity-Driven Multimedia Forgery Detection via Reference Assistance

Yu–Gang Jiang, Jingjing Chen, Han Feng, Xue Song, Haijun Shan, Junhao Xu
article en
5 citations

Abstract

Recent advancements in "deepfake" techniques have paved the way for generating various media forgeries. In response to the potential hazards of these media forgeries, many researchers engage in exploring detection methods, increasing the demand for high-quality media forgery datasets. Despite this, existing datasets have certain limitations. Firstly, most datasets focus on manipulating visual modality and usually lack diversity, as only a few forgery approaches are considered. Secondly, the quality of media is often inadequate in clarity and naturalness. Meanwhile, the size of the dataset is also limited. Thirdly, it is commonly observed that real-world forgeries are motivated by identity, yet the identity information of the individuals portrayed in these forgeries within existing datasets remains under-explored. For detection, identity information could be an essential clue to boost performance. Moreover, official media concerning relevant identities on the Internet can serve as prior knowledge, aiding both the audience and forgery detectors in determining the true identity. Therefore, we propose an identity-driven multimedia forgery dataset, IDForge, which contains 249,138 video shots sourced from 324 wild videos of 54 celebrities collected from the Internet. The fake video shots involve 9 types of manipulation across visual, audio, and textual modalities. Additionally, IDForge provides extra 214,438 real video shots as a reference set for the 54 celebrities. Correspondingly, we propose the Reference-assisted Multimodal Forgery Detection Network (R-MFDN), aiming at the detection of deepfake videos. Through extensive experiments on the proposed dataset, we demonstrate the effectiveness of R-MFDN on the multimedia detection task. The dataset is available at: https://github.com/xyyandxyy/IDForge.

Fudan University (CN)
Openalex Percentile: Top 19%
Digital Media Forensic Detection
1.25
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.