Identifying Adversarial Attacks in Crowdsourcing via Dense Subgraph Detection

Crowdsourcing is becoming increasingly important for contemporary applications in machine learning and artificial intelligence. However, crowdsourcing systems may be susceptible to adversarial attacks where a subset of annotators deliberately provide erroneous responses. This paper introduces a novel algorithm for identifying adversarial attacks in crowdsourcing systems by recasting the problem as dense subgraph detection in bipartite graphs. In particular, we represent crowdsourced data as a weighted bipartite graph between workers and data points which are connected with edges whose weights are computed from annotators’ responses. The constructed bipartite graph is then analyzed with a sequential peeling algorithm to detect the dense subgraph that includes adversarial attacks. Compared to previous methods where only adversaries are detected, our proposed method can simultaneously identify adversarial annotators as well as affected data points. Preliminary results on real datasets showcase the potential of this novel approach.