Crowd-Sourcing or AI-Sourcing? The Impact of GenAI on Data Annotation Tasks

Generative AI (GenAI) and its capabilities have led people of various professions to consider how expendable their jobs are. Meanwhile, others see an opportunity to maximize productivity, using GenAI to reduce their time and effort. Crowd workers are no exception; experimental evidence and workers’ testimonies concur that GenAI tools, especially ChatGPT, are being used in the completion of tasks. With this in mind, one is left to wonder if this is the end of “true” human annotations, and if so, if it even matters. We predict that the future of data annotation will be characterized by a spectrum of human-AI collaboration modes, ranging from tasks performed only by GenAI, to those requiring human-AI collaboration, to those that will be more efficient if they remain human-driven.We also foresee that a potential negative effect of the uncritical adoption of GenAI for annotation, will be the decreased diversity in resulting labels, which may not represent the views of all.