“Everybody knows what a pothole is”: representations of work and intelligence in AI practice and governance

Abstract In this paper, we empirically and conceptually examine how distributed human–machine networks of labour comprise a form of underlying intelligence within Artificial Intelligence (AI), considering the implications of this for Responsible Artificial Intelligence (R-AI) innovation. R-AI aims to guide AI research, development and deployment in line with certain normative principles, for example fairness, privacy, and explainability; notions implicitly shaped by comparisons of AI with individualised notions of human intelligence. However, as critical scholarship on AI demonstrates, this is a limited framing of the nature of intelligence, both of humans and AI. Furthermore, it dismisses the skills and labour central to developing AI systems, involving a distributed network of human-directed practices and reasoning. We argue that inequities in the agency and recognition of different types of practitioners across these networks of AI development have implications beyond RAI, with narrow framings concealing considerations which are important within broader discussions of AI intelligence. Drawing from interactive workshops conducted with AI practitioners, we explore practices of data acquisition, cleaning, and annotation, as the point where practitioners interface with domain experts and data annotators. Despite forming a crucial part of AI design and development, this type of data work is frequently framed as a tedious, unskilled, and low-value process. In exploring these practices, we examine the political role of the epistemic framings that underpin AI development and how these framings can shape understandings of distributed intelligence, labour practices, and annotators’ agency within data structures. Finally, we reflect on the implications of our findings for developing more participatory and equitable approaches to machine learning applications in the service of R-AI.