Slicing the past to predict the future: Recasting data slicing as curatorial work in ML development and evaluation

Abstract Data slicing is an inherent practice in machine learning (ML) evaluation, where different subsets of a dataset are used for training and evaluating ML systems. Drawing on ethnographic fieldwork conducted between September 2023 and February 2024 among the data scientists who develop ML-driven recommender systems for the British Broadcasting Corporation (BBC), this reflection piece highlights the important, yet often overlooked, ML practice of data slicing. Building on archival influences in Critical Dataset Studies and scholarship on critical curation, the paper proposes recasting data slicing as a curatorial practice. This shift makes visible the often highly tacit slicing practices undertaken by data scientists when working with ML datasets. Specifically, the paper traces three central considerations regarding data slicing: replicability, representativeness and generalisability. Using these as examples, the paper reflects on the implications of the selection, organisation, and choices of how to best represent the past to predict the future preferences of audiences. By engaging with data slices as curatorial constructs, we can better understand and intervene in their material politics by making visible how their orderings convey certain meanings more effectively than others. Through this approach, the paper expands on existing work within Critical Dataset Studies by identifying the political role of data slicing in the evaluation of ML systems.