Beyond Accuracy: Rethinking Outlier Detection in Asylum Data

In asylum decision-making, algorithmic tools are developed to optimize decision-making processes, for example, to streamline the application process. Accuracy, defined as correctly categorizing cases against a ground truth dataset, is a main concern. Data scientists employ outlier detection algorithms (ODA) to boost accuracy by fine-tuning datasets and discarding cases deviating from mathematical norms to prevent over-fitting. However, algorithms increasingly face scrutiny for possibly harmful effects on marginalized communities. Just as supervised machine learning is understood as a method to produce decision boundaries, ODAs are understood as methods to produce outliers – who sometimes are human beings. Organizations that use ODAs put people at risk of becoming marginalized by discarding them from a dataset, which motivates this paper. We critically engage with outlier handling as a significant juncture that molds representation within datasets. Using a dataset of asylum case summaries, we investigate the when individuals are constructed as outliers, by comparing three different ODAs and four different parameters for data modeling. This is significant because inclusion in datasets used for training automated decision tools and as ground truth, like the one under study, is crucial for ensuring transparent decisions. This study finds little overlap between the outliers constructed by the three algorithms and different data representations. This means that the outlier designation heavily depends on the choice of ODA and the parameters used in modeling the data; yet, data analysts rarely give much thought to this choice. Instead, decisions are primarily guided by technical considerations such as accuracy. Algorithms shape a specific understanding of key concepts such as text topics and norms, urging us to reconsider the design of algorithmic tools beyond merely achieving high accuracy and focusing on what makes an algorithmic decision tool useful for the particular context and the consequences for those who are constructed as outliers.