Reflective Design for Informal Participatory Algorithm Auditing: A Case Study with Emotion AI

This paper suggests how reflective design can aid informal participatory algorithm auditing. Drawing from reflective design, we designed a simple web-form probe to invite critical reflection on Emotion AI, ethically controversial techniques predicting individuals’ emotions. Participants engaged the probe throughout their daily lives for about a week. Then, we interviewed participants about their experiences and reflections. Our findings surface themes around participants’ (i) critiques of Emotion AI, (ii) factors contributing to inaccuracy, and (iii) patterns of miscategorization. Our discussion contributes (1) recommendations for Emotion AI and (2) how reflective design may offer considerations to inform algorithm auditing. Overall, our paper suggests ways critically-oriented design research can engage AI ethics through informal, participatory, exploratory algorithm auditing.