AI ethics unwrapped: an empirical investigation of ethical principles in collaborative ideation processes

Abstract Motivated by ongoing criticism of the practical operationalization of ethical principles in artificial intelligence (AI) development, this study targets the ethical practice of AI developers in Germany. We focus on design as a key technological practice and developers as designers of AI-based systems when we investigate the socially, historically, and contextually influenced practice of AI ethics in the design process. We embed our methodology in value sensitive design (VSD), conduct design thinking workshops prototyping AI tools for hypothetical use cases, and ground our analysis on established ethical guidelines for AI. The results of this study reveal not only awareness of ethical principles in developers, more importantly, a strong influence of ethics on design decisions. Developers adapt their designs with technical interventions in favor of those using and being affected by their solutions. Our contribution is threefold: we establish a fine-grained categorization system of ethical principles based on AI ethics guidelines and VSD. We corroborate previous empirical research examining the prompted and self-reported influence and prioritization of ethical principles. Finally, we synthesize our findings with tangible design recommendations for AI ethics by design. We focus on recommendations for human involvement, privacy, and non-discrimination: encourage participatory AI design and avoid end-to-end automation in cases where humans are impacted; empower developers to integrate technical interventions from the onset of the design process to establish AI privacy by design; and support developers in emphasizing non-discriminatory AI, especially in contexts historically associated with discrimination.