Mixed-Cultural Intelligent Virtual Agents

This cumulative dissertation presents interdisciplinary research in the field of human-computer interaction that is located in both, computer science and psychology. It investigates the implementation and subsequent evaluation of Intelligent Virtual Agents (human-like, interactive characters) with mixed-cultural backgrounds (multiple cultural backgrounds combined in one individual, e.g., as a result of migration). On the one hand, methods from computer science were applied to develop two novel tools that allow for the automatic generation of synthetic non-native speech and for the efficient implementation of Wizard-of-Oz-based interactive scenarios with mixed-cultural Intelligent Virtual Agents (IVAs). On the other hand, methods from psychology were applied to investigate how users perceive cultural behaviours in IVAs and whether their response is congruent to the one seen in human-human interactions. Additionally, to use IVAs in cultural interventions, effective intervention strategies from social psychology were adapted for interactive scenarios with IVAs. The research contribution is divided into two levels: On the first level, basic research on the design and subsequent perception of mixed-cultural IVAs is conducted. With non-native speech being the most reliable cultural marker of mixed-cultural individuals, this dissertation presents work on the design and perception of synthetically generated non-native speech for IVAs. Focusing on the two most salient markers of non-native speech, grammatical mistakes and non-native accents, it evaluates how users perceive IVAs exhibiting these patterns of non-native speech. Furthermore it investigates in which cases non-native accented synthetically generated speech can be used as a cost-efficient and flexible alternative to non-native accented natural speech. To simplify and standardise basic research on the perception of mixed-cultural IVAs with non-native speech, this thesis presents a novel tool for automatically generating mixed-cultural speech for IVAs. The tool enables fast and reliable generation of grammatically incorrect and non-native accented synthetic speech based on empirical research. On the second level, this dissertation presents novel applied research that for the first time implements virtual interactive interventions using mixed-cultural IVAs to raise inter-cultural empathy and reduce implicit cultural bias. Based on previous research from the social sciences and human-computer interaction, this dissertation also presents the implementation of a Wizard-of-Oz prototyping tool for fast and reliable implementations of complex interactive scenarios with mixed-cultural IVAs. This dissertation makes a significant contribution to research on the implementation and perception of mixed-cultural IVAs and their application to reduce implicit cultural bias. Presenting the two newly developed tools to automatically generate non-native speech and design interactive scenarios with convincing mixed-cultural IVAs, it also makes an important technical contribution for agent-based research.