A Toolkit for Creating Intelligent Virtual Humans in Extended Reality
This paper introduces an initial implementation and a preliminary evaluation of a toolkit for providing Intelligent Virtual Humans in Extended Reality (XR) environments. These virtual humans may serve as avatars for users, known as Smart Avatars, or artificial agents, known as Intelligent Virtual Agents. The presented toolkit consists of five modules that contribute to producing realistic human-to-human and human-to-agent interactions in XR. The primary focus lies in enabling natural, multimodal communication and interaction through a range of expressive capabilities, including speech, facial expressions, gaze tracking, and full-body animations. By enhancing the realism and responsiveness of virtual humans, this paper strives to facilitate their use in a variety of metaverse applications.
Authors
- Xenxo Alvarez
- Lucie Kruse (ORCID: https://orcid.org/0000-0002-1914-7008)
- João Orvalho (ORCID: https://orcid.org/0000-0002-9185-4479)
- Erik Wolf (ORCID: https://orcid.org/0000-0003-4881-9982)
- Joan Llobera (ORCID: https://orcid.org/0000-0002-9471-1334)
- Hannes Fassold
- Fariba Mostajeran (ORCID: https://orcid.org/0000-0002-7424-9031)
- Frank Steinicke (ORCID: https://orcid.org/0000-0001-9879-7414)
- Susanne Schmidt (ORCID: https://orcid.org/0000-0002-8162-7694)
- Adriano Tavares (ORCID: https://orcid.org/0000-0001-8316-6927)
- Sergi Fernández (ORCID: https://orcid.org/0000-0002-9138-0875)
- Sebastian Rings (ORCID: https://orcid.org/0000-0002-2020-4508)
- Michael Arz
- Caecilia Charbonnier (ORCID: https://orcid.org/0000-0002-7018-885X)
- Ke Li (ORCID: https://orcid.org/0000-0003-0828-029X)
- Nuno Santos
- Pierre Nagorny
Institutions
- Joanneum Research (AT)
- Universität Hamburg (DE)
- i2CAT (ES)
- Healing Foundation (AU)
Publication Details
- Published
- 2025-03-08
- DOI
- https://doi.org/10.1109/vrw66409.2025.00149
- Citations
- 3
- Primary Topic
- Human-Automation Interaction and Safety
- Type
- article
- Field-Weighted Citation Impact
- 7.32
Funders
- HORIZON EUROPE Framework Programme