How does it feel to fly in an air taxi? Exploring modern head-mounted display capabilities for mixed reality flight simulation

Abstract The acceptance of air taxi passengers is expected to be a relevant factor for the success of urban air mobility. The authors follow a user-centered design approach—which involves potential users as early as possible—to reach this goal. However, this requires a highly customizable simulation environment where different cabin arrangements and interface designs can be realized and evaluated quickly and cost-efficiently. At the same time, the experience must be immersive enough to give the users the feeling of a realistic flight. This article analyzes available mixed reality technologies and explains how these can be applied to create a simulator that fulfills these requirements. The focus is on head-mounted displays with video-see-through functionality, which allows the fusion of a customizable computer-generated world with a video stream of the real surroundings. The work includes the development of different approaches for the blending of real and virtual content as well as for the user interaction with mixed reality. The four resulting setups are then assessed in an experiment with twelve participants. Thereafter, the favored setup was improved and used for a human-in-the-loop study with 30 participants investigating passenger acceptance aspects of air taxi operations. Both studies confirm the usefulness of the mixed reality approach for the development of a future urban air mobility system. Regarding the various setups, users rated the completely virtual variant as the most immersive but favored the interaction with a physical input device over a virtual touch display. Further, the experiment emphasized the importance of precise alignment between real and virtual contents, which must be ensured by high-quality tracking systems and correct calibration of the video-see-through goggles.