Research on user experience of button controls based on multi-modal interaction

Abstract With the increasing research of multi-modal human-computer interaction technology in aircraft cockpits, the interaction process is developing in a more natural and convenient direction, but the application scenarios of multi-modal interaction are still in a state of continuous exploration. Aiming at the button controls in the human-computer interaction interface, this paper discussed the experience of using ordinary buttons and drop-down buttons in physical button interaction, touch interaction, eye movement interaction, hand-eye cooperation interaction, and gesture interaction, respectively. We conducted an experimental study on five ways of interaction, and collected 15 experts’ subjective satisfaction scores based on task completion and experience process. The results show that different interaction modes have a significant impact on the task completion time, and the task completion time is the fastest when using physical button interaction, touch interaction, and hand-eye collaborative interaction. Experts are most satisfied with physical button interaction and touch interaction in ordinary button interaction, and are most satisfied with hand-eye collaborative interaction in drop-down buttons. The scene of physical button interaction and touch interaction is more suitable for single-click tasks, and the scene of hand-eye collaborative interaction is more suitable for multi-level parameter setting tasks. The research results of this paper are helpful to the interactive function design of button controls.