Multi-Modal Interaction Systems: Combining Voice, Gesture, and Touch

Multi-modal interaction systems are a new way of enhancing user experience through the integration of more than one input modality such as voice, gesture, and touch. Multi-modal interaction systems allow users to interact with applications and devices in a more intuitive, natural, and efficient manner by leveraging the strengths of each modality. Voice recognition offers access to hands-free usage, gesture recognition offers the simulation of physical input, and touch interfaces offer directness and precision. Combining these modalities into one framework offers the user an invisible and multi-purpose interaction experience that can be ported across domains, from healthcare and smart homes to automotive systems and entertainment. This paper summarizes the application of voice, gesture, and touch in multi-modal systems based on proposed applications, challenges, and benefits. We further summarize recent developments, design implications, and future of multi-modal systems in enhancing HCI.