Design methodology and evaluation of multimodal interaction for enhancing driving safety and experience in secondary tasks of IVIS

• The multimodal interaction design method (MIDM) for secondary tasks of driving receives better expert evaluation, improved driving performance, and higher user ratings. • A theoretical tool for MIDM is developed using a user experience map to enhance interaction in secondary tasks of driving. • Expert interviews, simulated driving, and user surveys show that the proposed method improves driving safety and interaction experience. • The proposed framework supports the application and development of multimodal interaction design in in-vehicle information systems for secondary tasks of driving. Human-computer interaction (HCI) of in-vehicle information systems (IVISs) is crucial to the safety and experience of drivers. However, the operation of secondary tasks of IVIS will cause distraction when driving. This paper aims to reduce driving distractions by multimodal interaction design, to enhance the user’s driving safety and interaction experience. Firstly, we analyze the secondary task and the associated interaction modes and develop a theory tool for multimodal interaction design modes. Then, three multimodal interaction schemes of typical secondary tasks are designed. Finally, the data of three evaluation indexes, namely expert evaluation, lane position, and user score, are obtained through expert interviews, simulated driving, and user questionnaires. The results demonstrate that the proposed multimodal interaction design of the secondary task is generally superior to the traditional voice and screen interaction, which is conducive to improving driving safety, performance, and interaction experience. The theoretical framework presented in this work provides a potential opportunity for the expansion of theoretical methods and applications of multimodal interaction design for secondary tasks.