Multimodal speech-based human-computer interaction in automotive systems with integrated service recommendation

Human-computer interaction plays a key role in intelligent automobile systems, and the current system has deficiencies in multimodal fusion, interaction accuracy, and real-time performance, which makes it difficult to meet the user experience requirements. For this reason, this paper proposes a car service recommendation method based on an improved collaborative filtering algorithm. Then, based on the recommended multimodal car service information, text and speech features are extracted using bidirectional long short-term memory (BiLSTM) and the Transformer, respectively. A low-rank multimodal integration approach is proposed to integrate the text and speech features. Modal redundancy and fault tolerance strategies are designed to enhance the system's decision-making capability. The experimental outcome demonstrates that the accuracy of human-computer interaction for the proposed method is 96.21%, and the time consumed for a single interaction is 17 ms, which is superior to the comparative methods and significantly improves driving safety and comfort.