Research on a DeepSeek-Based Multimodal Human-Computer Interaction System

The existing general multimodal large models based on DeepSeek have severe limitations when directly applied to in-vehicle environments, such as fragmented static input, long-context forgetting, absence of vertical-domain knowledge, and failure to comprehend the implied intentions of the drivers. This paper will present a multimodal contextual reasoning human-computer interaction paradigm for intelligent vehicle cockpits. Four main directions exist in order to adapt DeepSeek-VL to specific scenarios, namely, temporal context modeling, dual-layer long-term memory expansion, RAG retrieval augmentation with an automotive vertical knowledge base, and joint environmental-driver causative contextual reasoning. They are tested on four publicly available datasets UTA-RLDD, DMD, InCarEmo, and CMU-MOSEI datasets as well as a self-built dataset, with native DeepSeek-VL, LLaVA-1.5, and Qwen-VL serving as the baseline. The findings indicate that the proposed scheme attains an implicit intention recognition rate of 86.4%, which is 34.1 percentage points higher than that of the native model; a driving risk recall of 91.1; and a decrease in the hallucination rate of 23.5 percent to 4.6 percent. The ablation experiments prove the independent contribution of each module quantitatively and the subjective evaluation of users shows major improvements in all aspects.